ZDC25: Mark Woods, Splunk - Walking the Tightrope: Navigating Risk, Impact and Change

### Overview In this talk, Mark Woods, Chief Technical Adviser from Splunk, explores the challenges of defining and measuring resilience across complex organizational systems. Reflecting on his past experience leading an engineering team through a major core system upgrade, Woods highlights the common dilemma of delivering projects on time and within tolerance while still lacking a clear understanding of actual service resilience. The presentation focuses on bridging the gap between low-level technical operations and high-level business leadership by grounding resilience in service context, capacity, and impact.

### Key Topics - **Defining Resilience**: Resilience cannot be treated as a simple tick-box exercise or merely surviving a crisis. Using a Formula 1 analogy, Woods explains that resilience is not just about passing pre-race inspections or crossing the finish line, but how well a system performs and endures throughout an entire dynamic operating environment. Broadly, resilience comprises both recovery (the ability to respond when something happens) and absorption (the capacity to absorb changes or disruptions without needing an active operational response). - **Bridging 'Board to Keyboard'**: Technical teams typically evaluate significance and health bottom-up from component-level or system-level metrics, while leadership and boards focus on business opportunities, change capacity, and top-level risk profiles. Woods argues for aligning these viewpoints through shared operational health and service context rather than relying on raw incident counts. - **Ambidextrous Delivery and Agility**: Organizations must balance risk mitigation (building robustness, absorption, and recovery) with opportunity exploitation (moving quickly, launching new features, and driving innovation). Over-indexing on pure resilience limits change, while moving fast without resilience creates excessive failure and maintenance toil. - **The Insight Ladder and Shared Context**: Operational teams and leadership often interpret data differently depending on whether they are focusing on day-to-day execution or macro-level business challenges. Woods notes that there is "no one ring to rule them all" in terms of tooling or metrics; instead, alignment comes from shared context around critical services and understanding end-user impact.

### Technical Details & Operational Measurement - **Right Capacity vs. Total Visibility**: Woods cautions against the trap of chasing end-to-end, full-fidelity visibility across all possible component interactions, noting that it assumes infinite budgets and team capacity. Instead, organizations must identify the right capacity tailored to prioritized critical services. - **Service Context & Impact**: Rather than viewing resilience strictly within functional silos (e.g., pure network, compute, or infrastructure levels), resilience must be measured at the service and product level. Risk and performance metrics should reflect the direct impact on internal and external end users rather than abstract system counts. - **Security's Role**: Security teams possess a unique advantage because their workflows inherently rely on data, detections, context, and response. By extending this contextual data approach to services and business products, security can actively enable broader organizational resilience.

### Takeaways - Resilience requires balancing the right capacity for absorption and recovery against the organization's appetite for change and agility. - Raw operational counts (e.g., reducing an arbitrary metric from 1,000 to 900) do not automatically translate to increased resilience unless tied to service impact. - Measuring operational resilience around prioritized critical services provides a common language between technical operators and executive leadership.