A case for threat informed penetration testing | Dan DeCloss | WWHF Deadwood 2022

Dan DeCloss presents a business case for integrating threat intelligence with penetration testing programs to shift organizations from reactive to proactive security postures. He begins by establishing that traditional penetration testing and threat intelligence functions, while valuable, operate largely independently and sub-optimally.

Traditional Limitations: Standard penetration testing engagements typically occur infrequently (often annually), are broadly scoped, and apply baseline methodologies uniformly across engagements. While comprehensive, these generate voluminous findings and multiple attack paths, overwhelming blue teams with prioritization decisions. Reports can feel overwhelming, leaving defenders unsure where to begin remediation. Meanwhile, threat intelligence remains largely reactive—teams ingest threat feeds, extract indicators of compromise (IOCs), and push them into detection systems (SIEMs, firewalls) to identify known threats post-discovery. This keeps organizations in a responsive posture. Additionally, collaboration between red and blue teams is typically limited, as consulting firms move to next engagements or internal teams work in isolation.

The Core Problem: DeCloss frames the mission as making organizations and the world safer, identifying security gaps, and demonstrating measurable improvement. Traditional approaches leave teams fatigued by alerts, struggling to prioritize, and unable to articulate security progress to executives.

Threat-Informed Approach: The solution involves merging threat intelligence with continuous, iterative penetration testing. Rather than relying solely on annual assessments, organizations break testing into small, recurring cycles—for example, every two weeks. This approach applies threat intelligence to scope pen testing engagements, identifies actual threats relevant to the organization, and emphasizes context: understanding how a specific threat would execute in that environment, considering business model, compensating controls, and industry factors. Collaboration between red teams, blue teams, and threat intelligence teams (purple teaming) ensures all disciplines inform test plans and interpret results together. Agreed metrics measure progress over time, demonstrating trends in detection and defense capabilities.

Continuous Testing Model: Rather than eliminating annual assessments, the approach layers continuous testing atop them. When urgent intelligence emerges (e.g., Uber MFA fatigue breach), teams rapidly develop test plans. Otherwise, threat intelligence feeds into a backlog of known threats to iteratively test. This produces what DeCloss terms "fight while we train while we fight while we train"—constant readiness through ongoing validation.

Case Study: An anonymous large organization conducts weekly purple team exercises incorporating threat intelligence feeds. When the Uber breach surfaced, they quickly developed a test plan, validated controls with the blue team, identified gaps, and remediated—then communicated concrete findings to their board, replacing vague assurances with demonstrable actions. This improved board confidence and team morale.

Context as Critical: DeCloss emphasizes that applying threat context is paramount. Threat intelligence on specific threat actors or ransomware variants means nothing unless teams ask: How would this threat execute in our environment? Do these TTPs apply to us? What would success look like for an attacker here? This mirrors vulnerability management—a severity rating means nothing without understanding asset importance and compensating controls.

Measurement and Culture: By measuring outcomes against specific threats, teams show demonstrable progress, combat alert fatigue, and justify investments in tooling or personnel. Research shows purple teams adopting this approach report better morale, clearer understanding of true gaps, and stronger business cases for resources.

Key Takeaways: Adopt a programmatic approach with continuous, iterative testing and metrics. Integrate threat intelligence to drive test scope, not just reactive alerting. Build purple teams merging red, blue, and threat intelligence functions. Apply threat context to understand relevance before testing. Maintain annual/broader assessments for compliance and other attack paths. Measure and celebrate wins to demonstrate value and prevent fatigue. Move the majority of effort from reactive firefighting to proactive validation of known threats.