2016 Day 2 Chris Horsley, Cosive Maturing your security team: haste makes waste

## Overview

Chris Horsley from Cosive presents a cautionary talk against rushing into advanced security techniques without foundational incident response capabilities. He argues that conference bias toward novel detection methods overshadows critical fundamentals, and he's observed systemic issues in how organizations conduct incident response. The presentation centers on a maturity model helping teams progress from no capability to advanced operations.

## Key Topics

**Four Emerging Technologies:** Hunting (manual or machine-assisted incident detection), threat intelligence (understanding attackers' operations and tools), big data analysis, and machine learning. While discussed extensively at conferences, these require solid foundations to be effective.

**Incident Response Fundamentals:** Horsley emphasizes that defenders must expect breaches—attackers need to succeed once; defenders must be perfect always. Therefore, organizations must have mature incident response capabilities before investing in advanced analytics.

## Technical Details

Horsley distinguishes between threat data (raw, unvetted IP/domain information), threat information (vetted with confidence measures), and true threat intelligence (understanding actor behavior and campaigns). He presents incident response cycles feeding into threat intelligence cycles through post-incident reviews—often the most neglected phase.

**The Five-Level Maturity Model:**

- **Level -1 (No Capability):** No incident response planning; organization hopes incidents won't happen. - **Level 0 (Ad Hoc):** Passionate but part-time responders; no budget or mandate; thinking about IR begins. - **Level 1 (Building):** Define team formally; establish ethics guidelines; classify incidents; gain management buy-in; define constituency; track incidents in a system; ensure responder availability; document procedures; centralize logs; negotiate internal data access; begin consuming external feeds. - **Level 2 (Refinement):** Team mandate with executive support; 24/7 coverage; media strategies; incident drills; centralized data storage (SIEM/ELK); isolated/non-attributable infrastructure for threat research; incident playbooks; multiple analyst tiers; proper software development practices for automation; formal escalation; business continuity planning; threat feed normalization. - **Level 3 (Advanced):** Delegate via tools and procedures; publish statistics; threat intelligence platforms; emerging analytical methods; full readiness for advanced techniques.

**Key Implementation Details:** - Data centralization requires diplomatically negotiating access from system administrators who hold logs (DNS, proxy, email)—address fears and control issues in peacetime. - Automation requires dedicated software development resources within the IR team; monitor that automation fatigue (operators lose situational awareness about when to pay attention) doesn't undermine effectiveness. - Post-incident review extracts indicators and feeds them back into threat intelligence processes, but this phase is often sacrificed under incident volume. - Incident playbooks should specify detection queries, business impact, and affected systems so any responder can take over.

## Takeaways

1. **Maturity First:** Organizations cannot effectively implement hunting, threat intelligence, or machine learning without foundational IR capabilities, data centralization, and documented processes. 2. **Management Alignment Required:** Technical teams need executive sponsors to secure budget, authority, and business participation in incident escalation decisions. 3. **Data Accessibility is Foundational:** Before advanced analytics, organizations must negotiate and centralize logs across systems, normalize external threat feeds, and integrate platforms via APIs. 4. **Neglected Post-Incident Review:** Most organizations abandon incident analysis once containment ends; this is where threat intelligence originates and where improvement metrics derive. 5. **Automation Has Hidden Costs:** Proper software development methodology, monitoring of automated processes, and resource allocation for script maintenance are essential; automation alone doesn't solve fatigue. 6. **Business Context Matters:** IR teams biased toward pure technology solutions miss practical business constraints and decision-making requirements that senior management must provide. 7. **Haste Makes Waste:** Rushing to adopt cutting-edge techniques without maturity wastes budget and effort; identify your current level and address gaps before climbing.