Breaking Brains, Solving Problems: Lessons Learned from 2 Years of Setting puzzles for InfoSec Pros

Breaking Brains, Solving Problems: Lessons Learned from Two Years of Setting Puzzles for InfoSec Professionals

**Overview**

Matt Wickie, who leads cyber security research for PwC UK's security practice while pursuing a part-time PhD at University College London, presents research on problem-solving cognition as applied to infosecurity. Drawing from two years of running an internal puzzle program at PwC, Wickie demonstrates that problem-solving ability is not an innate trait but a trainable skill with measurable components. The talk synthesizes cognitive psychology research on how the brain solves problems, how expertise creates both advantages and blind spots, and how cognitive biases impact security decision-making and vulnerability analysis. Rather than treating problem-solving as a black-box skill, Wickie deconstructs the cognitive processes: understanding problem structure, mental model formation, search strategies, and the moment of insight when a problem "clicks." The research shows that security professionals benefit from systematic puzzle training—not as games, but as deliberate practice in pattern recognition, assumption detection, and approach diversity. By understanding how expertise can trap analysts in schema-based assumptions, organizations can design better security training, testing programs, and incident response procedures that explicitly counter predictable cognitive failures.

**Technical Details**

Cognitive Processes Underlying Problem Solving:

1. **Brain Regions and Mechanisms**: - Primary regions: Prefrontal cortex; multiple regions activate based on problem type - "Aha moment": Specific brain regions activate when solution insight occurs - Mental model formation: Brain creates abstract internal representations of problems - Exact mechanisms not fully understood; problem-solving cognition remains partially mysterious

2. **Two-Phase Problem-Solving Process**: - **Understanding Phase**: Assimilating the puzzle stimulus, comprehending the problem statement, forming mental structures representing the problem state, mapping relationships between elements - **Searching Phase**: Finding or calculating a solution following the understanding phase; executing operators to change state; testing new states against goal state - Process is often circular, not strictly linear; may iterate between understanding and searching phases

3. **Problem Space Model**: - Initial problem state: Starting configuration - Operators: Transformations that change state; may be explicitly given or must be deduced/induced - Goal state: Desired end state to reach - Solution search: Apply operators, test if new state matches goal state, iterate until solution found

4. **Problem-Solving Strategies** (Explicit Approaches): - **Forward Chaining**: Linear progression—choose operator, test effect, repeat until goal reached - **Backward Chaining**: Start from known solution, work backward to determine how to reach goal from current state - **Sub-goaling**: Identify sub-problems that must be solved to reach goal; treat each sub-goal as distinct problem; chain solutions together - **Means-End Analysis**: Calculate difference between current state and goal; identify operators that reduce that difference; apply those operators - **Insight**: Restructure problem space itself, changing fundamental assumptions; generates novel solution paths - **Example**: Penetration tester wants to exploit web app; faces WAF (Web Application Firewall) blocking SQL injection; sub-goal becomes bypassing WAF, then exploiting vulnerability

5. **Knowledge-Lean Problems**: - Problems requiring only information in puzzle statement, not external domain knowledge - Example: River crossing puzzles, logic riddles, constraint satisfaction puzzles - Domain-specific advantages may not apply; forces problem solvers to rely on reasoning processes

6. **Schema Activation and Problem Categorization**: - Schema: Mental categorization of problem types; "this is one of those problems" - Experts trigger schema earlier than novices when presented with problems - Schema-driven approach usually enables faster solution if schema is correct - Problem: Premature schema activation can lead to incorrect assumptions, wrong solution paths - Example: Knights & Knaves puzzles follow specific logical structure; if solver assumes wrong structure, leads down unproductive path

Understanding and Insight Mechanisms:

7. **Insight Problem Examples**: - River crossing with boat: Insight is realization that items can be transported back across river, not just forward; changes problem space - Bridges and candles: Insight is recognizing that wall-mounting isn't required; creates new problem space - Insight-driven problems require restructuring assumptions, not just logical iteration

8. **Problem-Solving as Skill vs. Innate Ability**: - Conventional belief: Problem-solving ability is innate (you either have it or don't) - Research evidence: Everyone can improve problem-solving ability through practice - Measurable improvements: Speed, error rate, approach quality, comfort with ambiguity

The Role and Impact of Expertise:

9. **Expert vs. Novice Differences**: - Experts know larger variety of problem schemas (categories of problem types) - Experts make fewer mistakes (key differential in controlled studies) - Experts are better at self-monitoring: detecting when they're wrong, estimating problem difficulty - Experts often DON'T solve problems faster than novices; speed increase minimal in controlled settings - Difference: Quality of solution, error detection, estimation accuracy

10. **Expertise Paradox in Problem Solving**: - Algebra study: Mathematicians vs. undergraduates given algebra problems; mathematicians did NOT solve faster but made significantly fewer errors - Schema-triggered assumptions: Expertise creates preconceptions about problem types; can misdirect solving - Risk: Experts make confident decisions based on schema assumptions without validating those assumptions - Expertise doesn't guarantee better results on unfamiliar problem types

11. **Memory Chunking and Experience**: - Experts may perform faster due to memory chunking: ability to make deductive leaps by combining experience chunks - Chunking based on patterns recognized through extensive practice - Chunking fails when problem doesn't fit recognized patterns

Cognitive Biases in Problem-Solving:

12. **Experience Bias**: - Relying on past experience to inform decisions in new situations - Common in management and decision-making - Risk: Past solutions applied to situations where they don't fit; precedent treated as principle

13. **Confirmation Bias**: - Tendency to seek information confirming pre-existing hypothesis - In security: Analyst finds evidence supporting initial vulnerability hypothesis, stops searching for alternative explanations - Especially problematic in complex investigations where multiple hypotheses plausible

14. **Availability Bias**: - Problems or solutions coming to mind easily considered more frequent or important - In security: Common attacks (SQL injection, XSS) dominate thinking; overlooked are less common but equally damaging attack vectors

15. **Anchoring Bias**: - Initial information disproportionately influences judgment - In security assessments: Initial finding anchors severity assessment; subsequent findings filtered through that initial judgment

16. **Groupthink**: - Group conformity pressure overrides individual critical thinking - In incident response teams: Team consensus on root cause prevents questioning assumptions - Expert opinion in group can silence dissenting voices

17. **Dunning-Kruger Effect**: - Low-ability individuals overestimate competence; high-ability individuals underestimate relative competence - In security: Junior analysts overconfident in vulnerability assessments; senior analysts doubt expertise - Affects team dynamics and risk assessment accuracy

Puzzle Programs for Security Training:

18. **Measuring Problem-Solving Ability**: - Solution time: How quickly problems solved - Approach quality: Methodology and reasoning process - Comfort with ambiguity: Ability to work with uncertainty, incomplete information - Error rates: Mistakes made during problem-solving - Self-assessment accuracy: Ability to estimate problem difficulty before attempting

19. **Design of Effective Puzzle Programs**: - Variety of problem types: Forces cognitive flexibility, prevents over-reliance on schema - Increasing difficulty: Progressive challenge maintains engagement - Explicit teaching of strategies: Introduce problem-solving approaches; demonstrate application - Group problem-solving: Surfaces biases, enables perspective-sharing - Timed vs. untimed: Different cognitive processes; both valuable

**Defense & Takeaways**

Improving infosecurity decision-making through problem-solving training and bias awareness: (1) Establish organizational puzzle programs—create regular puzzle challenges for security teams; train problem-solving approaches systematically; measure impact on incident response speed/quality; (2) Counter schema-based assumptions—experts in security tend to pattern-match to known attack types; design training that presents unfamiliar scenarios; require explicit validation of assumptions before committing resources; (3) Implement structured problem-solving methodologies—teach forward chaining, backward chaining, means-end analysis, sub-goaling; apply consistently in incident response, vulnerability analysis, threat modeling; (4) Train in multiple solving strategies—don't rely on single approach; encourage trying different strategies when stuck; recognize that insight often comes from restructuring problem, not just executing standard operations; (5) Manage confirmation bias in investigations—require active search for disconfirming evidence; assign "red team" role to question prevailing hypothesis; document decision trees showing alternatives considered; (6) Diversity of analysis perspective—assemble incident response teams with varied experience; younger analysts bring fewer schema assumptions; diverse backgrounds generate diverse problem-solving approaches; (7) Teach metacognition—train analysts to monitor their own thinking; recognize when making assumptions; estimate confidence in conclusions; identify when falling back on schemas too quickly; (8) Create safe failure environments—puzzle training provides safe practice ground for wrong approaches; normalize failure; extract learning from failed solving attempts; (9) Measure problem-solving ability in hiring—use puzzle challenges in security hiring; assess approach quality, not just solutions; value candidates comfortable with ambiguity; (10) Design security processes that counter biases—require documented reasoning for technical decisions; implement peer review of vulnerability assessments; escalate assumptions to formal review before action; (11) Industry security challenges—sponsor or participate in security puzzle competitions; creates motivation for skill development; benchmarking against peers; (12) Develop threat modeling puzzles—create simplified versions of real systems; train analysts to systematically decompose complex systems; practice identifying attack surfaces; (13) Create domain-specific puzzles—use puzzles reflecting actual security work: packet analysis challenges, log analysis puzzles, configuration security challenges; (14) Measure and optimize—track metrics: what problem types most common in incidents? Which solving strategies most effective? Which biases most common? Use data to refine training; (15) Executive education—managers and senior leaders benefit from problem-solving training; improves risk assessment, decision quality, team oversight. The critical insight: Expertise creates both advantages and blind spots. Expert security analysts recognize patterns quickly but may fall victim to premature schema activation—assuming a problem is of a known type without validating that assumption. Cognitive biases compound this: confirmation bias, anchoring, availability all distort judgment in incident response and vulnerability analysis. The research on problem-solving suggests that organizations improve decision quality not by hiring smarter analysts but by training problem-solving approaches, exposing cognitive biases, and creating processes that force explicit validation of assumptions. Puzzle programs aren't games—they're deliberate practice for the problem-solving skills security professionals need. By understanding how the brain solves problems and where that process fails, organizations can design security training, incident response procedures, and vulnerability assessment processes that deliver more reliable, higher-quality security outcomes.