Phishing With a Net: The NIST Phish Scale and Cybersecurity Awareness

### Overview Presented at RSA by NIST computer scientists Shanee Dawkins and Jody Jacobs from the usability group, this talk introduces the NIST Phish Scale—a metric developed to contextualize employee click rates during phishing awareness and simulation exercises. The researchers argue that relying exclusively on click rates to measure training success provides incomplete and misleading data ("phishing without a net"). By factoring in the human element and user context, the NIST Phish Scale offers a systematic way to measure detection difficulty for phishing emails.

### Key Topics - **Limits of Technology and Raw Metrics**: Automated filters do not catch all malicious emails (approximately 10% get through), making informed users a necessary line of defense. However, click rates exhibit high variability across exercises because raw numbers fail to reflect why people click. - **Empirical Research on Clickers vs. Non-Clickers**: NIST conducted a 4.5-year study across 15 phishing exercises at a NIST operating unit. Follow-up surveys revealed four distinct behavioral differences between clickers and non-clickers: 1. *Email Premise Alignment*: Clickers responded when the premise matched their daily job roles (e.g., handling invoices). 2. *Interpreting Cues*: Non-clickers noticed suspicious cues (such as attachment mismatches like a `.doc` referenced in text being a `.zip` file), whereas clickers were swayed by familiar branding or cues from assumed colleagues. 3. *Reality Checking*: Only non-clickers performed fact-checking, such as searching organizational directories for unknown senders. 4. *Consequences*: Clickers feared being unresponsive to legitimate work, while non-clickers prioritized avoiding malware or security incidents. 5. *Security Mindset*: Clickers showed overconfidence and unrealistic faith in automated institutional security, whereas non-clickers maintained a realistic view of being targeted. - **The NIST Phish Scale Formula**: The scale conceptualizes detection difficulty as: `Email Cues + Premise Alignment = Detection Difficulty`.

### Technical Details - **Observable Cues (23 Cues)**: Cues can be red flags (e.g., spelling errors, domain spoofing, mismatched file types) or compelling cues (e.g., spoofed executive names, familiar branding, time pressure). The fewer cues present, the harder a phish is to spot. - **Premise Alignment Elements**: Relevancy is assessed across five elements: 1. Mimicking workplace processes/practices. 2. Workplace relevance. 3. Alignment with internal or external current events. 4. Concern over consequences if not clicked (e.g., threat of disciplinary action or unpaid invoices). 5. Targeted training or specific advance warnings received by the audience. - **Job Families Segmentation**: A one-size-fits-all organization assessment often masks vulnerabilities. When premise element #2 (workplace relevance) is broken down into specific job families (such as Administrative Support, Core Mission, Facility Field, Facility Office, Legal, Management/C-Suite, and Organization Support Staff), detection difficulty shifts dramatically depending on role responsibilities.

### Takeaways - Training implementers should use the Phish Scale to interpret training data and explain fluctuations to leadership (e.g., high click rates on highly difficult emails indicate sophisticated scenarios rather than program failure). - Training programs should tailor simulated attacks to job-specific contexts rather than relying on generic exercises. - Immediate recommended actions include evaluating existing phishing email templates against the scale's 23 cues and 5 premise elements, tailoring scenarios over the subsequent three months, and reporting context-driven metrics to leadership.