More Flows, More Bugs: Empowering SAST with LLMs and Customized DFA

## Overview

Static Application Security Testing (SAST) tools, such as GitHub's CodeQL and Fortify, automate source code vulnerability analysis without executing the application. While CodeQL's queries, libraries, and data flow analysis (DFA) components are open source (with over 418 vulnerabilities credited on its Wall of Fame), evaluating it against recent high-risk bugs revealed high false negative rates. The speakers from Tencent Security Winding Lab identified two primary root causes for these detection gaps: 1. **Incomplete Source and Sink Coverage:** Built-in propagation rules fail to cover many third-party framework APIs. 2. **Disrupted Data Flow:** Standard DFA engines lack robust support for complex language features, breaking taint propagation paths.

To address these limitations, the researchers combined an automated multi-agent LLM pipeline to discover sources and sinks with customized DFA engine patches to repair broken taint paths in Java.

---

## Automated Source and Sink Mining with LLM Agents

Manual definition and community-contributed rules are labor-intensive. Because sources and sinks are typically third-party framework APIs (such as SSRF sinks in the Resty framework), the authors developed a three-agent LLM pipeline to scan open-source frameworks:

- **Discover Agent:** Performs coarse-grained, file-level detection by analyzing source code with bug-specific prompts (e.g., matching HTTP request functions for SSRF or query execution for SQL injection). Results below a confidence threshold are discarded. It then performs fine-grained, function-level filtering on function names and bodies, noting that "long-thinking" models perform best at this stage. - **Judge Agent:** Filters the candidate functions using heuristics and expert community rules (e.g., verifying that functions are publicly accessible, handle return values that propagate tainted data, and avoid boolean-only return types). - **Validation Agent:** Confirms that discovered sources and sinks appear in real-world software. It checks the framework's dependency page on open-source repositories, downloads dependent projects ordered by star count, and executes SAST scans with the candidate rules. Unused sources or sinks are removed.

---

## DFA Mechanics and Language Feature Enhancements

CodeQL performs taint analysis by calculating **nodes** (via forward flow from sources and reverse flow from sinks) and tracking **access paths** (content lists and types across field, array, collection, and map structures over five stages). Custom node-to-node propagation is supported via user-level (`isAdditionalFlowStep`) or system-level (`additionalValueStep`) jump interfaces. The researchers patched CodeQL's Java DFA to overcome three complex language challenges:

- **Cross-Thread Propagation:** Standard analysis breaks when taint passes into concurrency constructs (e.g., the constructor of a `Runnable`). The authors resolved this using `additionalValueStep` to jump from the constructor call directly to the `run()` method's `this` instance parameter. For taint introduced prior to calling `.start()`, the flow jumps from the caller of `.start()` to the `run()` method's instance parameter. For taint arriving after `.start()`, the engine searches for a matching post-update node on a `Runnable` instance with a store operation and connects it to the `run()` instance parameter. - **Java Reflection (`Method.invoke`):** Resolving targets of `.invoke()` required either tracking method instances via global data flow or matching parameter counts and types. Integrating reflection analysis into `additionalValueStep` ordinarily causes non-monotonic recursion (circular dependency between the data flow engine and `additionalValueStep`). The authors resolved this by maintaining an isolated duplicate copy of the data flow implementation to resolve reflection targets before patching the primary DFA engine, correcting interprocedural argument propagation for `call in` and `call through`. - **Pass-by-Value Copy Discrepancies:** In Java, passing reference types creates copies of object heap addresses. Modifying a copy stored in an object field requires updating all other copies. The authors addressed this by locating non-primitive fields, identifying both non-post-update and post-update store nodes, using global data flow to link formal parameters to arguments, and adding explicit jump steps between related update nodes.

---

## Empirical Results and Findings

The combined approach yielded measurable improvements in vulnerability detection: - **Rule Expansion:** Discovered approximately 190 new source and sink functions across 18 Go frameworks. - **Data Flow Coverage:** Across a scan of more than 5,000 open-source projects, the customized DFA and expanded rule set increased detected data flow paths by over 15%. - **Vulnerability Discovery:** Uncovered real-world vulnerabilities, including a SQL injection bug in the Traffic Ops component of the Apache Traffic Control project where CodeQL was already enabled but previously failed to detect the issue due to missing sinks.