Ariel Herbert-Voss - Don't Red Team AI Like a Chump
Provides a structured adversarial ML red team methodology for attacking production AI systems beyond naive input perturbation.
Summary
Red Teaming AI Systems
Ariel Herbert-Voss (OpenAI safety researcher) presents a structured methodology for red teaming AI and machine learning systems, going beyond naive adversarial examples to address production attack surfaces.
Beyond Adversarial Images
The popular conception of ML red teaming focuses on adversarial perturbations (image classifiers fooled by small pixel changes). Herbert-Voss argues this misses the most impactful attack surfaces in production systems: training data poisoning, model extraction, and API-level inference attacks.
Model Extraction and Intellectual Property Theft
By sending carefully crafted queries to a black-box ML API, attackers can reconstruct an approximate clone of the target model. The talk demonstrates extracting a functionally equivalent model from a commercial API with a budget of under $100 in API calls.
Production Deployment Attacks
The talk covers supply chain attacks on ML pipelines: poisoning training datasets via malicious contributions to public datasets, inserting backdoors into pre-trained models distributed via model hubs, and exploiting serialization vulnerabilities (pickle deserialization) in model file formats.
Related talks
Blacks in Cyber Village·59 min
Birhanu Eshete
Dr. Birhanu Eshete evaluates the trustworthiness of machine learning models in healthcare, autonomous driving, and finance, analyzing adversarial attacks and fairness pitfalls.
AI Village·58 min
Journal Club Live
Discusses practical methodologies for discovering exploitable bugs, implementation flaws, and attack surfaces in ML systems.
AI Village·30 min
Erick Galinkin
A primer breaking down core vocabulary, concepts, and foundational theory in machine learning security.