Exploiting Shadow Data from AI Models and Embeddings
Demonstrates model inversion and embedding inversion attacks against LLMs and RAG systems, showing how sensitive personal data can be extracted from vector databases.
Summary
Shadow Data Exposure in AI Architectures
Modern applications integrating Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) routinely process sensitive personal data. However, data embedded within model weights and vector stores often escapes conventional privacy and DLP scanning.
Model Inversion Attacks on Fine-Tuned LLMs
Fine-tuning LLMs on private enterprise datasets creates memorization risks. Attackers craft targeted prompting sequences that force models to emit memorized training data, including personally identifiable information (PII) and credentials.
Embedding Inversion in Vector Databases
Vector embeddings used in semantic search and RAG pipelines are frequently treated as non-reversible mathematical representations. Inversion techniques demonstrate that high-dimensional embeddings can be mathematically reversed to reconstruct original text strings and sensitive records.
Privacy and Governance Implications
Securing AI ecosystems requires implementing differential privacy during model training, access controls on vector databases, and rigorous context sanitization to prevent sensitive data leakage.
Related talks
Creator Stage·52 min
Dane Sherrets · Shlomie Liberow
Deep dives into adversarial prompt injection, context poisoning, and RAG manipulation flaws to help researchers find and demonstrate high-impact AI vulnerabilities.
The Linux Foundation·46 min
Reviews guarding the LLM Galaxy: Security, Privacy, and Guardrails in the... Jigyasa Grover & Rishabh Misra and evaluates key implications for security practitioners.
Creator Stage·26 min
Andrea Downing
Introduces the Patient AI Rights framework and a community-driven red teaming methodology to stress test clinical AI models and chatbots for bias and failure modes.