Simple explanation
Beyond the core LLM, several supporting concepts show up constantly in real AI deployments — and each has its own security implications worth understanding.
Technical explanation
- Embeddings — numerical vector representations of text (or other data) that capture semantic meaning, allowing similarity between pieces of text to be measured mathematically. Underlies search, recommendation, and RAG systems.
- Retrieval-Augmented Generation (RAG) — supplies an LLM with relevant documents retrieved from an external knowledge base at query time, so it can answer using information beyond its original training data. This is the mechanism behind most "chat with your company's documents" style AI tools — and, critically, a major indirect prompt injection vector, since retrieved content is processed by the model the same way directly-typed input is.
- Inference vs. training — reinforcing the distinction: training happens once (or periodically), inference happens on every single request; most production security concerns (prompt injection, output handling) occur at inference time, not training time.
- Data lineage — tracking where a dataset originated and how it was transformed over time; important for trust, auditability, and compliance, and directly relevant to investigating a data poisoning incident, since you need to know exactly where training data came from to assess exposure.
Synonyms / related terms
| Term | Means | |---|---| | RAG | Retrieval-Augmented Generation | | Embedding | A numerical vector representation of text capturing semantic meaning | | Data lineage | Tracked origin and transformation history of a dataset |
Concept Check
"An attacker embeds hidden malicious instructions inside a company policy document that gets automatically retrieved and fed to an AI assistant answering employee questions." This is a textbook example of exploiting RAG specifically — the malicious content was never typed directly by the attacker into the chat, but was pulled in via the retrieval mechanism and processed by the model exactly as if it had been.
Interview-style Q&A
Q: Why does RAG's retrieval step deserve the same security scrutiny as user input, even though no human typed the malicious content directly? A: "Because the LLM has no inherent way to distinguish 'trusted instructions from my deployer' from 'text that happened to be retrieved from a document.' If a RAG system's retrieved documents can be influenced or tampered with — even documents an attacker doesn't have direct write access to, if they can get content into the knowledge base indirectly — that's a full injection vector, just one step removed from the chat box."
Memory trick
"Retrieve, then Generate" — the literal meaning of RAG's name is also the reminder of its security implication: retrieved content becomes part of what the model generates from, trusted or not.