July 14, 2026
Securing RAG Pipelines: Risks in Retrieval-Augmented Generation
Securing RAG Pipelines: Risks in Retrieval-Augmented Generation
Omair
New Update: Our new blog is updated.
Free download
Omair Manzoor is the Founder, CEO, and Chief Hacker of ioSENTRIX, a CREST-accredited cybersecurity firm. With 14+ years in offensive security, he has led penetration tests for Fortune 500 companies including engagements for Amazon, Bank of America, VISA, and the London Underground. His exploits are part of the Metasploit Framework and Immunity Canvas. Omair is a published researcher, conference keynote speaker (DOD, CISA, BSides), and technical reviewer for academic security textbooks.
At 2:14 a.m., an agent in your accounts payable workflow updates a vendor's bank details and releases a payment. Nothing looks wrong. The agent authenticated with a valid credential. The tool call returned success. The approval step was satisfied. The log line is there, timestamped, exactly where you would expect it.
Now answer one question about that action: was it real?
That question is what we mean by AI integrity, and most security programs cannot answer it. They can tell you whether the data was encrypted in transit, whether the service stayed up, and which identity held the token. Far fewer can tell you whether the action itself was genuine. That gap is where we think the next several years of security work happens.
Confidentiality and availability got decades of investment. Encryption is a default. Key management is a product category. DDoS protection, redundancy, and failover are budget line items boards understand. Not finished, but industrialized.
Integrity got less attention because, for most of computing history, it was easier. The actor was a human or deterministic code. Integrity meant hashes, digital signatures, change control, and tamper-evident logs. You verified that bytes had not changed and that a person with a badge had approved something.
Autonomy breaks it. An LLM-driven agent does not execute a fixed program. Its behavior is a function of the context it is given, and that context is reachable by an attacker: retrieved documents, tool outputs, ticket comments, web pages, memory from prior runs, and messages from other agents. In these systems the instruction channel and the data channel are the same channel. A poisoned document is not a data exposure. It is a forged instruction that your infrastructure will faithfully carry out, sign, and log.
Add non-human identity to that. Agents are not just workloads. They are identities that take consequential action, they proliferate faster than people do, and they are frequently provisioned with the standing permissions of whoever built them. It is also why threat modeling an AI system has to cover the whole pipeline rather than the model alone. The result is a system where the hard question is no longer "who saw this data." It is "did this action genuinely originate from an authorized principal, within its mandate, without interference." That is an integrity question, and the industry does not have an off-the-shelf answer for it yet.
This is our view rather than settled fact, formed from testing these systems rather than reading about them.
Here is a composite of what we see. It is illustrative, not any one client.
A mid-market company deploys an agent to triage support tickets and issue refunds under a threshold. The AI policy is genuinely reasonable. Agents run under least privilege. Financial actions require human approval. All agent activity is logged. Tool integrations are reviewed before use.
Then you look at the system.
The agent runs under a shared service account, because the SaaS platform it lives in does not support per-agent identity, so "least privilege" is scoped to a team, not to an agent. The human approval is a button in a chat channel, and the workflow bot that posts the request can also click it, so the control can be satisfied without a human. Two of the tool integrations were added by a developer in a config file and never went through the review the policy describes. The logs capture the tool call and its parameters, but not the retrieved content that led to it, so when someone asks why the agent did that, nobody can reconstruct the answer from evidence. They can only guess.
Every control on that list exists. On a questionnaire, the company is in good shape. Under adversarial conditions, all four fail, and the fourth fails in the way that makes the other three impossible to check after the fact.
That divergence is the entire problem. Existence and effectiveness are different claims, and only one of them is a security outcome.
Start by defining what you are trying to prove. For an autonomous action, we think there are four properties worth testing, and they are testable.
Genuine. The action traces back to an authorized principal and a legitimate instruction, not to content the agent ingested. The test is to inject through the data path the agent trusts most and see whether it reaches a consequential tool call. Not whether the model says something odd. Whether money moves, a record changes, or a permission gets granted.
Authorized. The action falls inside a scoped mandate at the moment it happens, not inside a role granted at build time. The test is to attempt privilege boundary crossing between agents: can agent A reach agent B's tools, or escalate through a shared identity, or reuse a token past its intended context.
Untampered. The instruction, the tool call, and the result are unchanged in flight, including across the tool and MCP layer. Treat that layer as supply chain, because it is one. The test is to re-verify a tool server against what you approved at integration time, then diff the instruction, the call, and the returned result at each hop. A tool server can change its behavior after you approve it, and nothing in the transcript will look different when it does. This is the same reason a point-in-time test stops being true the moment the system moves.
Reconstructable. You can replay the decision from evidence. Take a real action from last week and try to answer "why did it do that" using only logs and stored context. Most teams discover they recorded the call but not the cause. That is a serious finding on its own, because it means you cannot investigate an incident you have already had.
The frameworks help you organize this. The OWASP Top 10 for Agentic Applications, published in late 2025 as the 2026 edition, gives useful shared vocabulary for things like goal hijack, tool misuse, identity and privilege abuse, and memory and context poisoning. NIST's AI RMF and ISO/IEC 42001 give you governance structure. MITRE ATLAS gives you adversary technique language. CSA's AI Controls Matrix gives you control coverage. Use them. Just be clear about what they are. They are maps, not tests, and mapping your controls to a framework is not evidence that those controls work. Aligning to a framework is also not the same as being certified against it, and even where certification does exist, as it does for ISO/IEC 42001, it certifies a management system rather than proving that any given control held under attack.
This is why we keep coming back to the same principle: prove, don't assert. An assessment that ends in a spreadsheet of "implemented" and "partially implemented" is a collection of claims. Evidence is a record of a control being attacked and either holding or failing, with the attempt, the result, and the artifact attached. Those are different deliverables, and the second one is the only one that answers the question a board is actually asking.
We are also honest about what produces it. A governance assessment reviews the paper. A platform vendor observes its own telemetry, which is useful, but it is the vendor grading its own homework. Independent adversarial testing is the position that produces this kind of evidence, whoever performs it. That is the posture we have chosen, and it is why we stay services-first. We assure the stack you chose. We are not trying to sell you a different one.
None of this eliminates risk. Anyone promising that is selling something. Adversarial testing reduces risk and reveals it, and revealing it is often the more valuable half, because the failures that hurt are the ones nobody knew to look for.
If you want a self-check before you talk to anyone, pick one agent that is live in production today and answer four questions with evidence rather than intent. Can you prove its last consequential action came from an authorized instruction and not from ingested content? Can you show its permissions are scoped to it alone, and that every approval step in its path genuinely requires a human? Can you show the tool servers it calls still behave the way they did when you approved them? Can you reconstruct why it did what it did?
If the answers come back as "probably," that is your gap, and it is a normal place to be right now. If you want a second set of eyes on it, we are happy to have that conversation.
We are a CREST-accredited, ISO/IEC 27001 certified offensive security firm, and our AI and ML penetration testing runs on the same principle as the rest of our work: prove, don't assert. We attack the agent, the identity model, and the tool chain, then hand you the attempt, the result, and the artifact rather than a spreadsheet of control statuses. Where you need that assurance to keep pace with a system that changes weekly, we run it continuously rather than annually. We are services-first, so we assure the stack you already chose.
If you want a second set of eyes on an agent that is live in production, get in touch.
