September 10, 2026
Mastering IDOR: Finding Broken Access Control Behind Predictable IDs and UUIDs
Mastering IDOR: Finding Broken Access Control Behind Predictable IDs and UUIDs
Salman Khan
New Update: Our new blog is updated.
Free download
Omair Manzoor is the Founder, CEO, and Chief Hacker of ioSENTRIX, a CREST-accredited cybersecurity firm. With 14+ years in offensive security, he has led penetration tests for Fortune 500 companies including engagements for Amazon, Bank of America, VISA, and the London Underground. His exploits are part of the Metasploit Framework and Immunity Canvas. Omair is a published researcher, conference keynote speaker (DOD, CISA, BSides), and technical reviewer for academic security textbooks.
About this research. Everything described here was a proof-of-concept run by ioSENTRIX in an isolated lab, using only synthetic and simulated data. No customer, engagement, or production information was involved at any stage, and none of the infrastructure used in the experiment has ever held client data. Results describe our workload, our configuration, and our test period. They are not universal model rankings.
Here is a question most teams building an internal AI platform cannot comfortably answer: when your autonomous agent fails to call a tool, do you know whether the model failed, or whether your serving stack did?
We didn’t. It cost us weeks.
Open-weight models are now capable enough to be useful for real security work: reviewing code, reasoning through Active Directory attack paths, analyzing cloud IAM, drafting findings, and driving semi-autonomous testing agents. For a CREST-accredited penetration testing firm, the appeal is not novelty. It is control. Customer confidentiality, tenant isolation, and predictable access controls make a strong case for an organization-controlled AI layer rather than a pile of consumer subscriptions.
So we set out to answer a practical question: can open models provide a credible private AI platform for professional cybersecurity work, including agent workflows, without depending entirely on proprietary frontier APIs?
The short answer is yes. The longer answer is that the model turned out to be the least interesting part of the problem.
We built a small control plane in the lab: a chat interface for assessors, an open-source autonomous pentesting agent (PentAGI), and a model gateway in between that exposes stable internal aliases such as General, Security, Reasoning, Code, and Vision. Applications talk to the alias. The gateway decides which model and which backend serves it.
Our first backend was a self-hosted, FP8-quantized Qwen3 30B mixture-of-experts model, served by vLLM on a single rented 48 GB GPU. For ordinary chat it looked promising. Responses were fast, technically sensible, and cheap.
Then we pointed the agent at it.
PentAGI ships a provider compatibility tester (its ctester): 295 checks spread across a dozen agent roles, exercising plain responses, JSON output, streaming, tool calls, memory across turns, file read-and-edit workflows, and pentest tool selection. It is not an intelligence benchmark. It answers a narrower question: can this model reliably operate the protocol this agent expects?
The self-hosted configuration passed 187 of 295 tests, 63.39%.
At first glance, that is a bad result for the model. But every failed category was about tool generation or tool interpretation: echo functions, search-query functions, JSON response functions, file edits via unified diff, pentest memory with tool calls, pentest tool selection. The recurring error was the same each time:
no tool calls found, expected at least 1
When we opened the raw responses, the model was clearly trying to call the tool:
<tool_call>
{"name": "echo", "arguments": {"message": "hello"}}
</tool_call>
The model had produced the intent. The serving stack returned it as ordinary assistant text instead of translating it into the OpenAI-style tool_calls structure the agent was waiting for. From the agent’s point of view, nothing had been called.
A model can know how to call a tool and still fail as an agent. The failure was not "Qwen is bad at tool calling." It was closer to "the model, the chat template, the inference server’s tool parser, and the OpenAI compatibility layer did not reliably convert the model’s tool intent into the protocol the application consumes."
That distinction matters, and it is easy to miss. We also found version-specific parser behavior along the way: the installed inference server rejected the obvious parser name for this model family and required a variant. A one-line configuration detail sat between a 63% result and a working agent.
The serving layer had become a first-class benchmark target.
To test whether the serving path was the culprit, we ran open models through Amazon Bedrock’s OpenAI-compatible endpoint, keeping our gateway as the abstraction boundary so nothing upstream changed. Before involving the agent, we ran a five-minute direct tool-call test against Qwen3 Coder Next. It returned a proper structured tool call with the correct finish reason.
Then we reran the 295-test suite.
294 of 295, 99.66%. A repeat run scored 292 of 295. The three misses were small and nondeterministic: two malformed JSON argument cases and one search call that wasn’t emitted. Not a protocol failure. Noise.
We tested two more models through the same path. DeepSeek V3.2 scored 287 of 295 (97.29%). Kimi K2.5 scored 284 of 295 (96.27%). A smaller Qwen3 Coder 30B model scored 253 of 295 (85.76%): fast, but materially weaker at autonomous orchestration, which is a useful reminder that "coding model" and "agent model" are not synonyms.
Two honest caveats. First, this is not a clean model-versus-model test. The 63% run was our self-hosted Qwen3 30B mixture-of-experts; the 99% run was Qwen3 Coder Next on the managed path — a different, coder-tuned member of the family, not the same model moved between serving stacks. We did not re-run the identical self-hosted model through the managed endpoint, so the headline gap reflects a change of model and serving path together. What actually points at the serving layer is not that delta but the self-hosted model’s own raw output: it emitted a valid tool call that the stack handed back as plain text, and a direct tool-call test confirmed a properly translated call was achievable on the managed path. Second, none of this proves managed inference is inherently better than self-hosting. It proves something more useful.
The inference server, chat template, tool parser, and API compatibility layer are part of your AI system, and they need to be benchmarked with the model. A leaderboard score tells you what a model can do under someone else’s harness. It tells you nothing about whether it will operate your agent through your stack.
Self-hosting is a legitimate choice, and it remains in our architecture. But it transfers responsibility for inference correctness and protocol compatibility to you: GPU lifecycle, engine upgrades, parser configuration, chat templates, quantization behavior. If you take that on, you have to test it like you would test any other component you own.
If we were starting again, we would benchmark in this order: direct API chat, direct structured output, direct automatic tool calling, streaming tool calls, then the model through the gateway, then the actual agent, then correctness and refusal behavior, then repeat everything. Only after that would we optimize cost. A five-minute direct tool-call test at step three would have isolated our problem in an afternoon.
This is the same principle we apply to every security control we assess. A vendor asserting that a model "supports tool calling" is a claim. A 295-test run through your own serving path is evidence. The gap between the two was 36 percentage points in our lab, and nothing about ordinary chat testing would have revealed it.
The next question, which we cover in the companion post, is harder: once a model reliably calls tools and willingly answers authorized security questions, is the answer actually correct? Ours confidently gave us the wrong Hashcat mode.
Most often the inference server’s tool-call parser is not configured, or is configured for the wrong model family, so the model’s structured intent is passed through as ordinary content. Check the parser setting and chat template for your exact model version before concluding the model can’t call tools.
No. Models offered through a managed service such as Amazon Bedrock are open-weight models under managed inference. You get portability of the model and a simpler serving path, but the provider runs the infrastructure. Whether that fits your privacy and compliance requirements depends on your data classification and your contracts, which is why we ran this entire experiment on synthetic data.
A model gateway sits between your applications and your model providers, exposing stable internal names while controlling the backend, credentials, budgets, and allowlists. It lets you swap a model or a serving path without rewriting integrations or retraining staff.
No. Tool reliability, willingness to answer authorized security questions, and technical correctness are three separate dimensions. A model can score 99% on protocol compatibility and still hallucinate a command.
ioSENTRIX is a CREST-accredited, ISO/IEC 27001 certified penetration testing firm. Our AI and ML penetration testing and AI assurance engagements test whether the AI systems you deploy, including agents and the tool chains behind them, behave as claimed under adversarial conditions, with evidence rather than assertions. If you are standing up a private AI platform or an autonomous agent and want it benchmarked the way it will actually be used, we should talk.
