Mastering IDOR: Finding Broken Access Control Behind Predictable IDs and UUIDs

We self-hosted an open model for an autonomous pentest agent and it passed 63% of a 295-test compatibility suite. Moved to a managed serving path, the same model family hit 99%. The model was never the problem — the serving stack was.

“Test the customer-facing AI assistant” is one sentence, and the whole engagement will be argued from it. The seven surfaces an AI penetration test scope must name, what belongs in the rules of engagement, and what a finding must carry.

Two firms quote the same AI system and the numbers differ fourfold. Both are honest. The seven scope decisions that set the price of an AI penetration test, and the three questions that separate a scoped proposal from a priced guess.

Your test report names one model. Your gateway can reach six. Routers and cascades make the execution path probabilistic — what that does to a test plan, and how to sample representative paths honestly.

Your AI inventory survives "what are we running?" and fails "how do you know that's all of it?" Six discovery signals, the blind spot in each, and how to rate an inventory's confidence honestly.

Your agent telemetry is better than it has ever been, and it still cannot certify that your controls held under attack. Three limits of the signal layer, each with a test attached.

Every workflow you are automating had a person doing integrity work nobody logged as a control. Here is what has to replace it — and how to test that it holds.

MCP turned the agent tool-chain into a supply chain. Tool descriptions are instructions a model reads, scopes are granted in bundles, and chained agents lose provenance. Three boundaries, each with a test attached.

Agent and service identities now outnumber humans in most environments, and almost none go through joiner-mover-leaver. Six tests for non-human identity scope, credential lifecycle, and privilege propagation across an agent-to-tool chain.

Most shadow AI was never bought. It was switched on inside software you already approved — invisible to policy lists, procurement, and the proxy. Four reasons it escapes your inventory, and the three questions that find it.

With an authorized-testing context, capable open models answered our offensive-security questions with zero refusals — and still gave the wrong Hashcat mode for Kerberoasting. Why compliant is not correct, and the three axes a security-AI benchmark must measure separately.

Boards ask four questions about AI agents, and the honest answer is usually “we believe so.” Here is how each one maps to a concrete assessment domain — what gets tested, what the tester does, and what evidence comes back.