What AI Evidence Does a SOC 2 Auditor Want?

There is no shortage of AI vendor question lists. What is scarce is a view of which answers break when somebody tests them. Six claims that survive a questionnaire and not an engagement, the test that settles each, and what to ask instead.

An MCP server is an API with an unusually trusting client and an unusually powerful backend. Testing it means exposure and transport, three authorization attempts against a stated MUST NOT, the advertised tool surface, and what the server sends back — almost all of it without a model in the loop.

Forty prompt-injection payloads, none successful, risk low — and six weeks later the agent refunds the same customer twice. The unit of an agent test is the action, not the answer. Four phases, five channels, and why findings carry a rate instead of a checkmark.

“Test the customer-facing AI assistant” is one sentence, and the whole engagement will be argued from it. The seven surfaces an AI penetration test scope must name, what belongs in the rules of engagement, and what a finding must carry.

Two firms quote the same AI system and the numbers differ fourfold. Both are honest. The seven scope decisions that set the price of an AI penetration test, and the three questions that separate a scoped proposal from a priced guess.

Enterprise buyers have moved from attestation to evidence on AI. The premium does not show up as price — it shows up as sales cycle length, contract terms, and the deals you are never invited into. Four decisions for a board.

Your agent telemetry is better than it has ever been, and it still cannot certify that your controls held under attack. Three limits of the signal layer, each with a test attached.

Three proposals, all called an AI security assessment, priced five-fold apart. What actually drives the number, the four conditions that justify adversarial testing, what each deliverable contains, and the five questions to ask before you sign.

Every workflow you are automating had a person doing integrity work nobody logged as a control. Here is what has to replace it — and how to test that it holds.

Most AI risk registers collapse “untested” and “tested and failing” into the same amber, which inverts the priority a defender should have. Why a proven failure is a bounded, demonstrated path, why an unknown still deserves respect, and how a state field fixes the ranking.

Boards ask four questions about AI agents, and the honest answer is usually “we believe so.” Here is how each one maps to a concrete assessment domain — what gets tested, what the tester does, and what evidence comes back.

Governance asks whether a control exists. Assurance proves it works. Why an AI security assessment built on questionnaires tells you almost nothing, and what evidence looks like.