AI Vendor Risk Assessment: Vendor Claim Is Not Evidence
A practical evidence gap guide for reviewing AI vendors before procurement, renewal, or audit preparation.
AI vendor risk assessment is not mainly a questionnaire problem.
It is an evidence conversion problem.
Most teams already have vendor questionnaires. They ask about security, privacy, subprocessors, SOC 2, access control, incident response, data retention, and contractual terms.
That is useful.
But it is not enough.
A vendor can answer every question and still leave the buyer without reviewable evidence.
The harder question is not:
Did the vendor answer?
The harder question is:
Can the buyer rely on this answer later?
That is the purpose of AI Vendor Evidence Gap Pack.
It helps buyers and reviewers turn AI vendor claims into evidence requests, weak-answer patterns, buyer questions, review notes, and usage boundaries.
It does not approve vendors.
It shows where the vendor’s claim stops and where the buyer still lacks evidence.
Problem
A vendor may say:
We do not train on your data.
That may be true.
But a review should not stop there.
The buyer still needs to know what “training” means. Does it include fine-tuning, evaluation, abuse monitoring, support review, retained logs, human review, embeddings, metadata, subprocessors, or model providers?
The buyer also needs to know where the claim appears.
A DPA clause is not the same as a marketing page.
A SOC 2 report is not automatically proof that the specific AI feature, model path, data flow, or product tier is covered.
A trust center can provide useful source material, but it is not a completed risk assessment.
The problem is not that every vendor claim is false.
The problem is that many vendor claims remain narrative claims.
They sound reassuring. They may be directionally useful. But they do not automatically become evidence requests, evidence gaps, buyer questions, review notes, or residual risk language.
That creates a gap in the review file.
If the vendor is approved or renewed, the buyer may still be unable to explain why the answer was accepted.
What this pack does
AI Vendor Evidence Gap Pack converts vendor-controlled language into reviewable structure:
AI vendor claim -> evidence source -> evidence gap -> buyer question -> usage boundary
This is not a longer questionnaire.
A questionnaire collects answers.
This pack helps review whether those answers are evidence-complete.
For each claim, the review asks:
What exactly is being claimed?
Where is the claim written?
Is the source contractual, audited, official, public, marketing, or unsupported?
Does it cover the actual product, plan, region, and use case?
Does it cover the relevant data path?
What is missing?
What should the buyer ask next?
What usage boundary is reasonable until the missing evidence is resolved?
The output is not a pass or fail conclusion.
The output is evidence structure.
Example
Vendor claim:
We do not train on your data.
Why it sounds sufficient:
It appears to answer the main AI privacy concern.
What it actually proves:
It may support a narrow commitment about model training, depending on the source, scope, product, plan, contract, and date.
What it does not prove:
It does not prove prompt logging, output storage, file retention, metadata handling, embeddings, abuse monitoring, support access, human review, subprocessors, deletion controls, retention period, data residency, audit logs, or exportability.
Weak-answer pattern:
The vendor repeats that it does not train on customer data, but does not define training, retention, logging, evaluation, human review, support access, subprocessors, or exceptions.
Evidence request:
Provide the data flow for prompts, outputs, uploaded files, logs, embeddings, metadata, moderation events, support access, subprocessors, model providers, and human review queues. Include retention period, access roles, deletion controls, tenant settings, contractual commitments, and documented exceptions.
Review note:
The claim is directionally useful but not evidence-complete. It addresses model training use, but does not establish how operational data is handled after inference.
Usage boundary:
Do not rely on this claim for customer data, confidential data, regulated data, proprietary source code, or production approval until retention, logging, human review, subprocessor, and deletion boundaries are evidenced.
The point is not to accuse the vendor.
The point is to avoid accepting a claim before it becomes reviewable evidence.
Who it is for
This pack is for people involved in AI vendor review before procurement, renewal, internal approval, or audit preparation.
That includes security reviewers, compliance teams, procurement operators, privacy leads, IT managers, founders, operators, and small teams adopting AI tools.
It is especially useful for teams that do not yet have a mature AI vendor review process, but still need to ask defensible questions before real data use.
The review should not start with the vendor name alone.
The real review unit is:
vendor + product + plan + use case + data type + region + contract terms + evidence date
The same vendor may be acceptable for public marketing drafts and not acceptable for customer records, source code, employee data, regulated data, or automated decisioning.
Context matters.
What it is not
AI Vendor Evidence Gap Pack is not certification, vendor rating, legal advice, regulatory advice, audit opinion, procurement approval, compliance guarantee, or a pass/fail conclusion.
It is not a substitute for legal, security, privacy, audit, procurement, or business owner review.
It is also not a generic AI governance checklist.
The product boundary is simple:
Evidence gap review, not approval.
Reading path
Each claim teardown follows the same review path:
vendor claim → evidence source → evidence gap → buyer question → usage boundary
Start with the core frame:
Then read the claim teardowns:
Planned next:
Why SOC 2 Does Not Prove the AI Vendor Data Path Is Covered
Human Review Is a Data Exposure Path Unless It Is Bounded
Subprocessor Lists Do Not Show the Actual AI Data Path
This series starts with one claim:
Boundary
This material is for evidence structuring and review preparation.
It does not provide legal, regulatory, audit, procurement, certification, or implementation advice.
I now have a sanitized sample evidence gap memo showing how this looks in a review file.
Reply if you want the sample.