AI Vendor Risk Is Not a Questionnaire Problem
A vendor can answer every question and still leave the buyer without usable evidence.
AI vendor risk is not a questionnaire problem.
The problem is not that teams lack questions. The problem is that vendor answers are not converted into evidence requests, weak-answer patterns, red flags, and residual risk language.
Opening direction
Most AI vendor reviews start with a questionnaire.
That is reasonable. Questionnaires create structure. They force vendors to answer in writing. They give procurement, security, legal, risk, and audit teams a shared object to review.
But the questionnaire is not the control.
A vendor can answer every question and still leave the buyer with little usable evidence.
The gap appears after the answer arrives.
The vendor says:
We do not train on your data.
The review question should not stop there.
It should ask:
What does training mean?
Does that include fine-tuning?
Does that include evaluation?
Does that include abuse monitoring?
Does that include support review?
What is retained?
What is logged?
Which subprocessors touch the data?
Can the customer export evidence later?
The claim may be true. But it is not yet evidence-complete.
Core argument
A questionnaire collects vendor statements.
An evidence review converts those statements into reviewable objects.
Those objects include:
evidence requests
weak-answer patterns
red flags
review notes
residual risk language
That conversion step is where many AI vendor reviews are weakest.
Why AI vendors make this harder
Traditional vendor risk templates were built for more stable control surfaces.
They work reasonably well for questions about:
access control
encryption
incident notification
business continuity
SOC 2 availability
subprocessors
data processing terms
AI vendors add new ambiguity.
The ambiguity is not always in the security layer. It is often in the behavior layer.
Examples:
What data enters the model context?
What data is retained in logs?
What data is used for evaluation?
What changes when the model version changes?
Can the customer identify which model produced a prior output?
What actions can an AI agent take?
What approvals exist before an external action is triggered?
Can logs be exported for audit or incident reconstruction?
A generic questionnaire can ask about these issues, but the harder work is deciding whether the answer is evidence-complete.
Example: polished answer, weak evidence
Vendor answer:
We use industry-leading safeguards to ensure customer data is protected and is not used to train our models.
This answer sounds reassuring.
But as evidence it is weak unless it is supported by:
contractual language
retention periods
logging scope
support access boundaries
subprocessor handling
opt-out configuration
evaluation and fine-tuning scope
exportable records
The issue is not whether the vendor is bad.
The issue is whether the buyer can rely on the claim later.
The review shift
The review should move from:
Did the vendor answer the question?
To:
Can the vendor answer be preserved as evidence?
And then:
If the evidence is weak, what residual risk language should appear in the review file?
What AI Vendor Risk Pack is trying to do
AI Vendor Risk Pack is a practical attempt to make that conversion easier.
It starts with common vendor claims and asks:
What evidence should be requested?
What answer would be weak?
What should be treated as a red flag?
What review note should be preserved?
What residual risk language might be needed?
It is not a certification system.
It is not a vendor rating.
It is not legal or audit advice.
It is a review aid for turning AI vendor claims into evidence-aware risk language.
Possible closing
The next phase of AI vendor risk will not be won by longer questionnaires.
It will be won by better evidence conversion.
The real question is not whether the vendor gave an answer.
The question is whether the buyer can later explain why that answer was accepted.
CTA draft
I am building a lightweight AI Vendor Risk Pack under CodeYourCompliance.
The first version focuses on five domains:
Data use.
Model change and governance.
Security and logging.
Human approval and agent permission.
Auditability and evidence export.
The goal is simple:
Turn AI vendor claims into evidence requests, weak-answer patterns, red flags, review notes, and residual risk language.
The next article applies this to the most common AI vendor claim: “We do not train on your data.”
Read next: Vendor Says It Does Not Train on Your Data. What Evidence Should You Ask For?