AI Vendor Evidence Gap Notes #10
The first nine notes looked at different AI vendor claims: training use, Trust Centers, SOC 2 scope, human review, subprocessors, retention, audit logs, data residency, and questionnaire answers.
The subjects were different.
The review problem was usually the same.
A vendor statement was available, but the supporting evidence was narrower than the statement suggested.
A claim-evidence matrix keeps that difference visible.
Claim
The review file says:
“Vendor assessment completed.”
The questionnaire has been returned.
The Trust Center, DPA, SOC 2 material, subprocessor information, and product documentation have been reviewed.
That shows that a review took place.
It does not show how individual claims were supported.
The file may still contain statements such as:
Customer data is not used for training.
Human review is limited.
Data residency is supported.
Audit logs are available.
Each statement may have a supporting source.
The question is what that source actually covers.
Why it sounds sufficient
Procurement reviews already collect a lot of material.
Adding another matrix can look like duplicate work.
The problem appears when answers and evidence are stored separately.
The questionnaire contains the claim.
The Trust Center contains one source.
The DPA may contain a stronger commitment.
Product documentation may narrow the feature scope.
A SOC 2 report may cover only part of the relevant environment.
If those relationships are not recorded, another reviewer has to reconstruct the reasoning later.
What it actually proves
A claim-evidence matrix records that relationship.
For each material claim, capture:
Claim: What did the vendor state?
Evidence source: What document or page supports it?
Scope: Which product, plan, feature, region, and data type are covered?
Exceptions: What sits outside the statement?
Evidence date: When was the source checked?
Evidence gap: What remains unresolved?
Buyer question: What needs to be clarified next?
Usage boundary: What should remain limited while the gap is open?
The useful sequence is:
vendor claim → evidence source → scope → evidence gap → buyer question → usage boundary
A simple example
Take:
“We do not train on customer data.”
The review should not stop at “No training.”
The matrix may show:
Source: Enterprise data-use documentation.
Scope: The statement applies to a defined commercial product.
Exceptions: Support, abuse monitoring, human review, or opt-in workflows require separate checking.
Evidence gap: Retention, logging, subprocessor handling, and deletion are not established by the training statement.
Buyer question: Which sources establish those remaining data-handling boundaries?
The vendor answer has not changed.
The assumptions around it are now visible.
A public evidence example
The CodeYourCompliance public evidence profile for Cohere separates public evidence surfaces rather than treating the vendor name as one source.
That is useful for locating the source side of the review.
But the profile does not decide whether a particular claim is sufficient for a buyer’s product, plan, use case, data type, region, or contract.
The claim-evidence matrix performs that next step:
evidence surface → claim → scope → remaining gap
Trust Signal helps locate and structure public evidence.
The matrix records what that evidence can actually support in the review.
What it does not prove
The matrix does not prove that a vendor is low risk.
It does not make a source stronger than its scope.
A SOC 2 report can still leave the AI data path unclear.
A subprocessor list can identify third parties without showing actual routing.
A Trust Center can provide useful evidence without resolving the buyer’s use case.
The matrix records those limits.
It does not remove them.
Weak-answer pattern
A common review pattern is:
Claim identified. Supporting document attached. Item closed.
The document may be relevant, but the review file does not record:
which part of the claim it supports;
which product or plan is covered;
whether exceptions exist;
whether another source is required;
when the evidence was checked.
The result is a documentation package without a clear evidence chain.
Evidence request
For material claims, ask for the source and its boundary.
For each material AI data-handling, security, privacy, logging, retention, subprocessor, residency, and human-access claim, identify the supporting source, applicable product and plan, covered data types, known exceptions, evidence date, and contractual basis where applicable.
The goal is not more documents.
It is to identify which claims still lack usable evidence.
Review note
Material vendor claims have been mapped to available supporting sources. Some remain partially evidenced because product scope, data-type coverage, exceptions, evidence freshness, or contractual applicability are incomplete.
That is more useful than a spreadsheet containing only Yes, No, and See Trust Center.
Usage boundary
The matrix should always be read in context.
The same claim may support different use depending on the product, plan, data type, region, contract terms, and evidence date.
If the evidence supports only low-sensitivity internal use, record that.
If customer data requires stronger retention or subprocessor evidence, keep that limitation open.
A claim-evidence matrix is not a vendor score.
It is a dated record of what the available evidence supports and what remains unresolved.
The first nine notes focused on vendor statements and supporting sources.
The same question continues after documentation:
What evidence shows that a documented control actually works as described?
Boundary
The goal is not to approve or reject a vendor.
The goal is to make the evidence gap visible before real data use.
Reply if you want the sanitized sample claim-evidence matrix.
#AIVendorRiskAssessment #ThirdPartyRisk #AIGovernance