A collector runs against a system.
The collection partially fails.
The output file still exists.
A downstream policy engine reads the file, cannot find the expected field, and returns fail.
The report now says the control failed.
But the evidence may have failed instead.
That distinction has to survive the pipeline.
If it does not, compliance automation can manufacture control failures from collection failures, integrity failures, or invalid evaluation context.
The audit problem starts before the policy engine.
A file is not automatically admissible evidence
A common pipeline looks like this:
collect
↓
evaluate
↓
pass / fail
↓
reportThe hidden assumption is that collection produced usable evidence.
That assumption is unsafe.
An artifact can exist while still being unusable. Its schema may be invalid, its integrity check may fail, its freshness window may have expired, or the evaluation context may be malformed.
None of those conditions proves that the control failed.
They mean the system lacks a defensible basis for a control decision.
That requires an explicit gate.
Evidence admissibility is not control evaluation
The pipeline should answer two different questions.
First:
Is this evidence admissible for evaluation?
Then:
Does the observed condition satisfy the control policy?
Those questions should not share the same result field.
The CodeYourCompliance reference implementation uses this sequence:
evidence schema gate
→ context schema gate
→ integrity gate
→ freshness gate
→ derived facts
→ decision evaluation
→ assessment resultIf an earlier gate fails, the control decision does not execute.
The state remains:
decision_executed = false
control_status = unknownThat is not a failed control.
It is a blocked decision.
Invalid evidence should stop the pipeline
Take an integrity mismatch.
The evidence package no longer matches the sealed object the pipeline expected.
The wrong transition is:
invalid_evidence
→ control_status = failThe correct transition is:
invalid_evidence
→ decision_executed = false
→ control_status = unknown
The policy engine has nothing admissible to evaluate.
Returning fail would create a control conclusion that the evidence does not support.
The implementation rule is simple:
No admissible evidence.
No control decision.Stale evidence uses the same boundary
Freshness is a different failure mode, but the same gate applies.
Evidence may still have a valid integrity hash and known provenance while being too old for the current evaluation window.
That evidence is not corrupt.
It is stale.
A freshness policy may classify it as stale_evidence, but that classification belongs to evidence admissibility.
It is not the downstream control decision.
This is why the public example separates:
admissibility_evaluationfrom:
control_decisionThe control decision remains unexecuted and unknown.
Evidence evaluation is not control evaluation.
OPA is not the boundary
OPA can evaluate either kind of policy.
A Rego freshness policy can decide whether evidence may proceed. A control policy decides whether admissible facts satisfy a declared condition.
The tool is not the point.
The stage is.
Those decisions should remain separate even if both are implemented in OPA.
The state transition should be explicit
The useful rule is not a longer list of statuses.
It is a prohibited transition.
This should never happen:
invalid_evidence
→ control_status = failNeither should this:
stale_evidence
→ control_status = failThe allowed path is narrower:
admissible evidence
→ decision executed
→ pass / failEverything before that belongs to evidence qualification.
fail means the control policy evaluated admissible evidence and found the condition unsatisfied.
unknown means the pipeline could not reach that judgment.
Reports should preserve the blocked decision
A reporting layer should not translate every non-pass state into non-compliant.
It should preserve the failed gate, evidence status, whether the decision executed, and the control status.
Reports summarize a decision path.
They should not invent one.
MAS TRM-inspired means engineering interpretation
This is not a claim that MAS TRM prescribes this state machine, OPA, Rego, SHA256, or this implementation.
The engineering question is narrower:
Can the evidence pipeline distinguish failure of the control from failure of the material used to assess it?
If not, the final report may look deterministic while its underlying decision path is not.
A failed control is a control conclusion.
Invalid evidence is the absence of a defensible basis for that conclusion.
That is the boundary.
Public reference artifacts
The admissibility boundary described here is implemented in the public codeyourcompliance/evidence-validation-pipeline repository.
The main reference package is:
The TLS example separates evidence validation from control decision evaluation. Schema validation, integrity verification, freshness checks and evaluation-context validation occur before the control decision.
If a gate fails, the decision does not execute. The result remains unknown, not fail.
These examples do not prove that the underlying system is compliant. They show whether the pipeline had a defensible basis to make a control decision at all.
They also stop before remediation. Certificate renewal, configuration changes and other corrective actions belong to a separate workflow with separate permissions and evidence.
Origin and scope
CodeYourCompliance
Website: https://www.codeyourcompliance.com/
GitHub: https://github.com/codeyourcompliance
Attribution is requested for forks, references, adaptations and technical discussions.
MAS TRM-inspired means engineering interpretation. This project does not provide legal, regulatory, audit, certification or compliance advice.