A collector queries a source for a configuration object, but nothing comes back.
It is easy to normalize that result immediately as:
configured = false
Sometimes that is correct.
But sometimes the source was never successfully observed in the first place.
API timeouts, permission problems, incomplete pagination, an unstable or unreachable path between the collector and the source, or even a successful request that did not actually cover the full intended scope can all leave the pipeline with “no object returned.”
Those situations are not the same.
If their collection outcomes are collapsed too early, a failed observation can quietly become negative evidence.
Two empty results that mean different things
The latest CodeYourCompliance example models two collection attempts against the same synthetic target.
In the first scenario, we assume the collector successfully reached the intended source and scope. The query completed normally, the result set was complete, and no matching legacy TLS configuration object was found.
The modeled result is:
collection_status = success
result_completeness = complete
observed_state = observed_absent
absence_assertion_supported = true
In the second scenario, the same intended query timed out before the source state was obtained:
collection_status = failed
result_completeness = not_observed
observed_state = unobserved
absence_assertion_supported = false
Neither scenario returns a matching object.
Only the first supports an assertion that the object was absent.
They describe two different observation states, so normalization should not erase that difference.
Absence is still an observation
Negative evidence is easy to describe as “nothing was there.”
That is too loose.
To state that something was absent, the collection path still needs to show what was observed, where it was observed, how it was queried, and whether the observation actually completed.
Not seeing something and confirming its absence are different things.
A bounded absence statement might look like this:
At timestamp T,
through query X,
against synthetic-endpoint-443,
no matching legacy TLS protocol configuration was observed.
That is not the same as:
Legacy TLS is disabled.
The first statement describes an observation.
The second is already a configuration conclusion and may later become part of a control conclusion.
Moving from the first to the second may require more evidence or policy logic.
HTTP 200 is not enough
The first version of the synthetic example exposed an important problem during adversarial review.
It was too easy to read:
HTTP 200
+ empty result
as sufficient proof that the object was absent.
That is too strong.
A successful request can still produce an incomplete observation.
The collector may have queried only the first page.
The authenticated account may not be able to see every object.
A filter may exclude the target.
The query may point to the wrong account, region, or namespace. Some APIs may even return an empty structure when a feature is unsupported.
These responses can look perfectly normal without proving that the intended observation was complete.
The revised synthetic example therefore makes several assumptions explicit. For the observed_absent scenario, we assume by construction that:
the intended source is authoritative for the synthetic condition
the intended scope was covered
the query semantics are defined
pagination is complete
the collecting identity has sufficient visibility over the declared scope
These are scenario assumptions.
They are not runtime validations.
The example therefore does not claim that:
HTTP 200 + empty payload
generally means absence.
Collection status belongs in the evidence chain
The new Observed Absence Evidence Contract keeps collection outcome separate from the later control result.
A minimum record can include:
observation_outcome:
source_ref:
collection_method:
collection_attempted:
collection_status:
query_semantics_ref:
scope_ref:
scope_reached:
response_status:
result_completeness:
observed_state:
absence_assertion_supported:
failure_reason:
observed_at_utc:
The point is not to preserve every transport detail.
The point is to stop materially different collection outcomes from being normalized into the same fact.
At minimum, the pipeline may need to distinguish states such as:
present
observed_absent
unobserved
indeterminate
observed_absent means the observation completed and the query semantics support an absence statement.
unobserved means the intended source state was never successfully obtained.
A timeout belongs in the second state. It should not be converted into false.
A failed observation should stay failed
Suppose an API request times out.
The collector does know one thing: the collection attempt failed.
It does not know whether the configuration object exists.
A more defensible record is therefore:
collection_status = failed
observed_state = unobserved
absence_assertion_supported = false
not:
configured = false
This becomes important once the evidence reaches policy evaluation.
If unobserved is converted into false, downstream logic can no longer distinguish between:
the condition was observed to be absent
and:
we never successfully checked it
And that is how a collection problem quietly turns into a claim about the target state.
This happens before evidence sufficiency
Last week’s control-to-evidence mapping work asked whether all required proof inputs had been assembled before policy evaluation.
This week’s problem happens earlier.
Before the pipeline can ask whether the evidence set is complete, it first has to know what each collection attempt actually produced.
The path is closer to:
source system
-> collection attempt
-> observation outcome
-> raw evidence
-> transformation
-> normalized fact
-> evidence admissibility
-> evidence sufficiency
-> policy evaluation
-> control result
An unobserved outcome may eventually leave an evidence requirement unsatisfied.
An observed_absent outcome may itself become valid evidence for a particular requirement.
But those decisions should happen after the collection outcome has been preserved, not after both states have already been flattened into the same Boolean value.
Observed absence still does not mean PASS
There is another boundary.
Even when observed_absent is recorded correctly, it does not mean the control automatically passes.
Suppose the collector confirms that no legacy TLS configuration object was observed.
Whether that absence supports the control still depends on the control design, source authority, evidence mapping, and policy semantics.
The collection layer should make only the narrower statement it can support.
It should not make the final control decision.
For that reason, the synthetic comparison stops before control evaluation.
At that point it models only the observation outcome.
Public reference artifacts
The implementation references are available in the public CodeYourCompliance evidence-validation-pipeline repository:
The current artifact models collection-outcome semantics through synthetic examples.
It does not yet validate query semantics, pagination completeness, permission visibility, or source authority automatically.
It also does not determine whether an observed absence is sufficient to satisfy a real control requirement.
Origin and scope
CodeYourCompliance
Website: Compliance Evidence Automation | CodeYourCompliance
GitHub: https://github.com/codeyourcompliance
Attribution is requested for forks, references, adaptations and technical discussions.
MAS TRM-inspired means engineering interpretation. This project does not provide legal, regulatory, audit, certification or compliance advice.