AI Audit Logs: What Events Are Missing?

Audit logs may exist without recording the AI-specific events needed for incident reconstruction, accountability, or review.

AI Vendor Evidence Gap Notes #7

AI vendors often say:

“Audit logs are available.”

“Enterprise administrators can monitor activity.”

“Logs can be exported to your SIEM.”

These statements sound like evidence of accountability.

They may prove that some logging capability exists.

They do not prove that the events needed to reconstruct an AI workflow are actually recorded.

The review question is not only:

Does the vendor provide audit logs?

The better question is:

Which events are recorded, which events are missing, and can the buyer reconstruct what the AI system accessed, produced, changed, or exposed?

An audit log can exist while the decisive AI event remains invisible.

That is the evidence gap.

Claim

The vendor says:

“Enterprise customers have access to audit logs.”

Or:

“Key platform activity is logged.”

Or:

“Security and administrative events can be exported.”

These statements may establish that the product records certain identity, administrative, operational, or security events.

They do not establish that the actual AI activity is visible.

Why it sounds sufficient

Audit logs are familiar enterprise controls.

They suggest that administrators can monitor activity, investigate incidents, identify configuration changes, and retain evidence for internal review.

A vendor questionnaire may therefore ask whether audit logs exist and treat a “yes” answer as complete.

But conventional SaaS logs often focus on:

• login and logout;

• account creation;

• role and permission changes;

• workspace settings;

• administrative actions;

• security alerts.

An AI workflow may create additional events:

• prompts and model invocations;

• uploaded files and connector access;

• retrieval of documents or records;

• model-provider routing or fallback use;

• tool or agent execution;

• generated outputs and external sharing;

• human review or support access;

• retention and policy-setting changes;

• blocked, failed, overridden, or retried actions.

If those events sit outside the logging scope, the buyer may have an enterprise log without having an AI activity record.

The feature may be real.

The incident-reconstruction capability may still be incomplete.

What it actually proves

A current, product- and plan-specific audit-log statement may establish that:

• an audit-logging mechanism exists;

• certain identity or administrative events are recorded;

• administrators or vendor personnel can review selected activity;

• logs may be exportable through an API or SIEM integration;

• the vendor maintains operational records for security or troubleshooting.

These are meaningful evidence points.

They establish that logging exists.

They do not identify the complete event taxonomy, field coverage, retention period, customer-access model, export capability, or known exclusions.

Availability is not scope.

A public evidence example

The CodeYourCompliance public evidence profile for Decagon records an audit-logging claim among the public enterprise-readiness evidence observed for Decagon’s AI customer-support platform.

The profile is an evidence index.

It is not verification that customers can access the logs or that the logging scope is sufficient for incident reconstruction.

Decagon’s security page states that tamper-protected logs capture key events ranging from logins to unusual activity. The same page says access to those logs is limited to senior engineering leadership.

A separate security and compliance page states that Decagon maintains audit logs of activity, errors, and warnings on production systems.

These statements provide useful evidence that internal logging mechanisms exist.

They do not establish that enterprise customers receive a customer-facing audit trail.

The public materials do not identify:

• the complete event taxonomy;

• which AI-specific actions are recorded;

• whether customer administrators can access or export the records;

• how long the logs are retained;

• whether prompts, connector retrievals, model routes, tool calls, human access, policy changes, exports, and failed actions are included;

• which important events are explicitly excluded.

The evidence therefore supports a narrow conclusion:

The vendor publicly describes internal audit-logging mechanisms.

It does not establish the event-level visibility available to the buyer for the intended workflow.

What it does not prove

An audit-log claim does not automatically establish:

AI event coverage. The log may not record model invocation, prompt submission, retrieved context, file use, generated output, tool execution, or agent action.

Data-access visibility. The buyer may not be able to see which connector, repository, document, database, or customer record was accessed.

Routing visibility. The log may not identify which model provider, region, deployment, fallback route, or integration handled the request.

Human and administrative access. Support access, abuse review, human evaluation, retention changes, permission changes, connector changes, and model-selection changes may not be visible.

Actor attribution. The record may not distinguish users, administrators, service accounts, vendor personnel, contractors, and automated agents.

Outcome visibility. The event may not show whether an action succeeded, failed, was blocked, was retried, or was overridden.

Evidence usability. The buyer may not know whether timestamps are consistent, fields are documented, records are retained long enough, logs can be exported, or administrators can alter or delete them.

A vendor may have strong authentication and infrastructure logs while leaving the customer-relevant AI workflow largely unobservable.

None of these gaps proves that the events are absent.

They show that the available evidence does not establish whether the buyer can reconstruct them.

Weak-answer pattern

This is the feature-without-event-scope pattern.

The vendor confirms that audit logs exist.

The buyer records:

“Audit logs available.”

But the review file does not identify:

• which events are included;

• which AI-specific events are excluded;

• who can access the records;

• how long they are retained;

• whether customers can export them;

• whether the records support incident reconstruction.

The answer describes the feature.

It does not establish the evidence boundary.

Evidence request

Do not ask only:

“Do you provide audit logs?”

Ask for the event taxonomy.

A stronger request is:

Please provide the audit-event taxonomy for the exact product and plan under review, including AI-specific events, administrative events, human-access events, tool execution, data access, routing, outcome status, and known exclusions.

For each event type, identify:

• actor and actor type;

• timestamp;

• action;

• target object;

• success, failure, block, retry, or override result;

• feature, integration, connector, model, or route involved;

• customer-access model;

• log-retention period;

• export method;

• documentation source.

Then ask:

• Are prompts, model invocations, and generated outputs represented as events?

• Are connector access, retrieved sources, file actions, and tool execution visible?

• Are model routing, fallback use, and agent actions recorded?

• Are support access, human review, and administrative changes logged?

• Can customers export the records, and how long are they retained?

• Which relevant events are explicitly excluded?

The vendor does not need to expose model internals.

It does need to show whether the buyer can reconstruct the events that matter for the intended use case.

Review note

A weak review note says:

“Audit logs available.”

A stronger review note says:

The vendor publicly describes audit-logging mechanisms, but the current evidence does not establish customer access, export capability, retention, or coverage of AI-specific events such as model invocation, connector access, routing, tool execution, human review, output sharing, and policy changes. The response is not evidence-complete for incident reconstruction or accountability in the intended workflow.

That note does not say the vendor lacks logging.

It says the buyer does not yet know whether the available logs support the decision being made.

Usage boundary

The required log scope depends on what the AI system is allowed to do.

A drafting assistant used with low-sensitivity internal content may require less event detail than an agent that accesses customer records, executes tools, changes systems, or produces external communications.

Until the relevant event scope is evidenced:

Limit use to workflows where missing audit events would not prevent incident investigation, accountability, or reversal. Do not rely on the vendor’s audit logs for high-impact actions, sensitive data access, autonomous execution, system changes, or external decisions until event coverage, customer access, retention, export, and known exclusions are confirmed.

That is not approval.

That is review preparation.

The review unit is:

vendor + product + plan + use case + data type + region + contract terms + evidence date.

Audit-log availability can support that review.

It cannot replace event-level scope.

A log that omits the decisive event cannot support the conclusion the buyer expects from it.

Boundary

This material is for evidence structuring and review preparation. It does not provide legal, regulatory, audit, procurement, certification, or implementation advice.

The goal is not to approve or reject a vendor.

The goal is to make the evidence gap visible before real data use.

Reply if you want the sanitized sample evidence gap memo.

#AIVendorRiskAssessment #AuditLogs #AIGovernance