Audit evidence for AI agents

What will an auditor actually ask?

An audit of AI agents usually follows a familiar pattern. The auditor asks for the inventory. They sample some agents. For each one, they ask for the owner, the purpose statement, the current permissions and the last access review. Then they pick an action and ask you to trace it.

The trace is where weak evidence shows. Can you say which agent acted, for which person, using which tool, under which policy, and whether a human approved? If any link is missing, the auditor records a finding.

The register weights audit evidence at 15 of 100 points. It is also the area most likely to decide whether a governance programme passes its first review.

What makes a good per-action log?

Control GOV-13 asks for every agent action to be logged with five fields.

• The agent: its unique ID from the inventory.
• The person: who the agent acted for, or who started the session.
• The tool: what was called, with key parameters.
• The decision: allowed, sent for approval, approved, refused or denied, with the policy that decided.
• The time: precise and in one time zone.

Two habits make the log much more useful. Log denied and refused actions, not only allowed ones. A pattern of denials is often the first sign of a misbehaving or hijacked agent. NewCore treats denied attempts as a threat signal for this reason. And link sessions, so an auditor can see the sequence of actions in one task, not isolated rows.

Why does the person matter so much?

Logs that name only a service account cannot answer the auditor's main question: which person is accountable? Agents often act on behalf of someone. That person's identity, and the agent owner's, must be recoverable from each entry.

This is what the register's sources call chain of custody or linking actions to the initiating person. Cyera Identity describes a chain of custody from the human to the actions. Silverfort describes an immutable audit trail linking each action to the person who started it. NewCore's gateway identifies the agent and the person behind it on every tool call.

What other records count as evidence?

The action log is the centre, but auditors will ask for several surrounding records.

• The inventory entry and owner history (GOV-01, GOV-02).
• The purpose statement and when it was approved (GOV-03).
• Approval records, including refusals and timeouts (GOV-07).
• Change tickets for permission and policy changes (GOV-20).
• Policy test results before go-live (GOV-09).
• Completed access reviews with owner decisions (GOV-15).
• Decommissioning records with dates (GOV-12).
• Incident reviews (GOV-19).

Keep these linked to the agent ID. An auditor should be able to start from any record and reach the others.

Why send agent logs to the SIEM?

Security teams investigate incidents in the SIEM. If agent activity lives only in a vendor console, investigations miss it. Control GOV-14 asks you to send agent activity and policy decisions to the SIEM in a structured format, close to real time.

Aembit logs every access event with SIEM export. Clutch Security sends alerts to SIEM, SOAR and ticketing tools. For other platforms, ask the vendor for the export format and delay, and test it before you rely on it.

Check that fields arrive parsed. An agent ID buried in a free-text message is hard to search. Agent, person and decision should be separate fields.

How long should evidence be kept?

Set a written retention period that meets your legal and contract duties (GOV-23). For high-risk AI systems, the EU AI Act sets at least six months for automatically generated logs. That applies to providers under Article 19 and to deployers under Article 26. Many audit regimes and contracts expect longer, often a year or more.

Retention also means integrity. Evidence that can be edited after the fact is weak evidence. Use write-once storage or a log store with tamper evidence, and restrict who can delete.

How do you answer a sample quickly?

Practise before the audit. Pick five random actions from last month and trace each one end to end. Time it. If any trace takes more than a few minutes, find the missing link and fix it.

A good trace answers these questions in order. Which agent? Who owns it? What is its purpose? Who did it act for? What did it call? Which policy decided? Was a human involved, and who? Was the agent's access reviewed recently? Each answer should come from a system of record, not from someone's memory.

How do the platforms compare on evidence?

Audit evidence scores in the register range from five to ten. Cyera Identity scores ten for a prompt-level audit trail and regulator-ready evidence. NewCore, Silverfort and Aembit score nine. NewCore keeps tool calls, sessions and approvals in one trail. Silverfort links each action to the person who started it. Aembit logs every access event with SIEM export. Descope scores five because audit export is not detailed in its public material.

A high score means the vendor describes strong evidence publicly. It does not replace your own test. Ask each shortlisted vendor to trace a sampled action live.

What should the board see?

Boards do not need raw logs. They need a few consistent measures each quarter: agents in the inventory, the share with owners, approvals and denials, reviews completed on time and incidents. Each measure should name its system of record, so the board knows the number comes from evidence. The board one-pager template sets this out.

What mistakes should you avoid?

Three mistakes recur. Teams log only allowed actions, so the denials that reveal a problem are missing. They keep evidence in a vendor console that expires before the audit window closes. And they cannot join agent logs to identity logs because the agent ID differs between systems. Agree one agent ID across every system on day one, and test retention before you need it.

Where to go next

The audit evidence ranking reweights the register toward this area. The control pages for GOV-13, GOV-14 and GOV-23 list which platforms document each, with sources. The agent incident runbook shows how to preserve evidence before cleanup.