Are audit logs without request text enough for a PII leak?

Audit logs without request text are usually enough to establish the chain of actions: who sent the call, which key they used, which policy applied, where the request went, and how the access check ended. They cannot reconstruct the exact national ID number, phone number, or diagnosis. If an investigation must answer the narrower question, "Which personal data left the system?", metadata alone is not enough.

That is not a reason to record request bodies all the time. Such a log often creates a second leak next to the first one: more people can access logs, retention is longer, and copies spread across a search index, SIEM, and backup archives. I prefer to separate three jobs in advance: correlate events, prove that a control ran, and, when necessary, match a specific secret to an event without storing the entire request in plain text.

Metadata proves an event, not its content

Without text, you can prove only what appears in separate fields. A record such as 200 POST /chat/completions is nearly useless: it does not link the call to a person, show the route actually used, or say anything about masking. A good record answers six questions independently of the body: who acted, through which authority, which policy ran, where the request went, when it happened, and which control allowed or stopped the operation.

Teams often blur observability and auditing here. Metrics show rising error rates or latency. Tracing connects work across services. Auditing records an action by a subject and the control's decision in a form that can withstand internal review. The same trace_id can help in all three layers, but it does not turn a technical span into evidence by itself.

The boundary is straightforward. Metadata can establish: "User 42 used the key with fingerprint k_7f3a; policy pii-kz-v4 found one national ID, replaced it with a token, and then the gateway sent the call to the local pool and received a response." It cannot establish which national ID the policy found. Put that distinction into the incident response plan before an incident, or investigators will ask the log for an answer it cannot provide.

A count of detected entities does not prove the absence of PII either. A zero result means only that a specific detector version found nothing under a specific configuration. It is evidence that the control ran, not a certificate that the text was clean.

Store user identity and key identity separately

The minimum record contains a stable actor_id and an API key fingerprint. The user identifier names the human or service owner of the action, while the fingerprint shows which authority was exercised. If you retain only one, the investigation will end either at a shared technical account or at a key that has already been deleted from the live database.

Do not log the secret itself, even as a fragment that makes guessing easier. Calculate an irreversible fingerprint when issuing the key, for example an HMAC of the complete key with a separate server secret, and store it beside the ownership record. A regular fast hash is weak for tokens with low entropy or a known prefix, while HMAC prevents offline guesses without the secret key. Send a short key-record identifier and the full fixed-length fingerprint to the audit stream.

Do not calculate the ownership link only when an investigation begins. A user may leave, a service account may be renamed, and a key may be reissued. The event therefore needs the actor_id, actor_type, tenant_id, and key_id as they existed when the call happened. Avoid duplicating names and email addresses: they change and are personal data themselves. Keep historical mappings in a protected access-management registry.

It helps to distinguish the initiator from the delegate. A call from a background agent can carry the service as its actor_id and the user who started the job in on_behalf_of. Without the second field, ten thousand automated requests will look like the actions of one faceless process. When there is no delegation, store an explicit null rather than letting the field disappear unpredictably.

For compromise investigations, add auth_context: the authentication method, session identifier, verification result, and reason code. Do not use an IP address as a substitute for identity. NAT, mobile networks, and proxies make IP a weak attribute, although a separate normalized source field remains a useful signal.

A policy version matters more than "masking enabled"

The flag masking=true proves almost nothing. An investigator needs the policy identifier, its immutable version, the enforcement mode, and the result. A policy with the same name may have detected national IDs yesterday and phones today; its mode may have observed text without changing it.

Record at least policy_id, policy_version, policy_digest, enforcement_mode, and policy_decision. The identifier locates the rule, the version points to a configuration snapshot, and the digest exposes a silent file edit under the same version number. The mode should distinguish monitor, redact, block, and bypass. Keep the decision to a controlled set such as allow, deny, allow_after_redaction, and error.

Store detector results separately: its version, the entity types found, the count for each type, and the action. An example without source text looks like this:

{"policy_id":"pii-kz","policy_version":4,"policy_digest":"sha256:8c...","enforcement_mode":"redact","detector_version":"ner-12","findings":{"iin":1,"phone":0},"action":"tokenize","decision":"allow_after_redaction"}

This object answers which control ran without exposing the national ID. It also exposes a serious configuration error: findings.iin=1 together with enforcement_mode=monitor means the system saw the data and deliberately let it pass without masking.

A policy is sometimes bypassed under an approved exception. Do not hide that inside a generic allow. Add the exception_id, the exception owner, and its expiry time; the approval system can retain the full justification. The audit record needs only an internal identifier. For an emergency bypass, store a separate reason code and the fact that stronger access controls applied.

A detector error is not a clean result. If the masking service does not respond, the event needs policy_decision=error and a specific failure_code. Gateway configuration then determines whether the request fails closed. In systems that handle sensitive data, I choose denial because allowing a request with an unknown result turns a failed check into a leak path.

Never reconstruct the actual route from old configuration

The requested model and the actual destination often differ. A router may choose another provider because of availability, a limit, region, price, or a residency rule. The audit event must therefore record the routing decision itself instead of asking an investigator to reproduce an old configuration.

The minimum set includes requested_model, resolved_model, route_id, provider_id, endpoint_class, processing_region, and residency_policy. If a request made several attempts, one final row is insufficient. Create a child event for every attempt, all sharing a request_id, with a sequential attempt_no and an individual result. A failed first transmission to an external provider then cannot disappear behind a successful retry on a local model.

Do not log a provider's full address with parameters if it may carry secrets. A normalized provider_id and endpoint class usually provide enough context. A cluster identifier and zone can help for private deployments, but a node name rarely adds audit value and sharply increases cardinality.

Record a residency rule as the applied decision, such as kz_only:v3, plus a result of matched or violated. A region field supplied by the application is weaker than one written by the boundary gateway: the application may be wrong or unaware of a retry route. Every material field needs one unambiguous source.

AI Router fits this design as the single point where a route to a provider or local model can be connected to PII masking and an audit event. A gateway does not repair a poor log contract, though: the team still has to define the fields, versions, and access rules in advance.

Time needs boundaries, clocks, and event order

One timestamp is insufficient when a request passed through a check, queue, routing, and several attempts. Store received_at, policy_checked_at, dispatched_at, and completed_at in UTC at the precision the platform actually supports. Measure durations with a monotonic clock inside the process because system-clock correction can make the difference between two wall-clock timestamps negative.

Every record needs a unique event_id, a stable request_id, and a distributed trace_id. W3C Trace Context defines how to carry traceparent between services, but it does not promise that this identifier is authenticated or suitable as the only audit key. Accept incoming trace context for tracing, issue your own request_id at a trusted boundary, and do not let the client replace it.

A local sequence_no can lock down the order of events for one request. It helps when batch delivery changes row order in storage. Across services, do not pretend that you know an absolute sequence down to the microsecond. Retain the known clock-synchronization error and rely on causal identifiers: a parent span, attempt number, and a reference to the preceding event.

Capture the receive time before parsing the body, or a parser crash will leave the call without a start. Write a completion for every outcome, including a client timeout and an internal exception. If the process dies before the final record, a separate delivery control should mark the request as unfinished. The absence of completed_at then becomes an observable state rather than an ambiguous gap.

Retention is also part of the evidence model. If events live for seven days, an investigation on day eight cannot reconstruct the chain. Select retention based on detection time and internal obligations, state it in policy, and confirm deletion with its own log-management event. Do not promise investigation coverage beyond the shortest-lived part of the chain.

The access result must explain the decision

An HTTP status describes the client response, not the control's decision. A 403 may come from the gateway, provider, or application; a 200 may carry a model response that refuses the task. An access audit needs separate access_decision, decision_source, reason_code, and evaluated_permissions fields.

Reason codes should be stable and machine-readable: KEY_REVOKED, TENANT_MISMATCH, MODEL_NOT_ALLOWED, RESIDENCY_BLOCK, RATE_LIMIT, or POLICY_SERVICE_ERROR. You may add a human message, but do not build investigation queries on it because the wording will change in a later release. Never put part of the source request or token value in the message.

Store the evaluated permissions as a compact snapshot, for example llm.invoke, model.external.use, and pii.redaction.required, together with the role-set version. A bare allow is not enough after an administrator changes a role a month later. The snapshot need not copy the complete IAM policy, but it must reveal which rule granted access.

Store the final call outcome separately: the status class, normalized error code, input and output token counts, and whether the client canceled. Sizes help with anomaly searches but are not content fingerprints. Do not decide that two requests were identical just because their lengths matched.

Write a denial event before returning the client response, through a channel that does not depend on the same failed component. Otherwise an overloaded policy service will both mishandle checks and lose records of its own errors. A local queue with delivery acknowledgements and a lost-event counter is more practical for high-impact decisions.

A content fingerprint helps only with a clear threat model

A team can match a known PII value to an event without retaining the body if the log contains a protected fingerprint of the normalized value. For example, the detector extracts a national ID, converts it to a canonical form, and calculates an HMAC in a separate key domain. During an investigation, an authorized process repeats the operation for a known ID and searches for a match.

This is not a general-purpose hash of the whole request. A space, field order, or system instruction changes the body hash even when the personal data stays the same. A regular SHA-256 of a national ID is dangerous: the set of possible values is bounded, so an attacker with the log can enumerate candidates. HMAC reduces that risk if its secret stays outside the log system and access to the comparison operation is audited too.

A minimal finding record looks like this:

{"entity_type":"iin","fingerprint":"hmac-sha256:v2:ab...","normalization":"iin-digits-v1","key_version":2,"match_scope":"tenant"}

key_version supports rotation, while match_scope prevents an accidental search for one person across all tenants. The fingerprints remain sensitive because they create a stable connection between events. Restrict access, assign separate retention, and do not send them to ordinary developer dashboards.

A fingerprint cannot provide context around the value. If investigators need that context, use a separate evidence store: selective recording under a strict trigger, encryption under a separate key, dual approval for access, and a short lifetime. Do not enable such a store permanently "just in case." It should collect exactly the material allowed by the investigation policy.

I also advise against encrypting every body and declaring the problem solved. An encrypted archive remains a personal-data archive and requires key management, deletion of copies, and an account of every read. It is justified when a legal and operational scenario requires reconstruction, not because a team failed to agree on audit fields.

The minimum event contract must be testable

A good minimum set fits into one versioned object. Here, "minimum" means that removing any field would change at least one conclusion in an investigation. It does not mean a short line without context.

{"schema_version":3,"event_id":"evt_01...","request_id":"req_01...","trace_id":"4bf92f...","sequence_no":4,"received_at":"2026-07-27T10:15:12.184Z","completed_at":"2026-07-27T10:15:12.941Z","actor":{"tenant_id":"t_17","actor_id":"u_42","actor_type":"human","on_behalf_of":null},"credential":{"key_id":"key_9","fingerprint":"hmac-sha256:7f3a..."},"auth":{"method":"api_key","decision":"allow","reason_code":"ROLE_MATCH","role_set_version":8},"pii":{"policy_id":"pii-kz","policy_version":4,"policy_digest":"sha256:8c...","mode":"redact","detector_version":"ner-12","findings":{"iin":1},"decision":"allow_after_redaction"},"route":{"requested_model":"model-a","resolved_model":"model-b","route_id":"kz-local","provider_id":"local-pool","processing_region":"kz","residency_policy":"kz_only:v3"},"result":{"status_class":"2xx","error_code":null,"input_tokens":812,"output_tokens":144}}

Test the contract with automation, not a document review. A test should send an allowed call, a blocked call, a detector failure, and a retry route, then confirm that each path produced a record with all required fields. Another test deletes or renames a key after the event and verifies that its former owner can still be resolved through protected history.

Test negative conditions too. The log must not contain Authorization, the source prompt, the complete model response, an email address, or an extracted PII value. Run a secret scanner on the log stream before indexing. If it triggers, quarantine should stop the record from spreading to secondary systems.

Schema versioning is mandatory. Add fields compatibly and issue a new version when meaning changes. OpenTelemetry carries trace context and technical attributes well, but its semantic conventions do not replace your agreement about identity, policy decisions, and retention. These fields belong to your threat model, so the security owner must approve them explicitly.

Log integrity and access are part of the evidence

A complete field set is useless if an application administrator can silently modify or delete rows after an incident. The log should leave the production boundary through a separate channel, and its storage should support appending without normal record updates. Permission to submit an event must not automatically grant permission to read accumulated data or change retention rules.

Tamper protection is more than an immutable label. A hash chain inside an ordered batch can help: every event contains the previous hash, and a separate service signs the batch root. Periodically retain the signed root in another administrative domain. A deleted or edited row will then leave a verifiable break, although this design cannot prove that the application originally submitted truthful fields.

The source determines the weight of a record. An actor_id supplied by an untrusted client is not an established identity. The gateway should derive it from verified authentication context. The policy service writes the masking result, while the component that opens the outbound connection records the actual provider. Document the producer of every material field and the boundary where it becomes trusted.

Reading the log creates leak risk as well. Separate roles for metadata search, HMAC-fingerprint matching, and access to protected bodies. An ordinary operator can search by request_id and decision codes but should not see stable fingerprints for people in other tenants. A large-range export needs separate permission and a stated access reason.

Record every search and export in a second audit stream: who ran the query, under which case, which fields they requested, how many rows they received, and where the file went. Do not copy the results themselves into this meta-log. Otherwise access control on the first store can be bypassed through the less protected query history.

Availability matters as much as immutability. For asynchronous delivery, monitor the last acknowledged queue offset, the age of the oldest event, and the number of records in quarantine. An alert for missing audit events should fire independently of an application alert. Product failure cannot excuse an invisible log gap during the same period.

A backup must preserve access restrictions and deletion periods. I have seen designs where the primary index was correctly purged after a month while an unrestricted archive lived for years in an infrastructure account. Investigation capability barely improved, while the amount exposed by a possible leak grew. Test deletion by restoring a test copy and searching for an expired event_id.

Regularly compare the number of accepted requests with the number of terminal audit events per tenant and time window. The counts will not always match one for one because of early denials and retries, so compare expected state transitions rather than just totals. An unexplained gap means the investigation has already lost part of the chain even if nobody has noticed an incident.

Separate access to clocks, collector configuration, and the signing key too. An application operator should not be able to change policy, stop export, and re-sign an event batch at the same time. Where full separation of duties is impossible, require independent approval for these operations and notify the security owner immediately. The management log should retain the old and new values, author, approver, and related change number.

Run integrity checks on a schedule and on every investigation export. The check should recalculate the chain, verify batch signatures, find sequence gaps, and compare actual retention with declared retention. A report with no error still does not prove completeness, so show expected and received event counts beside it for each source. This control turns "we have logs" into a testable state.

Do not send raw audit events directly to every analytics system. First separate fields for operational dashboards from fields for investigations, then give each system its own view. Status counts and latency can be aggregated, while key fingerprints, subject identifiers, and exception details stay in the protected index. Fewer copies with persistent identifiers make it easier to prove who could read them.

Rehearse the investigation before the leak

Suppose the security team receives a national ID found in an external provider's response and a twenty-minute time range. The investigator first calculates the approved HMAC fingerprint of the known value and searches for matches only in the relevant tenant. If no fingerprints exist, they filter events by the iin finding type, time, external routes, and allow or allow_after_redaction decisions.

They then group results by request_id, expand attempts by sequence_no, and check four connections: the key owner at that time, the policy version, the actual provider, and the source of the access decision. A storage query may look like this:

SELECT request_id, actor_id, key_id, policy_version,
       provider_id, processing_region, decision, reason_code
FROM audit_events
WHERE tenant_id = :tenant
  AND received_at >= :from_utc
  AND received_at < :to_utc
  AND pii_types @> ARRAY['iin']
ORDER BY request_id, sequence_no;

The result narrows the candidate events, but without a fingerprint or protected body it cannot establish which one contained the specific national ID. The report should separate confirmed facts, likely conclusions, and unknowns. It may confirm, for example, that three calls detected a national ID and one went to an external provider in monitor mode. The leak probably relates to that call. The exact content remains unknown.

A rehearsal usually exposes an unpleasant gap: the allow event exists, but the route record was lost during a retry; the policy version points to configuration that has already been overwritten; one service clock lags; or a deleted key can no longer be linked to its owner. Finding this in a training dataset is cheaper than explaining it after a real incident.

The rehearsal should end with an evidence matrix, not a presentation. For every question, list the field, system of record, retention period, role with access, and expected gap. If "Which exact value left?" is marked unprovable, leadership can knowingly accept the risk or approve a fingerprint store. Discovering that boundary only after a leak is the bad outcome.

Store request bodies only where a specific obligation requires reconstruction and the team can protect the entire lifecycle of the copy. Detailed metadata, immutable policy versions, and controlled HMAC fingerprints are better for most investigations. Such a log avoids unnecessary knowledge while answering questions about the user, key, check, route, time, and decision without guesswork.