AI observability vs inference audit trail: what each one answers

Which question does each one answer?

The quickest way to tell them apart is by the question someone is asking.

Is p95 latency rising this hour?

Observability: metrics and dashboards.

Why did this request fail?

Observability: traces and logs, while they are kept.

Is the model drifting compared with last month?

Observability: aggregated predictions and model monitoring.

Which exact model file produced this result in March?

Inference audit trail: the model hash in the result’s execution record.

What input did it receive, and is the file we have the one it produced?

Inference audit trail: input and output hashes or immutable references.

Where did it run, on which runtime and hardware — and who says so?

Inference audit trail, with each fact marked as reported, observed or verified.

What AI observability is built for

Observability describes the behaviour of a running system: latency distributions, error rates, traces across services, resource use, throughput and, for models, drift and quality signals. It is designed for engineers who need to see a problem quickly and act on it.

Its data model follows that job. Metrics are aggregated, traces are often sampled, retention is usually measured in days or weeks, and records can be enriched or rewritten as pipelines evolve. Those are the right choices for operating a service; they are simply not the properties you want when the question is about one result, months later.

What an inference audit trail is built for

An inference audit trail is one record per execution, written when the model runs and kept with the result it explains. It is designed for whoever must rely on that result later: a customer, an auditor, a reviewer, an engineer investigating a dispute.

• The exact model artifact: name, version and a content hash such as SHA-256.
• The input: an immutable reference or a hash.
• The runtime that executed the model, and its version, when known.
• The environment and hardware: provider, kind of environment, region, device, when known.
• The result: an output reference and hash, the status and the timeline.
• The evidence for each fact: who reported it, who observed it, what verified it.

Where execution provenance fits

Execution provenance is the broader record that links these facts together for one execution: which artifact ran, on what input, in which environment, with which result, and on whose word each fact rests. An inference audit trail is the collection of those records, kept for later questions. Observability can carry some of the same facts as signals, but it does not tie them to one result.

Where they overlap

Both deal with timings, errors and environment labels, and both can travel through the same pipelines. The differences are identity, retention and trust: the audit record identifies the artifact by hash rather than by label, is kept for as long as the result matters rather than for an operational window, and keeps the source of each fact instead of blending them into one metric.

A sensible design shares the plumbing and separates the record: the same execution ID appears in traces, metrics and the audit record, so either side can find the other.

Reported, observed, verified

Dashboards often place a value reported by the infrastructure next to a value measured by the platform, with nothing to tell them apart. That is fine for spotting trends. In an audit record they must stay apart: a provider’s statement about its device is a claim; the platform’s own timestamps and stored hashes are observations; an output checked against a known expected output is a verification. Never promote one to the other.

Example: the same week, two questions

Illustrative example. On Tuesday, latency for an image model doubles. The observability stack shows it within minutes: one region’s queue is backing up, traces point to a slow storage call, the team fixes it. No audit record is needed for that.

In June, a customer disputes one result from that Tuesday. The dashboards have long since rolled up and the traces have expired. The execution record answers directly: model hash, input hash, environment, runtime, timings, result hash, and which of those facts the provider reported and which the platform observed.

How to run both without duplicating work

• Write the execution record from the system that runs the job, at run time — not reconstructed later from telemetry.
• Put the execution ID in traces and logs, so operational data and the record can be joined.
• Let observability consume the record’s timings and outcomes for aggregates; never the other way round.
• Give the record its own retention, tied to how long results are relied on, independent of telemetry retention.
• Protect the record itself (append-only storage, signing) if third parties must rely on it.
• Keep observability for what it does well: alerting, debugging, capacity and quality trends.

What neither one does

Neither proves that a specific chip executed the model; hardware attestation is a separate layer. Neither guarantees that a model can be re-run to the same bits. And neither makes an AI system compliant: logging obligations, such as those for high-risk systems under the EU AI Act, can be supported by good records, but compliance depends on much more than records.

Where Auvryn fits

Auvryn is execution infrastructure for ONNX models that keeps an execution record of the inference audit trail kind for every run it executes. It is not an observability or monitoring platform, and it does not replace one.

• Models identified by SHA-256 at upload, validated before any run, and testable in an isolated sandbox.
• Each run records the model version, the input’s SHA-256 when you provide your own input, the environment and whether the run was simulated, and every result artifact with its SHA-256.
• Provenance in three groups — reported by the provider, observed by Auvryn, verified by Auvryn — with each telemetry value labelled by its source.
• The record is readable in the web app and through the REST API, TypeScript SDK, CLI and MCP server.

What is available to you

The guided sample is simulated and shows a complete run record in about a minute. Uploading, validating and sandbox-testing your own ONNX model is self-service. Runs on real cloud infrastructure are opened for approved pilots only. Edge execution is not offered to external users, and Auvryn does not run workloads in orbit.

From execution records to execution intelligence

Today

An execution record per run, with provenance and sourced evidence.

Next

Expected versus actual results, and evidence that can be compared across environments.

Future

Execution intelligence: using real execution evidence to help decide where and how a workload should run. Auvryn is building toward this; it does not do it today.

Try it

• Start with the sampleCreate an account and run the guided, simulated sample to see a full run record.
• What to record for every model runThe fields of an inference audit trail.
• How to prove which model version produced a resultHashes, a record schema and an implementation checklist.
• Developer docsREST API, OpenAPI document, SDK, CLI and MCP server.

Frequently asked questions