What is provider-neutral AI execution?

Why it matters

Every infrastructure provider has its own way to accept a model, authenticate, start a run, report progress, fail and return results. Without a neutral layer, each new provider is a new integration, and each integration ties the workload to that provider.

A neutral layer keeps the model portable as infrastructure changes: cloud regions, edge devices and, over time, compute in orbit.

What it is not

Not an LLM gateway

LLM gateways route prompts between hosted language-model APIs. Provider-neutral execution runs your own model file on infrastructure.

Not ONNX Runtime execution providers

ONNX Runtime execution providers pick the hardware backend inside one process on one machine. Provider-neutral execution decides nothing inside the model runtime: it gives every infrastructure provider the same contract for accepting, running and returning a job.

Not automatic routing

Neutrality is a contract, not a decision: in Auvryn you choose the provider for each run.

What neutrality requires

• Declared capabilities: each provider states which model formats, operator sets, input types and sizes it accepts, so compatibility is checked before submission.
• Normalized states and errors: every provider’s statuses and failures map to the same small set.
• Idempotent submission: a retry never starts a second run, whatever the provider supports natively.
• Results that outlive the provider: artifacts are copied and checksummed by the neutral layer.
• Provenance with sources: what each provider reports is kept apart from what the layer measures itself.

How Auvryn implements it today

Auvryn has one provider contract and one ExecutionJob lifecycle. A simulated Sample Environment and a real external edge integration (under internal evaluation, not yet offered to customers) implement the same contract. More providers are next, as their access opens.