Providers & routing

A model ID can be available from several providers. In the request you choose the model, not the provider. The gateway picks the provider for each request and names it in the response in the X-Noirdoc-Provider header.

Eligible providers

A request can go to the providers that offer the requested model and meet every condition:

• The model fits the endpoint. A chat model, for example, cannot be called through /v1/embeddings. The provider’s API format must fit as well: you call models from providers in the Anthropic format through /v1/messages, all others through the OpenAI-compatible endpoints.
• The provider is approved for professionals bound by secrecy if your organisation is classified as such.
• The provider meets the key’s restrictions: allowed providers, data residency, CLOUD Act and § 203.
• The model is one of the key’s allowed models, if the key specifies any.
• The model and provider are included in starter credit if your organisation is still using only its starter credit. When several providers offer the same model ID, the gateway skips non-starter providers while on starter credit and answers with 403 starter_model_not_allowed only if no eligible provider remains.

Your own provider keys (BYOK) take precedence: if your organisation has set up its own provider for the model ID under Models → Providers, the gateway considers only your own providers for this model ID. Providers managed by Noirdoc are eligible only if your organisation has no provider of its own for the model ID and managed models are enabled for it.

The gateway checks the conditions in order. If no provider is left after a condition, it answers with that condition’s error:

How to restrict models and providers for a key is shown in Restrict models & providers.

Provider order

The gateway sorts the eligible providers by these rules. The first rule that makes a difference decides:

1. Your own providers (BYOK) before managed providers.
2. Providers with a complete price for the model before providers without a price.
3. The cheaper provider before the more expensive one. The gateway compares a weighted price per 1M tokens: three times the input price plus once the output price. The weighting reflects typical requests, which contain much more input than output. Cache prices do not count here.
4. At the same price, the provider with the higher priority. Noirdoc sets the priority per provider.
5. At the same priority, the provider that has been in the catalogue longer.

The order is fixed and does not depend on chance. The gateway sends the request to the first provider in the order. If that provider has no free capacity, the request goes to the next free provider in the failover order, see Capacity. If capacity is free everywhere, the same request always goes to the same provider as long as the catalogue is unchanged. GET /v1/models lists every model ID your key may call exactly once. The owned_by field shows whether the first provider is your own (personal) or one managed by Noirdoc (platform).

Failover

Failover means: if a provider fails, the gateway falls back to the next provider in the order. The gateway tries at most three providers per request. For failover, it orders the remaining providers by priority first, then by price. The next provider may therefore be more expensive than a later one.

It switches provider on these errors:

• The provider is unreachable, or the connection times out.
• The provider answers with status code 401, 402, 429 or a 5xx status code.

If a managed provider answers with 429, the gateway skips it for the seconds from its Retry-After, limited to 1 to 60 seconds (10 seconds without a usable header, 60 seconds when the quota is exhausted), and tries the next provider. Once the pause has passed, it can return to the skipped provider. Your own providers (BYOK) are never paused.

Any other 4xx status code is the provider’s answer to the request itself. The gateway reports it with the provider’s message as provider_rejected_request and tries no further provider.

The gateway switches only as long as your application has not yet received any part of the response. It decides based on the status code the provider sends with the headers of its response. If a stream breaks off after that, the gateway no longer switches.

Two rules limit failover:

• The gateway never switches from your own provider (BYOK) to a managed provider. Otherwise Noirdoc would charge model costs even though you wanted to use your own provider key.
• The gateway never switches from a provider with a price to one without a price.

If the last provider fails too, the gateway reports the error in the uniform error object. A capacity problem (status code 429 or 402) becomes 429 resource_exhausted, an overload (503 or 529) becomes provider_overloaded with status 503 on OpenAI endpoints and 529 on /v1/messages and /v1/skills, both with Retry-After, and any other provider error becomes 502 provider_error. On routes with your own provider key (BYOK), the provider’s status codes 401, 402, 403 and 429 arrive as provider_rejected_request with the provider’s status code and message, as described under Error codes. If the gateway could not reach the provider at all, it answers with:

Noirdoc Shield inspection runs once before the first attempt. Every further provider receives the same inspected and sanitised request.

Capacity

Noirdoc can set for each provider how many requests it receives at the same time and per time window, so that no provider is overloaded. The gateway counts the requests it sends to each provider.

If the first provider in the order is at its limit, the request goes to the next free provider in the failover order. If all providers of the model are at their limit, the gateway waits up to 10 seconds for a free one. Then it answers with 503 capacity_exhausted. Retry-After gives the seconds until a provider is expected to be free again. This applies as long as the gateway has not yet sent the request to any provider. Otherwise it reports the last provider’s response as described above.

The OpenAI and Anthropic SDKs wait the time from Retry-After and send the request again on their own.

Provider in the response

For requests with a model ID, the response contains the X-Noirdoc-Provider header. It gives the short name of the provider that answered the request. After a failover, this is the provider that answered last. An error response also carries the header if a provider was involved. In streams it is missing when the provider answers only after 15 seconds; a comment line : provider=<slug> then names the provider, see Streaming.

curl -i https://api.noirdoc.de/v1/chat/completions \
  -H "Authorization: Bearer $NOIRDOC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

HTTP/2 200
content-type: application/json
x-noirdoc-provider: aki-io

In the SDKs you read the header from the raw response. The OpenAI SDK guide shows this for Python and TypeScript.