We do not retry
your typo.

A model name has to become a destination first.

Selection is an ordering, not a preference. A model served by your own pool is used first, because a machine you are already paying for by the hour adds nothing per token. Below that sit the provider accounts you configured, cheapest tier that fits the request. Below those sits whatever alternate route you nominated — and nothing else.

A model with no destination at all is rejected at selection, before a single upstream call is made, and the rejection names the other entry point — the one that provisions hardware on demand — instead of guessing a provider. That is a smaller detail than it looks. The alternative is a platform that accepts the request, tries something plausible, and bills you for discovering that it was never going to work.

The chain reads the failure before it decides.

Failures are not interchangeable. A provider that is rate-limiting you is telling you something about itself — its quota, its capacity, this minute. A provider that rejects your request body is telling you something about the request, and every other provider on the market will tell you the same thing in the same second.

Cascading the first class of failure produces an answer. Cascading the second produces an identical rejection from a different vendor, arriving later, wrapped in a message that no longer says clearly who objected. Two of those calls are billed; one of them was never capable of succeeding.

The order below is the entire mechanism. It runs once per request — it is a decision, not a loop. Three of its six outcomes are a deliberate stop.

1. Refuse — the request itself was rejectedA malformed body, a credential the provider will not accept, or a model that does not exist on that account. None of these change when the destination changes. The chain returns the upstream's own error text immediately, names the provider that produced it, and makes no second call.
2. Act — the credential expiredA rotated key is solvable at this layer rather than by the caller. The credential is re-read, the connection to that provider is rebuilt in place, and the request goes again to the same destination. Nothing restarts and no operator is involved.
3. Act — the provider is rate-limitingA quota belongs to one account on one provider. A different account elsewhere has its own budget, which makes this the class where a second call has a genuinely different chance of succeeding rather than a hopeful one.
4. Act — the provider is unavailableServer errors and gateway timeouts say the upstream is unwell while the request is fine. Same reasoning, same single attempt on the alternate — and the response records which of the two served it.
5. Refuse — the alternate resolves to the same upstreamAn alternate that lands back on the provider that just failed is not an alternate. The chain recognises the collapse and skips the call instead of spending money to confirm it.
6. Refuse — no alternate was configuredThen the request fails, and says so, naming the provider and the class of failure. Traffic is never moved onto a route you did not choose, and no charge appears from a vendor you never selected.

A route you did not choose is not a fallback.

Every alternate in this chain is one you put there. There is no default aggregator quietly absorbing your overflow, because a platform that picks your destination has also picked your price, your data path and your jurisdiction. If you have configured nothing, a provider outage fails your request — and tells you which provider, which class of failure, and that exactly one attempt was made.

Turning fallback on is one piece of configuration, and it is reversible. What it is not is the default. That asymmetry is the whole design: the failure mode of an unconfigured chain is a visible error, not an unexplained line on an invoice from a supplier you never signed up with.

Two calls, or five.

A bounded chain has a ceiling: the selected provider, and at most one alternate. That ceiling is what makes the cost of a request predictable enough to price. It also keeps the failure legible — when both attempts fail, the error names both, in order, with the class each returned.

An unbounded chain has neither property. Retry five times across three vendors and the availability graph improves, because eventually something answers. What the graph does not show is that the tail latency is now the sum of five timeouts, that requests which were never going to succeed were billed four extra times, and that the error finally surfaced is a summary of five different objections rather than the one that mattered. Availability measured that way is a number optimised against the person paying for it.

The chain here is deliberately worse on that graph and better on the invoice. The stops are not gaps in the design; they are the design.

Machinery, part three of six.

Each page is one mechanism: what it decides, what it refuses, and why the obvious simpler version costs money. See all six →