What happens between your request and an answer

The four steps

Why step 4 is the one that matters

Steps 1 to 3 are conveniences. Step 4 is the difference between a demo and something you can leave running.

Any single provider fails often enough that a workload with real traffic needs a fallback path. Quotas refill on the provider's schedule, not yours. Keys get rotated or revoked. A model gets deprecated with a few weeks' notice. If your application talks to one provider directly, every one of those events is an outage you have to notice, diagnose, and fix by hand.

With a gateway in between, they become a log line. The request still returns an answer, and the log tells you the original choice was skipped and why.

Choosing a route instead of a model

A specific model id is a decision you made once and then stopped revisiting. Model quality moves, prices move, and free quotas appear and disappear. A route is a decision you describe once and let the router keep making.

A route is not a guarantee about which model you get. It is a guarantee about what the router optimises for. See the full list at Models.

What gets logged

Every routed request writes an entry you can read in the console: which route or model was requested, which provider actually served it, which candidates were skipped, and the reason for each skip. That log is why "why did this call cost more than I expected" has an answer rather than a guess.

A request, step by step

The four steps above are the shape of it. This is the same sequence at the level of the individual checks, in the order they run.

1. Step 1: Resolve the name. A specific model id maps to one provider entry and skips the rest of the choice. An auto/ route expands into the candidate set that matches its modality: 6 of the 199 listed entries are modality routes, which is 3% of the catalogue.[1]
2. Step 2: Filter by what the request needs, hardest constraint first. A context window smaller than the request cannot serve it at all, so that is checked first. The modality comes next: an image request is only served by image models, a transcription only by speech-to-text models. Each filter removes a different slice of the list, and the modality removes the largest.
3. Step 3: Order what survives. Free quota sorts to the front, then the weighting the route asks for — cost, speed, or uptime. A route that weights uptime puts providers with a clean recent record ahead of cheaper ones.
4. Step 4: Send to the head of the list and read the response. A normal answer ends the request here. A rate limit, an exhausted quota, an authentication rejection or a timeout removes that candidate and control returns to the list.
5. Step 5: Write the log entry: the route or model that was requested, the provider that actually served the call, the candidates that were skipped, and the reason for each skip.

What makes the router skip a candidate

Failures are not one thing. Each signal below is handled differently, and conflating them is how a fallback path turns into a retry storm.

The reason is recorded per skip, which is what makes the log readable later. Without it, "the answer came from somewhere else" and "the answer cost more than the list price" are the same unexplained event.

What routing does not decide

• It does not invent model settings. A parameter a model's API does not accept is not sent to it.[1]
• It does not upgrade a weak model. Filtering and ordering pick among models that fit the task. If the correct model for the job is not in the catalogue, routing cannot conjure it.
• It does not hide the bill. Failover changes which model served the call, and the price list follows the model, not the route. The log is where the two are reconciled.
• It does not retry forever. The candidate list has an end. When the end is reached, the request fails with the reasons attached.

Routing FAQ

Next

Sources

Every third-party figure quoted on this site was read from the vendor documentation linked below, on the date stated. None of these links are affiliate links and none of these vendors paid to be listed. If a number here disagrees with the page it came from, the page is right — tell us and it gets corrected.

• OpenRouter — credit-purchase fees and free-model rate limits
• OpenRouter — rate-limit reference
• OpenRouter — public model list
• LiteLLM — licence, deployment options and supported models
• Cloudflare AI Gateway — features and pricing
• Vercel AI Gateway — features
• Portkey — plans
• Helicone — plans
• Model Context Protocol — specification
• OpenAI Python SDK — base_url and OPENAI_BASE_URL
• Anthropic Messages API — request format

Where the percentages come from

Every share quoted on this site is counted from the live model list, not estimated. Divide the two counts in the last column, round to the nearest whole percent, and you get the number in the first.

[1] Counted on 29 September 2026 from GET https://freemodel.online/v1/models, which needs no key and no account. Every row uses all 199 listed entries as the denominator, because the only thing the endpoint states for every entry is its modality. It no longer publishes per-model capability flags such as tool calling or reasoning, so this site quotes no shares for them. Fetch the list and recount whenever you want to check the arithmetic.