Models and routing
Model IDsDirect link to Model IDs
Models use provider/model IDs, for example openai/gpt-6.1-sol or anthropic/claude-sonnet-5.5. Browse IDs in the model catalog, or call GET /v1/models to list every ID available to your key. A model is listed in the catalog only when Gateway has a published price for it.
From a Mastra agent, add the mastra/ prefix: mastra/openai/gpt-6.1-sol.
RoutesDirect link to Routes
One model can be served by more than one route, such as the provider's own API and a partner provider. For each request, Gateway:
- Finds the routes that serve the model on the requested endpoint and support the request's features (for example tools, images, or context length).
- Removes routes that your key or organization policy doesn't allow.
- Orders the remaining routes by platform preference.
If no route remains, the request fails with no_eligible_route, or with unsupported_model_endpoint when the model exists but not on that endpoint.
The x-gateway-provider response header names the provider that served the request.
FailoverDirect link to Failover
When a route fails with an eligible error, such as a timeout, a provider outage, or a provider rate limit, Gateway tries the next route. Failover rules:
- Before output only: Gateway fails over only until the first byte reaches your client. After a stream starts, an upstream error ends the request.
- Bounded: A request makes up to 3 attempts by default.
- Not for client errors: Invalid requests aren't retried on another route.
- Cooldowns: When a provider returns
429, Gateway pauses that route for the provider'sRetry-Aftervalue (5 seconds by default, 60 at most) and skips it during that time. If every route is cooling down, Gateway returns429withx-gateway-rate-limit-scope: upstream_provider.
These requests stay on one route and don't fail over:
- Requests with a key that has failover turned off. See Keys and BYOK.
- Requests that use a BYOK connection.
- Responses continuations, which return to the provider that stored the response.
You're charged only for the attempt that served you. See Usage, pricing, and limits.