Custom model gateways

Custom model gateways allow you to implement private or specialized LLM provider integrations with the MastraModelGatewayInterface interface or the MastraModelGateway base class.

Overview

Gateways handle provider-specific logic for accessing language models:

• Provider configuration and model discovery
• Authentication and API key management
• URL construction for API endpoints
• Language model instance creation

When to use custom gateways

Use a custom gateway to connect Mastra to open-weight models that you run yourself, such as gpt-oss, Llama, Gemma, Mistral, Qwen, or DeepSeek. Self-hosting fits when:

• Your industry's rules don't allow sending prompts or customer data to a third-party API (HIPAA, GDPR, finance, defense)
• You make enough model calls that per-token API pricing costs more than running your own GPUs
• You've fine-tuned an open-weight model on internal data and want your agents to use it
• You need a model that only changes when you upgrade it, so a provider update can't break your prompts
• Your agents run offline, on air-gapped networks, on edge hardware, or in the same data center as your app

The gateway lists the models behind your endpoint, so they get typed model IDs and appear in the Studio model picker. It can also resolve credentials on every call, and you can limit the Studio model picker to your own models.

To call a single endpoint with an OpenAI-compatible API, you don't need a gateway: set url in the model config instead. Mastra doesn't read an API key from environment variables for a url model, so also set apiKey when the endpoint requires authentication. See Use local models with Mastra.

Creating a Custom Gateway

Implement MastraModelGatewayInterface for a plain object gateway, or extend the MastraModelGateway base class.

Choose a plain object or a class

Both work the same way at runtime. The base class provides default getId(), shouldEnable(), handlesModel(), and serializeForSpan() methods, and Mastra applies the same defaults to a plain object that leaves them out.

Use a plain object when the gateway has fixed configuration and no state. Extend MastraModelGateway when the gateway:

• Takes options in its constructor, such as a base URL or API key
• Keeps private state, such as cached tokens
• Runs as several instances, such as one per region or version

The following example implements the interface with a plain object:

import { type MastraModelGatewayInterface, type ProviderConfig } from '@mastra/core/llm';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible-v5';

const myPrivateGateway: MastraModelGatewayInterface = {
  id: 'private',
  name: 'My Private Gateway',
  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    return {
      'my-provider': {
        name: 'My Provider',
        models: ['openai/gpt-6.1-sol'],
        apiKeyEnvVar: 'MY_API_KEY',
        gateway: 'private',
        url: 'https://api.myprovider.com/v1',
      },
    };
  },
  buildUrl() {
    return 'https://api.myprovider.com/v1';
  },
  async getApiKey() {
    return process.env.MY_API_KEY ?? '';
  },
  async resolveLanguageModel({ modelId, providerId, apiKey, headers }) {
    return createOpenAICompatible({
      name: providerId,
      apiKey,
      headers,
      baseURL: 'https://api.myprovider.com/v1',
    }).chatModel(modelId);
  },
};

The following example extends MastraModelGateway and takes its ID, base URL, and API key as constructor options:

import { MastraModelGateway, type GatewayLanguageModel, type ProviderConfig } from '@mastra/core/llm';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible-v5';

interface MyPrivateGatewayOptions {
  id?: string;
  baseURL?: string;
  apiKey?: string;
}

class MyPrivateGateway extends MastraModelGateway {
  readonly id: string;
  readonly name = 'My Private Gateway';

  private readonly baseURL: string;
  private readonly apiKey?: string;

  constructor(options: MyPrivateGatewayOptions = {}) {
    super();
    this.id = options.id ?? 'private';
    this.baseURL = options.baseURL ?? 'https://api.myprovider.com/v1';
    this.apiKey = options.apiKey;
  }

  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    return {
      'my-provider': {
        name: 'My Provider',
        models: ['openai/gpt-6.1-sol', 'anthropic/claude-sonnet-5-5'],
        apiKeyEnvVar: 'MY_API_KEY',
        gateway: this.id,
        url: this.baseURL,
      },
    };
  }

  buildUrl(): string {
    return this.baseURL;
  }

  async getApiKey(): Promise<string> {
    const apiKey = this.apiKey ?? process.env.MY_API_KEY;
    if (!apiKey) {
      throw new Error('Missing MY_API_KEY environment variable');
    }
    return apiKey;
  }

  resolveLanguageModel({
    modelId,
    providerId,
    apiKey,
    headers,
  }: {
    modelId: string;
    providerId: string;
    apiKey: string;
    headers?: Record<string, string>;
  }): GatewayLanguageModel {
    return createOpenAICompatible({
      name: providerId,
      apiKey,
      headers,
      baseURL: this.baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

To run one instance per endpoint, give each instance its own id. Mastra keeps only the first gateway registered with a given ID, and the ID is the model ID prefix:

import { Mastra } from '@mastra/core';

const mastra = new Mastra({
  gateways: {
    privateEu: new MyPrivateGateway({ id: 'private-eu', baseURL: 'https://eu.api.myprovider.com/v1' }),
    privateUs: new MyPrivateGateway({ id: 'private-us', baseURL: 'https://us.api.myprovider.com/v1' }),
  },
});

// Models: 'private-eu/my-provider/openai/gpt-6.1-sol' and 'private-us/my-provider/openai/gpt-6.1-sol'

Gateway-owned authentication

Add resolveAuth when the gateway owns credential lookup. Mastra uses this hook before falling back to getApiKey().

const myPrivateGateway: MastraModelGatewayInterface = {
  // ...gateway fields and methods
  async resolveAuth() {
    const apiKey = process.env.MY_API_KEY;
    return apiKey ? { apiKey, source: 'gateway' } : undefined;
  },
};

Registering Custom Gateways

During Initialization

Pass gateways as a record when creating your Mastra instance:

import { Mastra } from '@mastra/core';

const mastra = new Mastra({
  gateways: {
    myGateway: new MyPrivateGateway(),
    anotherGateway: new AnotherGateway(),
  },
});

After Initialization

Add gateways dynamically using addGateway:

const mastra = new Mastra();

// Add with explicit key
mastra.addGateway(new MyPrivateGateway(), 'myGateway');

// Add using gateway's ID
mastra.addGateway(new MyPrivateGateway());
// Stored with key 'my-private-gateway' (the gateway's id)

Using Custom Gateways

Reference models from your custom gateway using the gateway ID as prefix:

import { Agent } from '@mastra/core/agent';

const agent = new Agent({
  id: 'my-agent',
  name: 'My Agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/openai/gpt-6.1-sol', // Uses MyPrivateGateway
});

mastra.addAgent(agent, 'myAgent');

When you create an agent or use a model, Mastra's model router automatically selects the appropriate gateway based on the model ID. The gateway ID serves as the prefix. If no custom gateways match, it falls back to the built-in gateways.

TypeScript Autocomplete

Automatic type generation in development

When running in development mode (MASTRA_DEV=true), Mastra automatically generates TypeScript types for your custom gateways.

1. Set the environment variable:

   export MASTRA_DEV=true

2. Register your gateways:

   const mastra = new Mastra({
     gateways: {
       myGateway: new MyPrivateGateway(),
     },
   });

3. Types are generated automatically:

   • When you add a gateway, Mastra syncs with the GatewayRegistry
   • The registry fetches providers from your custom gateway
   • TypeScript types are regenerated in ~/.cache/mastra/
   • Your IDE picks up the new types within seconds

4. Autocomplete now works:

   const agent = new Agent({
     model: 'my-gateway-id/my-provider/model-1', // Full autocomplete!
   });

How it works

The GatewayRegistry runs an hourly sync that:

• Calls fetchProviders() on all registered gateways
• Generates TypeScript type definitions
• Writes them to both global cache and your project's dist/ directory
• Your TypeScript server automatically picks up the changes

tip

The first time you add a gateway, it may take a few seconds for types to generate. Subsequent updates happen in the background every hour.

Manual Type Generation Alternatives

If you're not running in development mode or need immediate type updates:

Option 1: Use type assertion (simplest)

const agent = new Agent({
  id: 'my-agent',
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' as any, // Bypass type checking
});

Option 2: Create a custom type union (type-safe)

import type { ModelRouterModelId } from '@mastra/core/llm';

// Define your custom model IDs
type CustomModelId =
  | 'private/my-provider/model-1'
  | 'private/my-provider/model-2'
  | 'private/my-provider/model-3';

// Combine with built-in models
type AllModelIds = ModelRouterModelId | CustomModelId;

const agent = new Agent({
  id: 'my-agent',
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' satisfies AllModelIds,
});

Option 3: Extend ModelRouterModelId globally (advanced)

// In a types.d.ts file in your project
// The import is required: it makes this file a module, so the block below
// merges into the existing types instead of replacing the module.
import '@mastra/core/llm';

declare module '@mastra/core/llm' {
  interface ProviderModelsMap {
    'my-provider': readonly ['model-1', 'model-2', 'model-3'];
  }
}

This extends the built-in type to include your custom models, giving you full autocomplete support.

Gateway Management

getGateway(key)

Retrieve a gateway by its registration key:

const gateway = mastra.getGateway('myGateway');
console.log(gateway.name); // 'My Private Gateway'

getGatewayById(id)

Retrieve a gateway by its unique ID:

const gateway = mastra.getGatewayById('my-private-gateway');
console.log(gateway.name); // 'My Private Gateway'

This is useful when:

• Gateways have explicit IDs different from their registration keys
• You need to find a gateway by its ID across different instances
• Supporting gateway versioning (e.g., 'gateway-v1', 'gateway-v2')

getGatewayById() throws a MastraError if no registered gateway has the given ID. It doesn't return undefined.

listGateways()

Get all registered gateways:

const gateways = mastra.listGateways();
console.log(Object.keys(gateways)); // ['myGateway', 'anotherGateway']

Gateway Properties

Required

Property	Type	Description
id	string	Unique identifier for the gateway, used as the gateway prefix for the model string
name	string	Human-readable gateway name

Methods

Method	Description
fetchProviders()	Fetch provider configurations
buildUrl(modelId, envVars?)	Build API URL for a model
getApiKey(modelId)	Get API key for authentication
resolveLanguageModel(args)	Create language model instance
getId()	Get gateway ID (returns id or name)

Provider Configuration

The fetchProviders() method returns a record of ProviderConfig objects:

interface ProviderConfig {
  name: string;                    // Display name
  models: string[];                // Available model IDs
  apiKeyEnvVar: string | string[]; // Environment variable(s) for API key
  gateway: string;                 // Gateway identifier
  url?: string;                    // Optional API base URL
  apiKeyHeader?: string;           // Optional custom auth header
  docUrl?: string;                 // Optional documentation URL
  pricing?: Record<string, ModelPricing>; // Optional prices per model ID
}

Model pricing

models.dev does not list private deployments, so their calls have no cost in metrics. Add a pricing entry per model to give them one. Prices are in USD per 1 million tokens and use the same keys as a model's cost on models.dev: input, output, cache_read, cache_write, reasoning, input_audio, and output_audio. @mastra/observability reads these prices on the first call to a model it can't price, which also picks up gateways added later with addGateway(). Under mastra dev, or with MASTRA_AUTO_REFRESH_PRICING=true, it also reads them when Mastra starts, so the first call is priced too. models.dev prices take precedence. A gateway price is used only when models.dev has none for that provider and model ID, and then it applies to every call with those IDs, including direct calls. Give a reseller its own provider ID when its rates must stay separate from direct calls. A gateway whose fetchProviders() fails is logged and retried about once an hour. Set MASTRA_OFFLINE=true to skip loading them. The same shape as models.dev also means you can add tiers for long-context pricing.

async fetchProviders(): Promise<Record<string, ProviderConfig>> {
  return {
    'my-provider': {
      name: 'My Provider',
      models: ['model-1'],
      apiKeyEnvVar: 'MY_API_KEY',
      gateway: 'private',
      url: 'https://api.myprovider.com/v1',
      pricing: {
        'model-1': { input: 0.5, output: 1.5, cache_read: 0.05 },
      },
    },
  };
}

Gateway IDs vs Keys

Understanding the distinction:

• Key: The registration key used when adding the gateway to Mastra (record key)
• ID: The gateway's unique identifier (via id property or name if not set)

class VersionedGateway extends MastraModelGateway {
  readonly id = 'my-gateway-v2';     // Unique ID for versioning and prefixing
  readonly name = 'My Gateway';       // Display name
}

const mastra = new Mastra({
  gateways: {
    currentGateway: new VersionedGateway(), // Key: 'currentGateway'
  },
});

// Retrieve by key
const byKey = mastra.getGateway('currentGateway');

// Retrieve by ID
const byId = mastra.getGatewayById('my-gateway-v2');

// Both return the same gateway
console.log(byKey === byId); // true

Model ID Format

Models accessed through custom gateways follow this format:

[gatewayId]/[provider]/[model]

Examples:

• private/my-provider/model-1

Advanced example

Token-based gateway with caching:

class TokenGateway extends MastraModelGateway {
  readonly id = 'token-gateway-v1';
  readonly name = 'Token Gateway';

  private tokenCache: Map<string, { token: string; expiresAt: number }> = new Map();

  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    const response = await fetch('https://api.gateway.com/providers');
    const data = await response.json();

    return {
      provider: {
        name: data.name,
        models: data.models,
        apiKeyEnvVar: 'GATEWAY_TOKEN',
        gateway: this.id,
      },
    };
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const token = await this.getApiKey(modelId);
    const siteId = envVars?.SITE_ID || process.env.SITE_ID;

    const response = await fetch(`https://api.gateway.com/sites/${siteId}/token`, {
      headers: { Authorization: `Bearer ${token}` },
    });

    const { url } = await response.json();
    return url;
  }

  async getApiKey(modelId: string): Promise<string> {
    const cached = this.tokenCache.get(modelId);

    if (cached && cached.expiresAt > Date.now()) {
      return cached.token;
    }

    const token = process.env.GATEWAY_TOKEN;
    if (!token) {
      throw new Error('Missing GATEWAY_TOKEN');
    }

    // Cache token for 1 hour
    this.tokenCache.set(modelId, {
      token,
      expiresAt: Date.now() + 3600000,
    });

    return token;
  }

  async resolveLanguageModel({ modelId, providerId, apiKey }: {
    modelId: string;
    providerId: string;
    apiKey: string;
  }): Promise<LanguageModelV2> {
    const baseURL = await this.buildUrl(`${providerId}/${modelId}`);

    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

Error handling

Provide descriptive errors for common failure scenarios:

class RobustGateway extends MastraModelGateway {
  // ... properties

  async getApiKey(modelId: string): Promise<string> {
    const apiKey = process.env.MY_API_KEY;

    if (!apiKey) {
      throw new Error(
        `Missing MY_API_KEY environment variable for model: ${modelId}. ` +
        `Please set MY_API_KEY in your environment.`
      );
    }

    return apiKey;
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const baseUrl = envVars?.BASE_URL || process.env.BASE_URL;

    if (!baseUrl) {
      throw new Error(
        `No base URL configured for model: ${modelId}. ` +
        `Set BASE_URL environment variable or pass it in envVars.`
      );
    }

    return baseUrl;
  }
}

Testing Custom Gateways

Example test structure:

import { describe, it, expect, beforeEach } from 'vitest';
import { Mastra } from '@mastra/core';

describe('MyPrivateGateway', () => {
  beforeEach(() => {
    process.env.MY_API_KEY = 'test-key';
  });

  it('should fetch providers', async () => {
    const gateway = new MyPrivateGateway();
    const providers = await gateway.fetchProviders();

    expect(providers['my-provider']).toBeDefined();
    expect(providers['my-provider'].models).toContain('model-1');
  });

  it('should integrate with Mastra', () => {
    const mastra = new Mastra({
      gateways: {
        private: new MyPrivateGateway(),
      },
    });

    const gateway = mastra.getGateway('private');
    expect(gateway.name).toBe('My Private Gateway');
  });

  it('should resolve models by ID', () => {
    const mastra = new Mastra({
      gateways: {
        key: new MyPrivateGateway(),
      },
    });

    const gateway = mastra.getGatewayById('my-private-gateway');
    expect(gateway).toBeDefined();
  });
});

Best practices

1. Use descriptive IDs for versioning: Set explicit id values when you need to version your gateways

   readonly id = 'my-gateway-v1';

2. Implement proper error handling: Throw descriptive errors with actionable messages

3. Cache expensive operations: Cache tokens, URLs, or provider configurations when appropriate

4. Validate environment variables: Check for required environment variables in getApiKey and buildUrl

5. Document your gateway: Add JSDoc comments explaining the gateway's purpose and configuration

6. Follow naming conventions: Use clear, consistent naming for providers and models

7. Handle async operations: Use async/await for network requests and I/O operations

8. Test thoroughly: Write unit tests for all gateway methods

Showing only custom gateways in Studio

By default, Studio lists every external model provider (such as OpenAI, Anthropic, and Gemini) alongside any custom gateways you register. To hide the external providers, set the AUTO_BLOCK_EXTERNAL_PROVIDERS environment variable:

AUTO_BLOCK_EXTERNAL_PROVIDERS=true

When this variable is set to true or 1, Mastra returns only the providers from gateways you register yourself. The static provider registry and the built-in gateways (models.dev, netlify, and mastra) are hidden from the model picker. This is useful when you route all model traffic through a single private gateway and don't want the other providers to appear. The filter applies to the server's GET /agents/providers endpoint, which Studio uses for the model picker, so API clients don't see the external providers either. It only changes which providers are listed. It doesn't stop agents from calling other providers in code.

With this variable set and no custom gateway registered, the model picker is empty. Register at least one custom gateway to expose its models.

Built-in Gateways

Mastra includes built-in gateways as reference implementations:

• NetlifyGateway: Netlify AI Gateway integration with token exchange
• ModelsDevGateway: Registry of OpenAI-compatible providers from models.dev

See Netlify, OpenRouter, and Vercel for examples of gateway usage.