Custom Model Gateways

Custom model gateways allow you to implement private or specialized LLM provider integrations by extending the MastraModelGateway base class.

Overview

Gateways handle provider-specific logic for accessing language models:

Provider configuration and model discovery
Authentication and API key management
URL construction for API endpoints
Language model instance creation

Create custom gateways to support:

Private or enterprise LLM deployments
Custom authentication schemes
Specialized routing logic
Gateway versioning with unique IDs

Creating a Custom Gateway

Extend the MastraModelGateway class and implement the required methods:

import { MastraModelGateway, type ProviderConfig } from '@mastra/core/llm';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible-v5';
import type { LanguageModelV2 } from '@ai-sdk/provider-v5';

class MyPrivateGateway extends MastraModelGateway {
  // Required: Unique identifier for the gateway
  readonly id = 'my-private-gateway';

  // Required: Human-readable name
  readonly name = 'My Private Gateway';

  // Optional: Prefix for model IDs (e.g., "private/provider/model")
  readonly prefix = 'private';

  /**
   * Fetch provider configurations from your gateway
   * Returns a record of provider configurations
   */
  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    return {
      'my-provider': {
        name: 'My Provider',
        models: ['model-1', 'model-2', 'model-3'],
        apiKeyEnvVar: 'MY_API_KEY',
        gateway: this.id,
        url: 'https://api.myprovider.com/v1',
      },
    };
  }

  /**
   * Build the API URL for a model
   * @param modelId - Full model ID (e.g., "private/my-provider/model-1")
   * @param envVars - Environment variables (optional)
   */
  buildUrl(modelId: string, envVars?: Record<string, string>): string {
    return 'https://api.myprovider.com/v1';
  }

  /**
   * Get the API key for authentication
   * @param modelId - Full model ID
   */
  async getApiKey(modelId: string): Promise<string> {
    const apiKey = process.env.MY_API_KEY;
    if (!apiKey) {
      throw new Error(`Missing MY_API_KEY environment variable`);
    }
    return apiKey;
  }

  /**
   * Create a language model instance
   * @param args - Model ID, provider ID, and API key
   */
  async resolveLanguageModel({
    modelId,
    providerId,
    apiKey,
  }: {
    modelId: string;
    providerId: string;
    apiKey: string;
  }): Promise<LanguageModelV2> {
    const baseURL = this.buildUrl(`${providerId}/${modelId}`);

    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

Registering Custom Gateways

During Initialization

Pass gateways as a record when creating your Mastra instance:

import { Mastra } from '@mastra/core';

const mastra = new Mastra({
  gateways: {
    myGateway: new MyPrivateGateway(),
    anotherGateway: new AnotherGateway(),
  },
});

After Initialization

Add gateways dynamically using addGateway:

const mastra = new Mastra();

// Add with explicit key
mastra.addGateway(new MyPrivateGateway(), 'myGateway');

// Add using gateway's ID
mastra.addGateway(new MyPrivateGateway());
// Stored with key 'my-private-gateway' (the gateway's id)

Using Custom Gateways

Reference models from your custom gateway using the gateway prefix:

import { Agent } from '@mastra/core';

const agent = new Agent({
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1', // Uses MyPrivateGateway
});

mastra.addAgent(agent, 'myAgent');

When you create an agent or use a model, Mastra's model router automatically selects the appropriate gateway based on the model ID prefix. If no custom gateways match, it falls back to the default gateways (Netlify and models.dev).

TypeScript Autocomplete

Automatic Type Generation in Development

When running in development mode (MASTRA_DEV=true), Mastra automatically generates TypeScript types for your custom gateways!

Set the environment variable:
```
export MASTRA_DEV=true
```

Register your gateways:

const mastra = new Mastra({
  gateways: {
    myGateway: new MyPrivateGateway(),
  },
});

Types are generated automatically:
- When you add a gateway, Mastra syncs with the GatewayRegistry
- The registry fetches providers from your custom gateway
- TypeScript types are regenerated in ~/.cache/mastra/
- Your IDE picks up the new types within seconds

Autocomplete now works:

const agent = new Agent({
  // If your gateway has a prefix:
  model: 'my-prefix/my-provider/model-1', // ✅ Full autocomplete!

  // If your gateway has no prefix:
  model: 'my-provider/model-1', // ✅ Full autocomplete!
});

note

The model ID format depends on whether your gateway has a prefix property:

With prefix: prefix/provider/model
Without prefix: provider/model

How It Works

The GatewayRegistry runs an hourly sync that:

Calls fetchProviders() on all registered gateways
Generates TypeScript type definitions
Writes them to both global cache and your project's dist/ directory
Your TypeScript server automatically picks up the changes

tip

The first time you add a gateway, it may take a few seconds for types to generate. Subsequent updates happen in the background every hour.

Manual Type Generation Alternatives

If you're not running in development mode or need immediate type updates:

Option 1: Use type assertion (simplest)

const agent = new Agent({
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' as any, // Bypass type checking
});

Option 2: Create a custom type union (type-safe)

import type { ModelRouterModelId } from '@mastra/core/llm';

// Define your custom model IDs
type CustomModelId =
  | 'private/my-provider/model-1'
  | 'private/my-provider/model-2'
  | 'private/my-provider/model-3';

// Combine with built-in models
type AllModelIds = ModelRouterModelId | CustomModelId;

const agent = new Agent({
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' satisfies AllModelIds,
});

Option 3: Extend ModelRouterModelId globally (advanced)

// In a types.d.ts file in your project
declare module '@mastra/core/llm' {
  interface ProviderModelsMap {
    'my-provider': readonly ['model-1', 'model-2', 'model-3'];
  }
}

This extends the built-in type to include your custom models, giving you full autocomplete support.

Gateway Management

getGateway(key)

Retrieve a gateway by its registration key:

const gateway = mastra.getGateway('myGateway');
console.log(gateway.name); // 'My Private Gateway'

getGatewayById(id)

Retrieve a gateway by its unique ID:

const gateway = mastra.getGatewayById('my-private-gateway');
console.log(gateway.name); // 'My Private Gateway'

This is useful when:

Gateways have explicit IDs different from their registration keys
You need to find a gateway by its ID across different instances
Supporting gateway versioning (e.g., 'gateway-v1', 'gateway-v2')

listGateways()

Get all registered gateways:

const gateways = mastra.listGateways();
console.log(Object.keys(gateways)); // ['myGateway', 'anotherGateway']

Gateway Properties

Required

Property	Type	Description
`id`	`string`	Unique identifier for the gateway
`name`	`string`	Human-readable gateway name

Optional

Property	Type	Description
`prefix`	`string \| undefined`	Optional prefix for model IDs

Methods

Method	Description
`fetchProviders()`	Fetch provider configurations
`buildUrl(modelId, envVars?)`	Build API URL for a model
`getApiKey(modelId)`	Get API key for authentication
`resolveLanguageModel(args)`	Create language model instance
`getId()`	Get gateway ID (returns `id` or `name`)

Provider Configuration

The fetchProviders() method returns a record of ProviderConfig objects:

interface ProviderConfig {
  name: string;                    // Display name
  models: string[];                // Available model IDs
  apiKeyEnvVar: string | string[]; // Environment variable(s) for API key
  gateway: string;                 // Gateway identifier
  url?: string;                    // Optional API base URL
  apiKeyHeader?: string;           // Optional custom auth header
  docUrl?: string;                 // Optional documentation URL
}

Gateway IDs vs Keys

Understanding the distinction:

Key: The registration key used when adding the gateway to Mastra (record key)
ID: The gateway's unique identifier (via id property or name if not set)

class VersionedGateway extends MastraModelGateway {
  readonly id = 'my-gateway-v2';     // Unique ID for versioning
  readonly name = 'My Gateway';       // Display name
  readonly prefix = 'versioned';
  // ... implementation
}

const mastra = new Mastra({
  gateways: {
    currentGateway: new VersionedGateway(), // Key: 'currentGateway'
  },
});

// Retrieve by key
const byKey = mastra.getGateway('currentGateway');

// Retrieve by ID
const byId = mastra.getGatewayById('my-gateway-v2');

// Both return the same gateway
console.log(byKey === byId); // true

Model ID Format

Models accessed through custom gateways follow this format:

[prefix]/[provider]/[model]

Examples:

private/my-provider/model-1 - With prefix
my-provider/model-1 - Without prefix (if prefix is undefined)

Advanced Example

Token-based gateway with caching:

class TokenGateway extends MastraModelGateway {
  readonly id = 'token-gateway-v1';
  readonly name = 'Token Gateway';
  readonly prefix = 'token';

  private tokenCache: Map<string, { token: string; expiresAt: number }> = new Map();

  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    const response = await fetch('https://api.gateway.com/providers');
    const data = await response.json();

    return {
      provider: {
        name: data.name,
        models: data.models,
        apiKeyEnvVar: 'GATEWAY_TOKEN',
        gateway: this.id,
      },
    };
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const token = await this.getApiKey(modelId);
    const siteId = envVars?.SITE_ID || process.env.SITE_ID;

    const response = await fetch(`https://api.gateway.com/sites/${siteId}/token`, {
      headers: { Authorization: `Bearer ${token}` },
    });

    const { url } = await response.json();
    return url;
  }

  async getApiKey(modelId: string): Promise<string> {
    const cached = this.tokenCache.get(modelId);

    if (cached && cached.expiresAt > Date.now()) {
      return cached.token;
    }

    const token = process.env.GATEWAY_TOKEN;
    if (!token) {
      throw new Error('Missing GATEWAY_TOKEN');
    }

    // Cache token for 1 hour
    this.tokenCache.set(modelId, {
      token,
      expiresAt: Date.now() + 3600000,
    });

    return token;
  }

  async resolveLanguageModel({ modelId, providerId, apiKey }: {
    modelId: string;
    providerId: string;
    apiKey: string;
  }): Promise<LanguageModelV2> {
    const baseURL = await this.buildUrl(`${providerId}/${modelId}`);

    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

Error Handling

Provide descriptive errors for common failure scenarios:

class RobustGateway extends MastraModelGateway {
  // ... properties

  async getApiKey(modelId: string): Promise<string> {
    const apiKey = process.env.MY_API_KEY;

    if (!apiKey) {
      throw new Error(
        `Missing MY_API_KEY environment variable for model: ${modelId}. ` +
        `Please set MY_API_KEY in your environment.`
      );
    }

    return apiKey;
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const baseUrl = envVars?.BASE_URL || process.env.BASE_URL;

    if (!baseUrl) {
      throw new Error(
        `No base URL configured for model: ${modelId}. ` +
        `Set BASE_URL environment variable or pass it in envVars.`
      );
    }

    return baseUrl;
  }
}

Testing Custom Gateways

Example test structure:

import { describe, it, expect, beforeEach } from 'vitest';
import { Mastra } from '@mastra/core';

describe('MyPrivateGateway', () => {
  beforeEach(() => {
    process.env.MY_API_KEY = 'test-key';
  });

  it('should fetch providers', async () => {
    const gateway = new MyPrivateGateway();
    const providers = await gateway.fetchProviders();

    expect(providers['my-provider']).toBeDefined();
    expect(providers['my-provider'].models).toContain('model-1');
  });

  it('should integrate with Mastra', () => {
    const mastra = new Mastra({
      gateways: {
        private: new MyPrivateGateway(),
      },
    });

    const gateway = mastra.getGateway('private');
    expect(gateway.name).toBe('My Private Gateway');
  });

  it('should resolve models by ID', () => {
    const mastra = new Mastra({
      gateways: {
        key: new MyPrivateGateway(),
      },
    });

    const gateway = mastra.getGatewayById('my-private-gateway');
    expect(gateway).toBeDefined();
  });
});

Best Practices

Use descriptive IDs for versioning: Set explicit id values when you need to version your gateways
```
readonly id = 'my-gateway-v1';
```
Implement proper error handling: Throw descriptive errors with actionable messages
Cache expensive operations: Cache tokens, URLs, or provider configurations when appropriate
Validate environment variables: Check for required environment variables in getApiKey and buildUrl
Document your gateway: Add JSDoc comments explaining the gateway's purpose and configuration
Follow naming conventions: Use clear, consistent naming for providers and models
Handle async operations: Use async/await for network requests and I/O operations
Test thoroughly: Write unit tests for all gateway methods

Built-in Gateways

Mastra includes built-in gateways as reference implementations:

NetlifyGateway: Netlify AI Gateway integration with token exchange
ModelsDevGateway: Registry of OpenAI-compatible providers from models.dev

See Netlify, OpenRouter, and Vercel for examples of gateway usage.

Overview​

Creating a Custom Gateway​

Registering Custom Gateways​

During Initialization​

After Initialization​

Using Custom Gateways​

TypeScript Autocomplete​

Manual Type Generation Alternatives​

Gateway Management​

getGateway(key)​

getGatewayById(id)​

listGateways()​

Gateway Properties​

Required​

Optional​

Methods​

Provider Configuration​

Gateway IDs vs Keys​

Model ID Format​

Advanced Example​

Error Handling​

Testing Custom Gateways​

Best Practices​

Built-in Gateways​