Custom Model Gateways

Custom model gateways allow you to implement private or specialized LLM provider integrations by extending the MastraModelGateway base class.

Overview

Gateways handle provider-specific logic for accessing language models:

• Provider configuration and model discovery
• Authentication and API key management
• URL construction for API endpoints
• Language model instance creation

Create custom gateways to support:

• Private or enterprise LLM deployments
• Custom authentication schemes
• Specialized routing logic
• Gateway versioning with unique IDs

Creating a Custom Gateway

Extend the MastraModelGateway class and implement the required methods:

import { MastraModelGateway, type ProviderConfig } from '@mastra/core/llm';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible-v5';
import type { LanguageModelV2 } from '@ai-sdk/provider-v5';

class MyPrivateGateway extends MastraModelGateway {
  // Required: Unique identifier for the gateway
  // This ID is used as the prefix for all providers from this gateway
  readonly id = 'private';

  // Required: Human-readable name
  readonly name = 'My Private Gateway';

  /**
   * Fetch provider configurations from your gateway
   * Returns a record of provider configurations
   */
  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    return {
      'my-provider': {
        name: 'My Provider',
        models: ['model-1', 'model-2', 'model-3'],
        apiKeyEnvVar: 'MY_API_KEY',
        gateway: this.id,
        url: 'https://api.myprovider.com/v1',
      },
    };
  }

  /**
   * Build the API URL for a model
   * @param modelId - Full model ID (e.g., "private/my-provider/model-1")
   * @param envVars - Environment variables (optional)
   */
  buildUrl(modelId: string, envVars?: Record<string, string>): string {
    return 'https://api.myprovider.com/v1';
  }

  /**
   * Get the API key for authentication
   * @param modelId - Full model ID
   */
  async getApiKey(modelId: string): Promise<string> {
    const apiKey = process.env.MY_API_KEY;
    if (!apiKey) {
      throw new Error(`Missing MY_API_KEY environment variable`);
    }
    return apiKey;
  }

  /**
   * Create a language model instance
   * @param args - Model ID, provider ID, and API key
   */
  async resolveLanguageModel({
    modelId,
    providerId,
    apiKey,
  }: {
    modelId: string;
    providerId: string;
    apiKey: string;
  }): Promise<LanguageModelV2> {
    const baseURL = this.buildUrl(`${providerId}/${modelId}`);

    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

Registering Custom Gateways

During Initialization

Pass gateways as a record when creating your Mastra instance:

import { Mastra } from '@mastra/core';

const mastra = new Mastra({
  gateways: {
    myGateway: new MyPrivateGateway(),
    anotherGateway: new AnotherGateway(),
  },
});

Using Custom Gateways

Reference models from your custom gateway using the gateway ID as prefix:

import { Agent } from '@mastra/core';

const mastra = new Mastra({
  gateways: {
    private: new MyPrivateGateway(),
  },
  agents: {
    myAgent: new Agent({
      name: 'my-agent',
      instructions: 'You are a helpful assistant',
      model: 'private/my-provider/model-1', // Uses MyPrivateGateway
    }),
  },
});

When you create an agent or use a model, Mastra's model router automatically selects the appropriate gateway based on the model ID. The gateway ID serves as the prefix. If no custom gateways match, it falls back to the built-in gateways.

TypeScript Autocomplete

Automatic Type Generation in Development

When running in development mode (MASTRA_DEV=true), Mastra automatically generates TypeScript types for your custom gateways!

1. Set the environment variable:

   export MASTRA_DEV=true

2. Register your gateways:

   const mastra = new Mastra({
     gateways: {
       myGateway: new MyPrivateGateway(),
     },
   });

3. Types are generated automatically:

   • When you add a gateway, Mastra syncs with the GatewayRegistry
   • The registry fetches providers from your custom gateway
   • TypeScript types are regenerated in ~/.cache/mastra/
   • Your IDE picks up the new types within seconds

4. Autocomplete now works:

   const agent = new Agent({
     model: 'my-gateway-id/my-provider/model-1', // ✅ Full autocomplete!
   
   });

How It Works

The GatewayRegistry runs an hourly sync that:

• Calls fetchProviders() on all registered gateways
• Generates TypeScript type definitions
• Writes them to both global cache and your project's dist/ directory
• Your TypeScript server automatically picks up the changes

tip

The first time you add a gateway, it may take a few seconds for types to generate. Subsequent updates happen in the background every hour.

Manual Type Generation Alternatives

If you're not running in development mode or need immediate type updates:

Option 1: Use type assertion (simplest)

const agent = new Agent({
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' as any, // Bypass type checking
});

Option 2: Create a custom type union (type-safe)

import type { ModelRouterModelId } from '@mastra/core/llm';

// Define your custom model IDs
type CustomModelId =
  | 'private/my-provider/model-1'
  | 'private/my-provider/model-2'
  | 'private/my-provider/model-3';

// Combine with built-in models
type AllModelIds = ModelRouterModelId | CustomModelId;

const agent = new Agent({
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' satisfies AllModelIds,
});

Option 3: Extend ModelRouterModelId globally (advanced)

// In a types.d.ts file in your project
declare module '@mastra/core/llm' {
  interface ProviderModelsMap {
    'my-provider': readonly ['model-1', 'model-2', 'model-3'];
  }
}

This extends the built-in type to include your custom models, giving you full autocomplete support.

listGateways()

Get all registered gateways:

const gateways = mastra.listGateways();
console.log(Object.keys(gateways)); // ['myGateway', 'anotherGateway']

Gateway Properties

Required

Property	Type	Description
id	string	Unique identifier for the gateway, used as the gateway prefix for the model string
name	string	Human-readable gateway name

Methods

Method	Description
fetchProviders()	Fetch provider configurations
buildUrl(modelId, envVars?)	Build API URL for a model
getApiKey(modelId)	Get API key for authentication
resolveLanguageModel(args)	Create language model instance
getId()	Get gateway ID (returns id or name)

Provider Configuration

The fetchProviders() method returns a record of ProviderConfig objects:

interface ProviderConfig {
  name: string;                    // Display name
  models: string[];                // Available model IDs
  apiKeyEnvVar: string | string[]; // Environment variable(s) for API key
  gateway: string;                 // Gateway identifier
  url?: string;                    // Optional API base URL
  apiKeyHeader?: string;           // Optional custom auth header
  docUrl?: string;                 // Optional documentation URL
}

Gateway IDs vs Keys

Understanding the distinction:

• Key: The registration key used when adding the gateway to Mastra (record key)
• ID: The gateway's unique identifier (via id property), used as the prefix in model strings

class VersionedGateway extends MastraModelGateway {
  readonly id = 'my-gateway-v2';     // Unique ID for versioning and prefixing
  readonly name = 'My Gateway';       // Display name
  // ... implementation
}

const mastra = new Mastra({
  gateways: {
    currentGateway: new VersionedGateway(), // Key: 'currentGateway'
  },
});

// Access models using the gateway ID as prefix
const agent = new Agent({
  name: 'my-agent',
  instructions: 'You are helpful',
  model: 'my-gateway-v2/provider/model-1', // Uses VersionedGateway
});

Model ID Format

Models accessed through custom gateways follow this format:

[gatewayId]/[provider]/[model]

Examples:

• private/my-provider/model-1

Advanced Example

Token-based gateway with caching:

class TokenGateway extends MastraModelGateway {
  readonly id = 'token-gateway-v1';
  readonly name = 'Token Gateway';

  private tokenCache: Map<string, { token: string; expiresAt: number }> = new Map();

  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    const response = await fetch('https://api.gateway.com/providers');
    const data = await response.json();

    return {
      provider: {
        name: data.name,
        models: data.models,
        apiKeyEnvVar: 'GATEWAY_TOKEN',
        gateway: this.id,
      },
    };
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const token = await this.getApiKey(modelId);
    const siteId = envVars?.SITE_ID || process.env.SITE_ID;

    const response = await fetch(`https://api.gateway.com/sites/${siteId}/token`, {
      headers: { Authorization: `Bearer ${token}` },
    });

    const { url } = await response.json();
    return url;
  }

  async getApiKey(modelId: string): Promise<string> {
    const cached = this.tokenCache.get(modelId);

    if (cached && cached.expiresAt > Date.now()) {
      return cached.token;
    }

    const token = process.env.GATEWAY_TOKEN;
    if (!token) {
      throw new Error('Missing GATEWAY_TOKEN');
    }

    // Cache token for 1 hour
    this.tokenCache.set(modelId, {
      token,
      expiresAt: Date.now() + 3600000,
    });

    return token;
  }

  async resolveLanguageModel({ modelId, providerId, apiKey }: {
    modelId: string;
    providerId: string;
    apiKey: string;
  }): Promise<LanguageModelV2> {
    const baseURL = await this.buildUrl(`${providerId}/${modelId}`);

    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

Error Handling

Provide descriptive errors for common failure scenarios:

class RobustGateway extends MastraModelGateway {
  // ... properties

  async getApiKey(modelId: string): Promise<string> {
    const apiKey = process.env.MY_API_KEY;

    if (!apiKey) {
      throw new Error(
        `Missing MY_API_KEY environment variable for model: ${modelId}. ` +
        `Please set MY_API_KEY in your environment.`
      );
    }

    return apiKey;
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const baseUrl = envVars?.BASE_URL || process.env.BASE_URL;

    if (!baseUrl) {
      throw new Error(
        `No base URL configured for model: ${modelId}. ` +
        `Set BASE_URL environment variable or pass it in envVars.`
      );
    }

    return baseUrl;
  }
}

Testing Custom Gateways

Example test structure:

import { describe, it, expect, beforeEach } from 'vitest';
import { Mastra } from '@mastra/core';

describe('MyPrivateGateway', () => {
  beforeEach(() => {
    process.env.MY_API_KEY = 'test-key';
  });

  it('should fetch providers', async () => {
    const gateway = new MyPrivateGateway();
    const providers = await gateway.fetchProviders();

    expect(providers['my-provider']).toBeDefined();
    expect(providers['my-provider'].models).toContain('model-1');
  });

  it('should integrate with Mastra', () => {
    const mastra = new Mastra({
      gateways: {
        private: new MyPrivateGateway(),
      },
    });

    const gateways = mastra.listGateways();
    expect(gateways?.private.name).toBe('My Private Gateway');
  });
});

Best Practices

1. Use descriptive IDs for versioning: Set explicit id values when you need to version your gateways

   readonly id = 'my-gateway-v1';

2. Implement proper error handling: Throw descriptive errors with actionable messages

3. Cache expensive operations: Cache tokens, URLs, or provider configurations when appropriate

4. Validate environment variables: Check for required environment variables in getApiKey and buildUrl

5. Document your gateway: Add JSDoc comments explaining the gateway's purpose and configuration

6. Follow naming conventions: Use clear, consistent naming for providers and models

7. Handle async operations: Use async/await for network requests and I/O operations

8. Test thoroughly: Write unit tests for all gateway methods

Built-in Gateways

Mastra includes built-in gateways as reference implementations:

• NetlifyGateway: Netlify AI Gateway integration with token exchange
• ModelsDevGateway: Registry of OpenAI-compatible providers from models.dev

See Netlify, OpenRouter, and Vercel for examples of gateway usage.