Introducing Private Model Gateways for Mastra Agents

You can now route agent requests to self-hosted open-weight models using a custom model gateway. Give the gateway a custom ID, then use it as the model prefix in your agent's config: private/ollama/gpt-oss:20b.

Using a hosted provider service can complicate HIPAA or GDPR compliance (unless data retention agreements are in place). Self-hosting removes the third party, preventing sensitive data from reaching models outside of your network.

Costs may also be reduced as you're only paying for infrastructure, not tokens, and you control which models your agents can use.

To hide external providers from Studio's model picker, set AUTO_BLOCK_EXTERNAL_PROVIDERS=true in your .env so users can only select models from your own gateways.

Get started

Install the latest @mastra/core and an AI SDK provider for your gateway's API:

npm install @mastra/core@latest @ai-sdk/openai-compatible@3.0.84

Run a local model

This example uses Ollama and gpt-oss:20b. Running ollama serve starts the server, with an OpenAI-compatible API at http://localhost:11434/v1:

brew install ollama
ollama pull gpt-oss:20b
ollama serve

Create a gateway

Create a gateway that sends every model call to the local Ollama server:

• id: The model ID prefix, e.g. private in private/ollama/gpt-oss:20b.
• name: The gateway's display name.
• fetchProviders(): Returns the providers and models the gateway offers, listed in Studio's model picker.
• buildUrl(): Returns the base URL for model calls.
• getApiKey(): Returns the API key to send with each call (not required for local models).
• resolveLanguageModel(): Creates the language model for each call.

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import type { MastraModelGatewayInterface } from "@mastra/core/llm";
 
export const privateGateway: MastraModelGatewayInterface = {
  id: "private",
  name: "Private Gateway",
  fetchProviders: async () => {
    return {
      ollama: {
        name: "Ollama",
        models: ["gpt-oss:20b"],
        apiKeyEnvVar: [],
        gateway: "private",
        url: "http://localhost:11434/v1"
      }
    };
  },
  buildUrl: () => {
    return "http://localhost:11434/v1";
  },
  getApiKey: async () => {
    return "";
  },
  resolveLanguageModel: ({ modelId, providerId, apiKey }) => {
    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL: "http://localhost:11434/v1"
    }).chatModel(modelId);
  }
};

Register the gateway

Add the gateway to your Mastra instance:

export const mastra = new Mastra({
  // ...
  gateways: { privateGateway }
});

Use the model

Reference the model by [gatewayId]/[provider]/[model]:

export const researchAgent = new Agent({
  id: "research-agent",
  name: "Research Agent",
  instructions: /* ... */,
  model: "private/ollama/gpt-oss:20b"
});

Hide external providers in Studio

Add the variable to your .env:

AUTO_BLOCK_EXTERNAL_PROVIDERS=true

For more information and full configuration options, see:

• Custom model gateways
• When to use custom gateways
• Showing only custom gateways in Studio
• Gateway-owned authentication
• Model pricing