Introducing TTS in Mastra

Building Text-to-Speech Applications with Mastra

We recently shipped a TTS module that integrates with OpenAI and ElevenLabs speech models.

Let's explore how to use it.

Basic Setup

First, install the required package:

npm install @mastra/voice-openai

Configure your environment:

OPENAI_API_KEY=your_api_key_here

Basic TTS Usage

Initialize the TTS client:

import { OpenAIVoice } from "@mastra/voice-openai";
 
const tts = new OpenAIVoice({
  speechModel: {
    name: "tts-1",
  },
});

Voice Selection

OpenAI provides several voices to choose from:

const voices = await tts.getSpeakers();
// Available voices: alloy, echo, fable, onyx, nova, shimmer

Generate Speech

Generate audio with your chosen voice:

const audioResult = await tts.speak("Hello, world!", {
  speaker: "nova",
  speed: 1.0,
});

Streaming Audio

For real-time audio streaming:

const audioResult = await tts.speak("This is a streaming response", {
  speaker: "alloy",
  speed: 1.2,
});
 
// audioResult is a PassThrough stream

Error Handling and Telemetry

The TTS system includes built-in telemetry and error tracing, so you can use your favorite tracing tools to get visibility into your TTS usage.

Usage with Mastra

Integrate TTS with a Mastra agent:

import { Agent } from "@mastra/core/agent";
import { OpenAIVoice } from "@mastra/voice-openai";
 
const tts = new OpenAIVoice({
  speechModel: {
    name: "tts-1",
  },
});
 
const agent = new Agent({
  id: "voice-agent",
  name: "Voice Agent",
  instructions: "You are a helpful assistant.",
  model: "openai/gpt-6.1-sol",
  voice: tts,
});
 
// Generate speech
const audio = await agent.voice.speak("Welcome to Mastra", {
  speaker: "nova",
});

The Mastra TTS system provides type-safe speech generation with telemetry and error handling. Start with basic generation and add streaming as needed.

Next Steps: Exposing TTS to Agents

One thing we're thinking about is how to expose TTS to agents.

Currently, our thought is to optionally let agents be configured with a TTS model, and then agent.tts.generate() and agent.tts.stream() would be available, as well as /agents/$AGENT_ID/tts/generate and /agents/$AGENT_ID/tts/stream endpoints.

Some other questions:

• How should we expose this functionality in the mastra dev UI?

We figured we would embed a sound clip in the chat UI for agents that have a TTS model configured.

• How should we expose this functionality in agent memory?

We figured we would probably add a new tts field to items in agent memory, and then we could store the TTS model name there.