Skip to Content
ReferenceVoicevoice.on()

voice.on()

The on() method registers event listeners for various voice events. This is particularly important for real-time voice providers, where events are used to communicate transcribed text, audio responses, and other state changes.

Usage Example

import { OpenAIRealtimeVoice } from "@mastra/voice-openai-realtime";
import Speaker from "@mastra/node-speaker";
import chalk from "chalk";
 
const speaker = new Speaker({
  sampleRate: 24100,  // Audio sample rate in Hz - standard for high-quality audio on MacBook Pro
  channels: 1,        // Mono audio output (as opposed to stereo which would be 2)
  bitDepth: 16,       // Bit depth for audio quality - CD quality standard (16-bit resolution)
});
 
// Initialize a real-time voice provider
const voice = new OpenAIRealtimeVoice({
  realtimeConfig: {
    model: "gpt-4o-mini-realtime",
    apiKey: process.env.OPENAI_API_KEY,
  },
});
 
// Connect to the real-time service
await voice.connect();
 
// Register event listener for transcribed text
voice.on("writing", ({ text, role }) => {
  if (ev.role === 'user') {
    process.stdout.write(chalk.green(ev.text));
  } else {
    process.stdout.write(chalk.blue(ev.text));
  }
});
 
// Register event listener for speaker responses
voice.on("speaking", (stream) => {
  // Stream the audio to node speaker
  stream.pipe(speaker)
});
 
// Register event listener for errors
voice.on("error", ({ message, code, details }) => {
  console.error(`Error ${code}: ${message}`, details);
});

Parameters

event:

string
Name of the event to listen for (e.g., 'speaking', 'writing', 'error')

callback:

function
Callback function that will be called when the event occurs

Return Value

This method does not return a value.

Standard Events

All voice providers that implement event handling support these standard events:

speaking:

event
Emitted when audio data is available. The callback receives { audio } where audio is typically an Int16Array or Buffer.

speaker:

event
Emitted when a new audio response is ready to be streamed. The callback receives a buffer that can be piped into node-speaker.

writing:

event
Emitted when text is transcribed or generated. The callback receives { text, role } where role is either 'user' or 'assistant'.

error:

event
Emitted when an error occurs. The callback receives { message, code, details } with information about the error.

Provider-Specific Events

Different voice providers may emit additional events specific to their implementation:

OpenAI Realtime

OpenAI Realtime events are prefixed with openAIRealtime: and include:

openAIRealtime:conversation.created:

event
Emitted when a new conversation is created.

openAIRealtime:conversation.interrupted:

event
Emitted when a conversation is interrupted.

openAIRealtime:conversation.updated:

event
Emitted when a conversation is updated.

openAIRealtime:conversation.item.appended:

event
Emitted when an item is appended to the conversation.

openAIRealtime:conversation.item.completed:

event
Emitted when an item in the conversation is completed.

Using with CompositeVoice

When using CompositeVoice, the on() method delegates to the configured real-time provider:

import { CompositeVoice } from "@mastra/core/voice";
import { OpenAIRealtimeVoice } from "@mastra/voice-openai-realtime";
import Speaker from "@mastra/node-speaker";
 
const speaker = new Speaker({
  sampleRate: 24100,  // Audio sample rate in Hz - standard for high-quality audio on MacBook Pro
  channels: 1,        // Mono audio output (as opposed to stereo which would be 2)
  bitDepth: 16,       // Bit depth for audio quality - CD quality standard (16-bit resolution)
});
 
const realtimeVoice = new OpenAIRealtimeVoice();
const voice = new CompositeVoice({
  realtimeProvider: realtimeVoice,
});
 
// Connect to the real-time service
await voice.connect();
 
// This will register the event listener with the OpenAIRealtimeVoice provider
voice.on("speaker", (stream) => {
  stream.pipe(speaker)
});

Notes

  • This method is primarily used with real-time voice providers that support event-based communication
  • If called on a voice provider that doesn’t support events, it will log a warning and do nothing
  • Event listeners should be registered before calling methods that might emit events
  • To remove an event listener, use the off() method with the same event name and callback function
  • Multiple listeners can be registered for the same event
  • The callback function will receive different data depending on the event type
  • For best performance, consider removing event listeners when they are no longer needed
voice.answer()Deepgram