How Riverside Built An Agent To Help 20K+ Teams Publish Videos Faster

Recording is just the start of creating video. Before publishing, creators spend hours editing, creating trailers, clips, notes, titles, thumbnails, social images, and platform-specific versions.

That's why Riverside, the video platform used by more than 1 million creators, built an agent with Mastra to automate the tedious parts of video content creation.

It's part of a bigger shift for Riverside. The company, which has raised $80M, is moving from recording software to an AI-first content creation platform.

Improving the video content creation process

Before agents, creators had to manually edit and bounce between tools to turn raw footage into publishable content. Now, they can do it with Riverside's Co-Creator. For example, to make a thumbnail:

1. The user asks the Co-Creator to make a thumbnail from the current recording.
2. The Mastra agent uses a tool to retrieve the relevant transcript and project context.
3. It loads Riverside's thumbnail skill, which explains what makes a good result and how the available tools should be used.
4. The model chooses the moment, framing, and copy (this is the part that requires creative judgment), and Riverside's tools perform the product operations.
5. The tool streams structured UI through Mastra's stream writer, so the browser renders options to choose from.

Choosing Mastra

Riverside needed more than a framework for one agent loop. Co-Creator combines product tools with task-specific skills and interactive UI. There's also the memory element and a separately built editing agent. Mastra's primitives let the team connect those pieces without limiting them to one orchestration pattern.

"What excites me about Mastra is that you really have this composable architecture. You can throw agents into the workflow, or a workflow into the agent, or a workflow in a workflow. The possibilities are endless in that sense." — Eli Peer, Tech Lead

Streaming from backend to clients

Riverside's main agent runs in a Node.js service built with Mastra. Product capabilities and the knowledge base are exposed as tools. Skills supply the procedure for a particular job when the agent needs it.

Input and output processors run around the agent loop. Mastra stores the chat history; its working memory carries information into later conversations.

Riverside has a React web application alongside native iOS, Android, and Mac clients. The work happens on the backend, but is consumed across clients. Over 20 engineers across the backend, platform, and native teams work with the Mastra-orchestrated system on a daily basis.

Orchestrating separate agents

Riverside has a separate agent for video editing, called the Director and written in Python. It understands the current edit and controls the editor's tools.

When users ask the Co-Creator to make changes to an edit, it delegates the job to the Director. Co-Creator then passes the Director's event stream directly to the user through Mastra's stream writer.

The Co-Creator agent stops after the delegation so it doesn't produce its own answer and doesn't send two responses to the user. The team didn't want the user to have to manage or interact with two agents.

Riverside connected the systems over A2A and exposed the Director to the Mastra agent as a tool.

This implementation is faster than sending the Director's answer through another model call, but it's not ideal. The main agent only knows what happened through the summary it gets afterward. Riverside is working on implementing Mastra's native A2A support and attaching the Director as a subagent.

Giving users more control

Riverside already runs automations after a recording, including generating Magic Clips and show notes. Today, users can control those actions through settings, but the team is already building a workflow experience. Users will be able to arrange different capabilities as steps and decide what should happen after each recording.

The goal is to give the user everything Riverside already offers in a much easier way to manage and see it all in one place.

"Our goal is basically: the user records, has to do nothing, and can publish good content anywhere they want in a click." — Eli Peer, Tech Lead