Mastra vs. LangSmith: Agents, Evals, and Deployment

Mastra and LangSmith both offer tracing, evaluation, and deployment tools. To choose between them, start with how you want to build and run your agent.

We built Mastra for developers writing TypeScript agents. It connects agents, tools, workflows, memory, and retrieval with tracing and evaluation. LangSmith provides observability, evaluation, prompt management, and services for deploying agents and running code. The comparison covers both the application framework and these services.

You can also build the application with Mastra and send its traces to LangSmith. Mastra runs the agent, while your team can review its traces alongside other applications in LangSmith. You can keep Mastra's own tracing enabled too.

Mastra vs LangSmith: At a glance

1. Runtime processors, tool hooks, and guardrails

Your application can check and change what an agent sees or does before a model call, between steps, around a tool call, or before returning output. A prompt can describe a policy. Your code needs to enforce it.

Suppose an infrastructure agent chooses a deployment tool with:

environment: production
service: payments-api

The application needs to decide whether the authenticated user can make that change.

Mastra exposes processors around model steps and tool hooks around tool execution. Use processors to redact model context, limit the next step's available tools, or validate a response and retry within a configured limit. Use a beforeToolCall hook to inspect a selected call and stop it before execution. A tool-level onInputAvailable hook receives parsed, validated input.

The tool or service it calls should check the caller's permissions before making the change. Removing a write tool from the next model step does not replace that check.

LangSmith LLM Gateway, currently in public beta, controls traffic to model providers. It offers provider fallbacks, spend controls, and rate limits. Enterprise customers can enable sensitive-data detection and redaction.

You can use a gateway alongside checks in your application. An application running through LangSmith Deployment can also enforce tool permissions in its own code.

Verdict: Use Mastra's processors and tool hooks to check what an agent sees and does while it runs. Use a gateway for rules that apply to model traffic across applications. Keep permission checks in the tools and services that act on the request.

2. RAG and context engineering

RAG retrieves information for the model to use. The quality of that information depends on how you prepare documents, filter results, and rank what comes back.

For example, imagine a documentation agent searching across four API versions. Semantic similarity might rank a deprecated page above the current one because its language happens to match the user's question more closely.

The model can then produce a perfectly reasonable answer from the wrong context.

For this agent, retrieval might look like this:

We support this path directly in Mastra. You can parse and chunk documents, generate embeddings, use your preferred vector store, filter results by metadata, retrieve a wider candidate set, rerank it, and pass the final context to the agent.

You can change each stage separately. If search finds the right documents but ranks them poorly, try retrieving more candidates and reranking them.

LangSmith can trace retrieval separately from generation. Evals can check whether retrieval found useful context and whether the answer used it correctly.

Use those results to decide which part of retrieval to change.

Verdict: Mastra provides the components to build retrieval into your agent. LangSmith can help you evaluate the results, whether you use Mastra or another framework.

3. Dynamic agents and request-scoped configuration

The same agent may need different tools, models, instructions, or credentials for each request. A user's permissions, region, and plan can affect those choices.

Suppose the same support agent serves a free user and an enterprise administrator. The free user might receive read-only tools and a lower-cost model. The administrator should receive account-management tools, tenant-specific credentials, and a different model policy.

In Mastra, we can derive those choices from typed request context.

tenant: acme
region: eu-west
role: admin
plan: enterprise

Get the user's identity and tenant from your server's authentication checks before starting the run. Use those values to choose instructions, tools, and a model or fallback list. Your tool code can look up the tenant's credentials. Keep credentials out of prompts and tool results, and check permissions where the tool accesses data or makes a change.

This avoids maintaining separate definitions such as:

supportAgentFree
supportAgentEnterprise
supportAgentEU
supportAgentUS
supportAgentAdmin

Most of those agents would duplicate the same behavior and eventually drift.

LangSmith Deployment also supports dynamic configuration. Assistants can hold model or tool settings, while custom authentication passes authenticated user identity and other fields into each run. Agent Server can use factory functions when an application needs to construct a graph for a request. Many applications can use the same graph with different settings.

Both can use information from the current request. Mastra passes request context through its agents, tools, and workflows. LangSmith Deployment provides assistant settings and authentication for applications running on Agent Server.

Verdict: Mastra's request context works across its agents, tools, and workflows. LangSmith Deployment also supports configuration for each request. Compare how each connects to your authentication and application code.

4. Model routing and provider portability

An application may use different models for different tasks. A coding agent might make these choices:

dependency lookup -> fast model
small code edit -> general model
architecture review -> reasoning model

Mastra's model router gives agents a common provider/model interface. Your application's model function can select a model or fallback list from request context, and processors can adjust the model for later steps. Your application defines which model to use for each task in the example above.

A configured fallback list can try another model when a provider call fails. Your selection code can also use the customer's plan, region, input format, or a previous step's result.

Mastra also offers a separate Gateway that provides access to multiple models and observational memory. Your application can choose a model and send the request through the Gateway.

LangSmith LLM Gateway provides a central endpoint with model/provider fallback rules, rate limits, spend controls, and optional Enterprise data protection. If several applications need the same provider policy, centralizing it can reduce duplicated configuration.

Verdict: Use application code to choose a model based on the request. Both Mastra and LangSmith offer gateways, so compare their supported providers, controls, and hosting requirements.

5. Multi-agent composition and MCP

Separate agents can help when tasks need different tools, context, or permissions. A development assistant might divide the work like this:

The coding agent might need repository access and test execution. Documentation search might need retrieval but no write permissions. A package lookup is deterministic enough that another agent would only add overhead.

In Mastra, an agent can call another agent, a workflow, or an ordinary tool. Use a tool for a fixed operation such as a package lookup.

Mastra's MCP client can load tools and retrieve resources and prompts. Its MCP server can expose agents, tools, and workflows, so other applications can call them.

LangSmith Deployment also supports MCP and can run applications built with different frameworks through supported adapters and APIs.

Verdict: Mastra lets you connect agents, workflows, and tools in your application code. LangSmith Deployment can run applications built with different frameworks. You can evaluate its hosting and serving options when starting a project.

6. Workspaces, files, and code execution

For an agent that edits code or documents, you need to decide which files it can access, which commands it can run, and what persists between runs.

A coding agent might follow these steps:

Mastra Workspaces give the agent files, indexed search, commands, skills, and permissions to work with as it moves through those steps.

Disabling a file-write tool does not prevent a shell command from writing the same mounted file. Set file-tool permissions, command permissions, and filesystem access together.

In Mastra, isolation depends on the backend. LocalSandbox runs on the application host by default. You can enable native isolation or choose a remote or container sandbox. Check the isolation guarantees of the backend you use.

LangSmith Sandboxes provide managed isolated environments for code execution and are generally available. Consider them if you want a service that manages the execution environment.

Verdict: Mastra workspaces connect files and commands to your agent. When choosing a sandbox, compare isolation, network access, persistence, and deployment requirements for the specific backends you would use.

7. Observability, evaluation, and production improvement

Use traces to inspect how an agent behaved. Use evals to test whether a change improves it.

With observability configured, Mastra instruments agents, workflows, model calls, tools, and other runtime operations. You can inspect those traces in Studio and Mastra Platform. Add instrumentation where you need more detail from application code or external services.

If latency increases after a release, for example, inspect an affected trace to see whether time was spent in a model call, tool, or workflow step.

Trace Intelligence, in private beta for enrolled platform projects using deployed Studio, clusters signals from completed traces into recurring goals, behaviors, outcomes, and sentiment. You can inspect example traces and follow changes in themes over time. Relationships show which themes occur together. They do not prove that one caused the other.

From there, Mastra lets you turn examples into datasets and run experiments against the agents or workflows you are changing. Scorers can use deterministic checks or model-based evaluation, and evals can also run through CI with Vitest.

A trace can become a test for the next release:

LangSmith brings traces, datasets, online evaluation, annotations, and dashboards into a shared platform across frameworks.

For patterns across runs, LangSmith Insights groups traces into generated or predefined categories, extracts attributes, and reports metrics for each category. Reports can run on a recurring schedule. Insights is available on Plus and Enterprise plans. Each report analyzes a sample of up to 1,000 traces.

LangSmith Engine looks for recurring issues, investigates likely causes, tracks matching traces, and creates dataset examples. With a connected repository, it can propose code changes for agents built with Deep Agents, LangChain, or LangGraph. That code-fix support is limited to those frameworks.

In Mastra, you can inspect a trace, change the agent or workflow, and run experiments against it. You can also export Mastra traces to LangSmith for your team to review alongside traces from other applications.

Verdict: Compare Mastra Trace Intelligence and LangSmith Insights for patterns in how people use your agent. Consider Engine if you want it to investigate recurring issues and propose fixes for a supported framework.

8. Deployment and runtime ownership

You can choose an agent framework separately from where you host it.

A production agent, for instance, usually has to integrate with databases, queues, internal services, secrets, authentication, network policy, logs, and deployment pipelines that already exist.

We designed Mastra so the application remains a TypeScript service and can run through Mastra Platform or on infrastructure your team already operates.

If your backend already runs in Kubernetes, the agent can run alongside it and use the same networking, secrets, and internal services.

LangSmith Deployment also supports several operating models: cloud, hybrid, a self-hosted control plane, or standalone Agent Server. Cloud requires Plus or above. Self-hosted Deployment with its control plane requires Enterprise with Deployment enabled. Standalone deployment requires PostgreSQL, Redis, and a LangSmith license. Applications built with other frameworks connect through supported adapters or APIs and run within Agent Server's execution model.

Verdict: You can run Mastra in your existing infrastructure or on Mastra Platform. If you want LangSmith Agent Server to run your application, check the integration work, license, and infrastructure required for your chosen deployment option.

How to pick between Mastra and LangSmith

Start with how you want to build, run, and debug the application.

Choose Mastra to build your agent application in TypeScript

Mastra connects agents, tools, workflows, memory, and retrieval with tracing and evaluation. Use it when you want to build and debug those parts together in TypeScript.

You can configure an agent for each request, coordinate its steps in a workflow, and use traces and experiments to test changes. Your code still needs to handle authorization and connect to your application's services.

Choose LangSmith to manage traces, evals, or deployment across applications

LangSmith gives teams a place to manage traces, evals, datasets, and prompts across applications. Insights helps find patterns in usage. Engine investigates recurring issues. You can also use LangSmith Deployment to run an application from the start.

You can use LangSmith tracing across different frameworks without adopting LangSmith Deployment.

Use Mastra to build the agent and LangSmith to review its traces

For example, build a TypeScript support agent and its workflows in Mastra, then send traces to LangSmith for your team to evaluate and investigate. Mastra's LangSmith exporter supports that setup. You can retain Mastra's own tracing destination as well.

Try the combination with representative traces and evaluation cases, then decide which tools your team wants to maintain.

Getting started with Mastra

Use the current quickstart to create a project and inspect its generated configuration.

1. Create a project

npm create mastra@latest agent-app -- --llm openai

Complete the setup prompts. The generated project can include workspace, shell, memory, and other tools. Check which tools are enabled before giving the agent access to files or services.

2. Configure your model provider

Add the OpenAI key to the project's environment file:

OPENAI_API_KEY=your_openai_api_key

Keep the real credential out of source control.

3. Run Mastra Studio

cd agent-app
npm run dev

Open http://localhost:4111. Use a development workspace and inspect the configured tools, model calls, and traces.

4. Test one behavior in a development workspace

Give the agent a fictional fact:

Then correct it:

In the same chat, ask for the launch date. This checks whether the agent uses the updated conversation context. To test persistence, configure persistent storage and reopen the same stored thread after a restart. A new thread can have different history.

Save a reproducible failure as an evaluation case, change the implementation, and rerun it. A successful context test does not validate write permissions: check authorization, tool access, and sandbox configuration separately before connecting production systems.

Ready to test Mastra? Sign up to try Mastra.