Computer
Sandboxes that run a desktop environment can expose screenshot, mouse, and keyboard control. The workspace registers computer tools when a statically configured sandbox supports the computer capability.
DaytonaSandbox and E2BDesktopSandbox support this capability. Other sandbox backends don't register the computer tools. Resolver-backed sandboxes don't register them because the workspace can't inspect the resolved sandbox's capabilities when it creates the tool list.
Computer toolsDirect link to Computer tools
| Tool | Does |
|---|---|
computer_screenshot | Captures the desktop as a PNG image and returns it to the model as native media. |
computer_click | Presses and releases the left mouse button at pixel coordinates. |
computer_double_click | Presses the left mouse button twice at pixel coordinates. |
computer_right_click | Presses and releases the right mouse button at pixel coordinates. |
computer_move_mouse | Moves the cursor to pixel coordinates without pressing a button. |
computer_drag | Presses, drags, and releases between two points. |
computer_type | Types text into the focused element. |
computer_press_key | Presses a key or key combination, such as Enter or ctrl+s. |
computer_scroll | Scrolls up or down. |
computer_get_screen_info | Gets the screen dimensions and cursor position. |
computer_wait | Waits for the interface to settle. |
Configure computer toolsDirect link to Configure computer tools
Action tools take a screenshot after each action by default. Configure the screenshot behavior for each tool:
src/mastra/workspaces.ts
import { Workspace, WORKSPACE_TOOLS } from '@mastra/core/workspace'
import { E2BDesktopSandbox } from '@mastra/e2b-desktop'
const workspace = new Workspace({
sandbox: new E2BDesktopSandbox(),
tools: {
[WORKSPACE_TOOLS.COMPUTER.CLICK]: {
screenshotAfterAction: true,
screenshotDelayMs: 1000,
},
[WORKSPACE_TOOLS.COMPUTER.TYPE]: {
requireApproval: true,
},
},
})
Computer tools accept the same per-tool enabled and requireApproval configuration as other workspace tools.