cua-driver
Drive a native macOS app via the cua-driver CLI (default) or MCP server — snapshot its AX tree, click/type/scroll by element_index, verify via re-snapshot. Use when the user asks you to operate, drive, automate, or perform a GUI task in a real macOS application on the host (e.g. "open a file in TextEdit", "navigate to /Applications in Finder", "click the Save button in Numbers").
npx skills add melandlabs/openloomi --skill cua-driver --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
What it does
Orchestrates macOS app automation via cua-driver. When a user asks to drive a native macOS app, follow the loop in this skill and require a snapshot-before-action invariant. It operates by issuing cua-driver commands to drive GUI elements, types, clicks, and scrolling using element_index, and verifies actions by re-snapshotting. It can operate a real macOS application in the background, without changing the frontmost app, and provides a canonical workflow for launching apps, querying window state, clicking by element_index, and stopping the daemon.
How it works
- Maintains the no-foreground contract: the target app must not be foregrounded unless explicitly requested; avoid commands that activate or bring apps to front. Before any shell command, it asks if the command would raise/activate/foreground the target; if so, it avoids running it.
- Uses cua-driver as the primary tool: default transport is the cua-driver CLI; commands take the form cua-driver <tool-name> '<JSON-args>'. It supports launching, getting window state, clicking, and taking screenshots, with explicit guidance to translate actions into cua-driver equivalents.
- Enforces the snapshot-before-action invariant: every action must be bracketed by get_window_state(pid, window_id) before and after the action to ensure AX state is current and element_index is valid.
- Provides a canonical workflow: start the cua-driver daemon, launch an app, retrieve a window state, perform a click by element_index, then stop.
- Details about safe usage, including how to handle frontmost state, how to map intents to cua-driver tools, and when to avoid certain commands that could disrupt the user’s current context.
When to use it
Use when the user asks you to operate, drive, automate, or perform a GUI task in a real macOS application on the host (e.g. "open a file in TextEdit", "navigate to /Applications in Finder", "click the Save button in Numbers").
What it can touch
- Tools and commands described as cua-driver commands:
cua-driver <tool-name> '<JSON-args>'for actions like launch_app, get_window_state, click, etc. The section referencesopen -n -g -a CuaDriver --args serveto start the daemon and commands likecua-driver launch_app,cua-driver get_window_state,cua-driver click.
Caveats
- The frontmost app must not change unless explicitly requested. If a command would foreground the target, it must be avoided. The no-foreground contract and activation rules are strict. If violations occur, the workflow directs to avoid them.
- The instruction set requires snapshot-before-action for every operation; missing a snapshot invalidates the action sequence.
- The agent should translate generic intents into cua-driver tool calls and refrain from using non-cua-driver methods that would bypass the snapshot protocol.
# cua-driver Orchestrates macOS app automation via `cua-driver`. Whenever a user asks to drive a native macOS app, follow the loop in this skill rather than calling tools ad-hoc — the snapshot-before-action invariant is not optional and silently breaks if you skip it. ## The no-foreground contract — read this first **The user's frontmost app MUST NOT change.** This is the whole reason cua-driver exists. Users pay for the right to keep typing in their editor while an agent drives another app in the background. Violate this rule and every other nice property the driver gives you (no cursor warp, no Space switch, no window raise) stops mattering — you just shipped the Accessibility Inspector with extra steps. Before running any shell command, ask: **"does this raise, activate, foreground, or make-key any app?"** If yes, don't run it. Every one of the commands below activates the target on macOS and is therefore forbidden unless the user **explicitly** asked for frontmost state: - **Every form of the `open` CLI — `open -a <App>`, `open -b <bundle-id>`, `open <file>`, `open <path-to-App.app>`, `open <url>` — always activates.** macOS routes all forms through LaunchServices, which unhide
- The no-foreground contract — read this first
- Defaults — always prefer cua-driver over shell shims
- Claude Code computer-use compatibility mode
- The narrow carve-out
- Self-check pattern
- Prerequisites — check before starting
- Using cua-driver from the shell
- Agent cursor overlay
- The core invariant — snapshot before AND after every action
- Why window selection is the caller's job now
- Behavior matrix
- The canonical loop
- 1. Resolve target pid — always via launchapp
- 2. Snapshot and act by elementindex
claude mcp add --transport stdio cua-computer-use -- cua-driver mcp --claude-code-computer-use-compat
write to file — stdout stays readable (AX tree / summary only, no base64)
cua-driver get_window_state '{"pid":N,"window_id":W,"screenshot_out_file":"/tmp/shot.jpg"}'
CLI --screenshot-out-file flag is equivalent and works for all capture modes
cua-driver get_window_state '{"pid":N,"window_id":W}' --screenshot-out-file /tmp/shot.jpgWhat does the cua-driver skill do?
Drive a native macOS app via the cua-driver CLI (default) or MCP server — snapshot its AX tree, click/type/scroll by element_index, verify via re-snapshot. Use when the user asks you to operate, drive, automate, or perform a GUI task in a real macOS application on the host (e.g. "open a file in TextEdit", "navigate to /Applications in Finder", "click the Save button in Numbers").
How do I install it?
Run `npx skills add melandlabs/openloomi --skill cua-driver --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From melandlabs/openloomi, a repository with 764 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.