Your mouse cursor is safe — for now. The open-source "computer-use" infrastructure project Cua just released cua-driver, a native macOS driver that lets any AI agent manipulate Mac applications in the background. When an agent clicks, types, or takes a screenshot, your cursor doesn't budge, your focus doesn't shift, and macOS doesn't bounce you between Spaces.
The core technology comes from reverse engineering Apple's private SkyLight framework. Normal synthetic events sent via CGEventPost travel through the HID event stream and physically move the cursor. CGEvent.postToPid can target a specific process, but Chromium's renderer filters those events out. Cua-driver uses SkyLight's SLEventPostToPid to send events through the WindowServer's trusted channel, bypassing HID entirely — even Chromium picks them up. For window activation, it borrows a trick from the window manager yabai: using SLPSPostEventRecordTo to toggle only the target app's AppKit activation state without raising its window level, so Spaces doesn't follow. For Electron apps (Slack, VS Code, Discord, etc.), it uses the undocumented _AXObserverAddNotificationAndCheckRemote to keep the accessibility tree updated even when the window is obscured.
Cua-driver offers three capture modes: ax mode returns only the accessibility tree (no screen recording permission required), vision mode returns only a screenshot, and som mode (default) returns both — agents can click either by element index or pixel coordinates. The driver supports the MCP protocol, meaning it can plug into clients like Claude Code and Cursor, or be invoked via the command line. Two known limitations: right-click doesn't work on Chromium web content, and Canvas-based apps (Blender, Unity, game engines) still need a brief frontmost activation.
After OpenAI acquired the former Apple Shortcuts team Sky, its Codex agent was the first to offer a background computer-use feature — and it recently added a chronicle feature that watches your screen. But Codex's implementation is not open-source. Cua's Francesco Bonacci says that a background computer-use driver should be universal infrastructure, not a gimmick locked to one product.