“Let the agent use my desktop” is a dangerously large kind of permission. It says almost nothing about which app, which window, who can pull the plug, or what happens when an old screenshot leads a model astray. On September 6, the different-ai/openwork repository merged PR #4463 into its dev branch, replacing its prior desktop-tool arrangement with a native computer-use runtime in which a person approves an app and chooses one window for a short-lived session.
That is a more useful question than whether an agent can click: what, exactly, did it receive authority over? The merged code and its accompanying runtime design make the answer unusually concrete. An agent gets a room, not a house key.
Authority now comes with an address
The session has three deliberately narrow modes: observe can read window text and screenshots; assist can use advertised accessibility controls without bringing an app forward or moving the pointer; and control adds targeted mouse and keyboard input only while the selected window is in front. The chosen app and mode stay fixed for that session. Raising the scope means asking the person again.
The boundary is enforced below the calling JavaScript. OpenWork says the native executable owns the authority, so an orchestration script cannot swap in another process, loosen the mode, or resume a session after a person has taken over. Sessions expire, idle sessions pause, one helper session holds the lock at a time, and only a person can resume. The design also declines several convenient escape hatches: no whole-screen capture, shell, clipboard, URL-opening, scope-upgrade, or global-input fallback tools.
A receipt is not a completed task
The other consequential choice is epistemic. Before an input, the runtime expects a fresh observation of the approved window; an observation lasts only 15 seconds and one action attempt. Its returned dispatched status means the operating system accepted or posted an operation, not that the intended outcome happened. The caller must observe again and make a task-specific assertion instead of letting a model turn a non-error receipt into a success story.
The project’s end-to-end fixture gives that claim some teeth. It selects a “Workspace window” rather than a second disposable window, rejects a peer connection attempting to use the grant, omits a protected field, pauses control when a person types, requires a fresh view after resuming, and revokes the session immediately on Stop. That is stronger evidence of a tested boundary than a demo click—though it remains evidence from a controlled fixture, not proof of safe agent work everywhere.
This is a boundary, not a bunker
OpenWork itself draws the crucial limit. This is a macOS 14+ preview, not a released cross-platform security perimeter; Windows and Linux backends are absent. More importantly, the native window boundary is not an operating-system sandbox. A permitted app can still save a file, send a message, or navigate somewhere consequential, while the calling host still owns action authorization and prompt-injection handling. The implementation also documents a narrow private macOS compatibility dependency for pointer events that fails explicitly when unavailable.
The next signal worth watching is not another claim that an agent can drive a desktop. It is a packaged release and broader repeatable evidence that this app-and-window boundary holds in ordinary, messy software—not only in two disposable test windows—when interfaces change, people interrupt, and the consequence of a mistaken click is real.