When an agent clicks a button on somebody else’s desktop, OK can be dangerously thin information. It may mean the driver accepted an input, not that the interface changed. A new NousResearch/hermes-agent computer-use change gives that difference a first-class result: the model can receive a verification flag, an effect verdict, an escalation recommendation, a delivery path, and a delivery mode instead of treating a non-error return as a completed action.
That NousResearch/hermes-agent commit describes the failure plainly: the wrapper had discarded the driver’s structured verdicts, allowing an unverified no-op to be reported as success. The repair keeps ok as transport status and adds a more useful question: was the effect confirmed, unverifiable, or a suspected_noop? That makes uncertainty visible to the model that must choose the next action.
Escalation is not a hidden retry
Hermes’ published computer-use instruction starts in the background so an agent does not steal the person’s cursor or keyboard. A returned unverifiable result calls for a fresh capture; a suspected_noop can recommend pixel addressing or foreground delivery. The point is to react to the driver’s evidence, not to guess that an application will fail because it uses Electron, Chromium, or another framework.
Foreground mode is deliberately a different rung. The public tool schema calls it a visible focus change and says it needs its own approval. The accompanying tests require an older driver to return foreground_unsupported rather than silently fall back to background input; they also keep a background approval from authorizing foreground work in another session.
Make the result describe the effect
The reusable lesson is less glamorous than “give the agent more control.” For any tool that touches a shared surface, distinguish “the request arrived” from “the intended state changed,” and let the evidence—not optimism—select the fallback. The new regression tests prove that contract against fake driver sessions, not across every real application or platform. The next meaningful public signal would be a released driver compatibility record or an end-to-end test showing which failures really climb the ladder and which must still stop for a person.