agent-desktop Audited: 16,842 Stars, 42 Watchers, and 44 Commands That Answer PERM_DENIED Where No Permission Exists

Vercel Labs has a habit of shipping small tools that look like demos and turn out to be infrastructure. agent-desktop is the version of that habit aimed at a genuinely hard problem: letting an AI agent drive a desktop application without screenshots and without pixel matching.

The pitch is one paragraph long, and it is the right paragraph. The agent reads the operating system’s accessibility tree, gets JSON back, clicks a ref, reads the UI again. No model inside, no vision model in the loop, no coordinates guessed from an image. The GitHub repository has 16,842 stars, was created on 2026-02-19, and shipped version 1.2.0 on 2026-11-05.

Then there is the number the README does not show you. subscribers_count — GitHub’s watchers, the people who asked to be notified — is 42. That is 0.25% of the star count. For comparison, the audits in this series have measured 0.22% to 0.94% across similar projects; 0.25% sits at the bottom of that band. Stars are a bookmark. Watchers are intent.

This is not a hit piece on a good idea. It is what I found when I cloned the repository, read the files nobody links to, installed the npm package on a headless Debian 13 box, and measured what the tool actually does when the environment it assumes is missing.

What it actually is

A Rust workspace with five crates plus the binary crate, compiled per platform and never cross-linked:

Two things stand out that the README’s architecture diagram does not convey.

The Linux crate is 2.75× the size of the macOS one. 124,060 lines against 45,118. Vercel Labs is a macOS shop; every screenshot in the docs is a Mac. The platform with the least personal use has the most code, which is either admirable rigor or a warning about how much of Linux behavior has to be special-cased.

45% of the Rust is in test files. 168,522 of 374,993 lines. The ratio is not uniform: Linux is 52.3% test lines, core is 60.3%, Windows 49.7%, and macOS — again, the platform the team actually runs — is 21.8%. If you believe a file’s test density tells you where the authors’ uncertainty lives, the ordering is Linux, core, Windows, macOS.

There is also a body of writing: 77 Markdown files totalling 1,701,165 bytes, of which 289,981 bytes are the six SKILL.md guides compiled into the binary.

The most interesting file in the repository is a ledger

Most projects hide their uncertainty in issues. This one publishes it as a table.

probes/linux/FINDINGS.md is 1,054 lines. Every row is one experiment with columns for the probe, the client stack (dbus, gi-atspi, tool), the scope (api-contract, app, box), the sentence in the project’s own spec that the experiment tested, the observed capture values, a verdict, and the action the finding obliges. The verdicts across the file:

Ninety-three rows where the project’s own design document lost an argument with the operating system. The rule at the top of the file is the part I would frame: where a raw-D-Bus row and a libatspi row disagree, the raw-D-Bus row wins, because the Rust adapter speaks raw D-Bus.

The numbers inside are the kind you can only get by running things:

• Reading a tree costs about 2.1 ms per node at 9 D-Bus calls per node. GNOME Files, 201 nodes: 434.8 ms median. VS Code, 252 nodes: 431.8 ms. A 2,000-node tree costs roughly 4 seconds.
• Concurrency buys 2.5–3.5×, not 10×. Files drops from 434.8 ms to 123.4 ms with 128 calls in flight; 16 in flight is within 5–15% of 128.
• Properties.GetAll beats five individual Gets by 3.3× — 189.76 ms to 57.62 ms on the same 201 refs. The spec had claimed AT-SPI offers no batch read; the measurement says it does, and the code was rewritten around it.
• GTK4 has a cache that is 7× faster — and GTK3, Chromium and Electron do not. Cache.GetItems returns 208 items in 16.68 ms where the concurrent walk needs 118.7 ms. GTK3’s Terminal and Chromium’s VS Code answer org.freedesktop.DBus.Error.UnknownMethod to the same call. So the fast path is a toolkit check, not a platform feature.

And then the rows that matter for anyone building on this:

The single best document in the repository is smaller and nastier: docs/solutions/runtime-errors/an-atspi-write-that-returns-success-has-not-changed-anything.md. It is a catalogue, measured on live GTK3, GTK4, Qt 6, Firefox and Chromium, of writes that answer true and change nothing (a destroyed GTK3 widget; every Gecko input; a read-only Qt field, because Qt does not enforce READ_ONLY on that path), and writes that answer failure and land anyway (a stopped application answers NoReply, and the queued call is delivered at SIGCONT).

That document is also where the product states its real thesis: a reply proves delivery to a live object at most, and not even that on GTK3. Everything else in the repository — the read-back after every write, the delivered_unverified disposition, the refactor that deleted every unconditional ok: true flag in 2.5 — follows from that sentence.

This is the unusual case of a project whose self-documentation is more valuable than its marketing. If you take one thing from this article, take the ledger.

What happens on a machine with no desktop

The README’s platform table says Linux needs Ubuntu 24.04 or later, x86_64 or aarch64, X11 or Wayland. I ran it on Debian 13, headless, with no display server and no D-Bus session bus — the environment in which CI runners, containers and most agent sandboxes actually live.

Install first, because install is where npm 12 changed the rules:

$ npm install -g agent-desktop
added 1 package in 1s
npm warn install-scripts 1 package had install scripts blocked because they are not covered by allowScripts:
npm warn install-scripts   agent-desktop@1.2.0 (postinstall: node scripts/postinstall.js)

The package ships 23 files in bin/ — six platform binaries across two crates plus checksums — for a 30 MB install, of which you need one 7.1 MB binary. With the postinstall blocked, every command routes through the Node launcher, which prints one warning (once) and tells you the cost:

this command runs through the Node launcher because npm did not run the install script, which adds about 125 ms per call.

I measured it, ten runs each, same command:

The launcher overhead is 41.5 ms, not 125 ms. To be fair to the project: they may mean a heavier command than version, where the launcher’s own cost would be the same 41.5 ms on top. Either way the number an agent harness budgets for is the measured one.

Then the environment itself:

{"version":"2.6","ok":true,"command":"status","data":{
  "permissions":{"accessibility":{"state":"denied","suggestion":"No D-Bus session bus is reachable. Run inside a desktop session, or start one with 'dbus-run-session -- <command>'."}},
  "platform":"linux","supported_surfaces":[],"state_root":"/opt/data/home/.agent-desktop","version":"1.2.0"}}

ok: true, exit code 0, and supported_surfaces: []. The Linux skill does document this: an empty supported_surfaces means the accessibility bus did not answer, not an empty desktop. Credit where due — an agent that reads the guide cannot be fooled. An agent that reads exit codes sees success.

Now the part that made me write this article. Every command that needs accessibility is gated before it touches the adapter, and the gate misreports why.

$ agent-desktop snapshot
{"ok":false,"command":"snapshot","error":{"code":"PERM_DENIED",
 "message":"Accessibility permission not granted",
 "suggestion":"No D-Bus session bus is reachable. ...",
 "disposition":{"delivery":"not_delivered","retry":"safe"}}}

$ agent-desktop list-apps
{"ok":false,"command":"list-apps","error":{"code":"PLATFORM_NOT_SUPPORTED",
 "message":"The AT-SPI accessibility bus is not available in this session",
 "platform_detail":"Failed to connect to address `unix:path=/tmp/hermes-runtime/bus`: No such file or directory (os error 2)",
 "details":{"retryable":false},
 "disposition":{"delivery":"not_delivered","retry":"safe"}}}

Same machine, same missing bus, two different error codes — and the message on the first one is false on Linux, where there is no accessibility permission to grant. The mechanism is in the source, and it is not a bug in one command:

• crates/linux/src/system/bus/bus_fault.rs maps every bus fault — no session bus, service unknown, spawn failure, dead socket — to a single ErrorCode::PlatformNotSupported with details.retryable: false.
• src/command_policy/mod.rs runs a preflight before dispatch. It classifies 44 of the 60 command names as needing PermissionNeed::Accessibility, and if the permission report says denied, it returns ErrorCode::PermDenied with the report’s suggestion attached, before any adapter call happens.

So the same fact — this process cannot reach an accessibility bus — is reported as PLATFORM_NOT_SUPPORTED by the fourteen ungated commands (list-apps, list-windows, clipboard-*, status, permissions, …) and as PERM_DENIED by the forty-four gated ones. I confirmed the split on snapshot, find, key-down, list-apps, list-windows, clipboard-get and screenshot.

The project’s own ledger says which one it intended. Row L1-7 tests the absent-bus case against the spec sentence “If the bus is not running, return PLATFORM_NOT_SUPPORTED” and records the action: map ServiceUnknown and Spawn.ExecFailed to PLATFORM_NOT_SUPPORTED with different platform_detail, and note that both fail in under 2 ms so no timeout budget is needed. The ungated path does exactly that. The gated path does not.

Why this matters beyond tidiness:

1. PERM_DENIED is documented as “Accessibility permission not granted” in docs/json-output.md. On Linux no such permission exists. An agent that branches on error.code — which is exactly what the README tells it to do — will take the macOS remediation branch on a Linux host.
2. disposition.retry is safe on all of them, including the one carrying details.retryable: false. The docs explain why, honestly: retry follows from delivery, and not_delivered is safe. Nothing was sent, so retrying is safe. But the README’s agent rule is “Retry a failed command only when error.disposition.retry is safe”, and the 2.6 notes admit the consequence in one sentence: disposition.retry stays safe because nothing was sent, but repeating the same ref returns the same refusal. A retry loop on a missing bus is 44 commands’ worth of “safe” that will never succeed.
3. The suggested fix names a binary Debian 13 does not install. dbus-run-session comes from dbus-x11; neither it nor dbus-daemon exists on my box. The suggestion is correct for Ubuntu desktop and unusable on the host that produced it.

I tested the batch path too, because a batch executor is where permission gates usually leak. It does not leak — but it does stop: one snapshot entry inside a three-command batch returns PERM_DENIED with details.batch_index and batch_command, and the batch halts at the preflight rather than executing entries around it. That is defensible; it is also not what “bounded, sequential, non-atomic” suggests to a caller that expects the other entries to run.

The four commands that cannot run, by design

The README is admirably blunt: of the 60 command names, 56 are operational and four — key-down, key-up, mouse-down, mouse-up — refuse to run until a daemon can release a held key or button.

The source states the same reason in one line: held input requires a daemon-owned transaction that can guarantee release. There is no daemon in the shipped binary, so the four commands exist to fail closed and point you at press, mouse-click and drag.

It is the right call for an agent-facing tool: an agent that dies mid-drag while holding a mouse button leaves a desktop that no subsequent command can reason about. It is also a shipping decision worth understanding before you design around it — those four names are in --help and in the command table, and nothing in the table marks them as unavailable.

What the Linux guide concedes

The platform skills are 7,771 bytes for macOS, 8,349 for Windows and 17,219 for Linux. The Linux file contains the word “refuse” and its variants 17 times. macOS: zero. Windows: five.

The Linux capability table is the most honest thing in the marketing:

Read that table against the README’s one-line platform claim — “Ubuntu 24.04 or later, x86_64 or aarch64, X11 or Wayland” — and the gap is not dishonesty, it is compression. The README sells the surface; the skill tells you the surface is conditional on the toolkit, the desktop, the display server and the session type.

That compression has consequences for the CI story. ci.yml’s test-linux job installs at-spi2-core, dbus-x11 and xvfb, then runs the Linux test suite under dbus-run-session -- xvfb-run. That is a session bus and a virtual display, with no window manager and no GNOME Shell. The live Linux end-to-end suite is a separate workflow, linux-e2e.yml, whose only trigger is workflow_dispatch, running on vars.LINUX_E2E_RUNNER. The measurements in the ledger were made on a real GNOME 46 desktop by a human driving probes. The gate that runs on every push cannot reproduce that environment.

What it costs to be one person

Ten contributors, 335 contributions. One of them, lahfir, has 230. Three bots account for 83 more, so of the 252 human contributions, 91.3% are one person. The sixth-largest human contributor has three commits; four humans have one each.

Release cadence over 8.5 months: 49 releases, 13 assets each — six platform archives per crate, so 369 assets — 51 tag refs. Commits by month: 81 in the launch month (2026-02), 11, 13, 21, 34, 11, 15, then 68 in September and 62 in October, 19 in the first five days of November.

The version discipline is where the single-maintainer risk shows. Between 2026-10-28 and 2026-11-03 the project shipped 1.0.0, 1.0.1, 1.0.2, 1.1.0 and 1.2.0. Version 1.1.0 carried three breaking changes in the same release: the C ABI major moved to 7 (and ad_resolve_element_exact plus ad_execute_ref_action_exact_with_policy now always refuse with ACTION_NOT_SUPPORTED), the response envelope moved to 2.6, and refs from earlier snapshots stopped resolving on Windows and Linux. CHANGELOG.md is clear about all of it. If you are the C library consumer, five versions in a week is not churn you can absorb casually.

Demand is real, and it is not the same shape as the stars: 38,585 npm downloads in the month to 2026-11-04. Whether that is CI pipelines or agents in production I cannot tell from outside.

The community signal is thin. The original Show HN in May 2026 (posted under the personal lahfir/agent-desktop, before the move to vercel-labs) reached 99 points and 44 comments. A second Show HN in August titled “I spent 3 months making desktop automation stop lying to AI agents” got 6 points and no comments. And the issue tracker reads like a maintainer’s own queue: of the 30 most recent open items, 20 are pull requests, and one of the genuinely user-shaped reports is #569 — snapshot of an elevated window comes back empty and is reported as complete: true — which is the same failure class the solutions directory was written to prevent.

One file that contradicts the product

skills/jev-desktop/SKILL.md is 16,283 bytes, the largest skill in the repository, and it ships in no installed package. It requires a clone of the repository and a TYPESAFE_API_KEY. Its own description says what it does: it drives a desktop app from a plain-language goal without ever reading the accessibility tree, so that “a 150-element JSON tree” stays out of your context.

That is the opposite premise from the tool it lives inside. I do not read it as a contradiction in bad faith — it is a documented experiment in the trade-off this whole category faces. If you want to skip the tree, you need a model that can look at a screen, and then you are back to vision, latency and cost. The interesting part is that the expensive path is checked in, referenced by a skill, and not mentioned in the README at all.

There is also AGENTS.md, which is more precise than the README in the way that internal documents usually are, and which contains the two sentences that best explain this codebase:

Shipped behavior is the code and its tests. A doc that contradicts the code is a defect; fix it in the same change.

Never add app-specific branches keyed by app names, bundle IDs, executables, window titles, or a known app’s UI layout.

The first is why the ledger exists. The second is why the Linux crate is 124,060 lines: an application-agnostic adapter over five toolkits has to absorb GTK3, GTK4, Qt 6, Chromium and Gecko disagreements in shared code, and every disagreement becomes a test.

What I would actually do with it

If you are wiring this into an agent harness, the decisions are concrete.

1. Read data.complete and data.identity_unproven, not ok. Since 2.2, a snapshot that runs out of budget returns ok: true with complete: false and truncated: true. An agent that only checks ok will act on a partial tree.
2. Branch on error.code and error.suggestion, in that order, and treat PERM_DENIED on Linux as a bus problem. The 44/14 split means the code alone does not identify the cause.
3. Do not retry on disposition.retry: safe alone. Gate retries on details.retryable when it is present, and cap them. Every error in this article was retry: safe.
4. Pin the version. Five releases in a week, an envelope that moved 2.1 → 2.6, and a C ABI that moved to major 7 inside 1.1.0. A floating range on the npm package is a reinstall away from a payload change.
5. Budget 2 ms per node and expect 2.5–3.5× from concurrency. A 200-node snapshot is roughly 400 ms sequentially, ~120 ms with 128 calls in flight. That is the number your loop’s timeout has to survive, not the tool’s own default.
6. Verify writes yourself if you care. The product already does a read-back for set-value, type and friends; it cannot do one for a secure field, and it says so with delivered_unverified. If your workflow writes text and then acts on it, treat delivered_unverified as unknown.
7. Check status on first contact per host, not once per deploy. supported_surfaces: [] is the single cheapest signal that the host will never work — and it has exit code 0.

And if you are evaluating it against Playwright-style browser automation: this is not that tool, and the README says so. It automates native apps through the accessibility tree, which means it is fast and headless on macOS and Windows, and conditional on the desktop, the display server and the toolkit on Linux. A desktop app that exposes no accessibility tree — a game, a canvas, a custom-drawn UI — is out of reach by design.

FAQ

Is agent-desktop from Vercel? It is a Vercel Labs project (vercel-labs/agent-desktop). Vercel Labs products are experiments, not supported platform products; the README links to the labs product page. The code is Apache-2.0, and the repository was originally published under the author’s personal account in February 2026 before moving to the labs organization.

Does it work on a headless Linux server? Not for reading apps. The binary starts and answers status and version, but every observation needs a reachable AT-SPI bus inside a graphical session. On a headless box, status returns ok: true with supported_surfaces: [] and accessibility.state: denied, and gated commands return PERM_DENIED. Plan for a desktop session, a virtual display with a session bus, or a different host.

Why 44 commands and 15 commands with two different error codes? Because two code paths describe the same fact. crates/linux/src/system/bus/bus_fault.rs maps bus faults to PLATFORM_NOT_SUPPORTED; src/command_policy/mod.rs preflights accessibility-gated commands and converts a denied permission report into PERM_DENIED. Ungated commands (app listing, clipboard, status) never see the policy layer.

Does it really need no screenshots? For observation, yes — the tree is read through AX (macOS), UI Automation (Windows) or AT-SPI (Linux). Screenshots exist as a separate command for app windows, and on Linux they are the most environment-dependent part: X11 needs no permission for an app window, a display capture needs a non-compositing manager, and Wayland goes through portal consent.

What is the 1,054-line findings file for? It is the project’s evidence ledger: one row per Linux experiment with the stack, scope, the spec sentence under test, the captured values, the verdict and the action. It is graded against the project’s own design document, and 93 rows are graded CONTRADICTS. It is the most useful thing in the repository for anyone deciding whether Linux support is real for their desktop.

Is it safe for an agent to run unattended? The design is unusually careful about this: refs are snapshot-scoped, stale refs are refused rather than re-targeted, actions carry a delivery disposition, and four input commands fail closed because no daemon can guarantee a release. The risk is not the tool inventing clicks; it is your harness misreading ok: true on a partial snapshot or retrying a safe error forever.

The bottom line

agent-desktop is the most serious attempt I have read at the problem of making desktop automation legible to an agent, and the reason is not the CLI design — it is that the team wrote down what the operating systems actually do, including the 93 places where their own spec was wrong, and then rewrote the code to match.

The cost of that seriousness is visible in the same files. Linux is 2.75× the macOS adapter and half tests, because five toolkits disagree; 44 of 60 commands report a macOS-shaped permission error on Linux for a condition that has nothing to do with permissions; and the retry advice in the README, applied literally, loops on errors that details.retryable: false already says cannot succeed.

42 watchers on 16,842 stars is not proof the project is unloved. It is proof that the people who starred it have not yet had to debug it on a host without a desktop session. Read probes/linux/FINDINGS.md before you are one of them.

If you are comparing this with other agent-facing infrastructure I have measured, the gstack audit is the same method applied to a prompt corpus, and the Worktrunk audit is the same method applied to a CLI whose hooks fail silently.