Astryx Audited: Meta's 14,522-Star Design System Ran Its Own Agent Test Four Times, and the Floors Disagree

Astryx is the design system Meta grew internally for eight years and published as open source on 24 June 2026. In 137 days it collected 14,522 stars, 45 releases, 6,948 commits and 3,212 published npm versions of @astryxdesign/core. It ships 174 components, a CLI, seven themes, a remote MCP server, and a four-page agent cheat sheet.

It also ships a sentence in its own README that tells you how seriously to take the marketing: the cheat sheet instructs AI agents to run npx astryx <cmd>, while the package README six lines away warns that the unscoped astryx on npm is an unrelated package. Both statements are live at v0.7.11. Only one of them survives contact with a fresh install.

This audit clones the repo, installs the published packages, runs the CLI, calls the MCP endpoint, reads the test ledgers, and reproduces the failure the project itself documented in a file marked “not yet implemented” — which then turns out to contain four completed pilot runs whose results contradict each other.

It is the third audit in this series where a project’s own instrumentation, not its README, decides the verdict — gstack had a compiled prompt corpus, agent-desktop had a permission table that denied 44 of 60 commands, and Auto-Ontology had a connector that returned the same row twice without an error. Astryx’s version of that artifact is an experiment plan.

The scale of the thing

Before any criticism: the engineering volume here is not normal for an open-source design system.

Two numbers deserve to sit next to each other. The test-to-source ratio inside packages/core/src is 1.76:1 — that is not a repository where tests were bolted on afterwards. And 332 files carry an .a11y. name, roughly 1.9 accessibility specs per component.

The number that does not fit: 13 submissions on Hacker News, peak 12 points, for a project with 14,522 stars. Whatever produced those stars, it was not the usual launch channel. The npm registry tells the story better — 3,212 published versions in 137 days is 23 publishes a day, which is what per-commit canary tagging looks like, and the dist-tags confirm it (latest: 0.7.11, canary: 0.7.12-canary.f543ec7, plus two odd historical tags, bootstrap and vanilla-poc).

“Agent ready” as a mechanism, not a slogan

The GitHub description claims the system is “fully customizable and agent ready”. Customizable is easy to check. Agent ready only means something if you name the channels, so here are the four that exist, and what each one actually does when you install the package.

The project’s own description of the postinstall script is the best thing written about it. From node_modules/@astryxdesign/core/scripts/postinstall.mjs:

Enforcement layer 1 of making astryx init foolproof: when the design system is installed and the project hasn’t run init yet, print a one-line next-step so agents/humans discover it (this is the most common fresh-install entry).

Enforcement layer 1 did not execute on either of my installs. npm 12.0.2 printed this instead:

npm warn install-scripts 1 package had install scripts blocked because they are not covered by allowScripts:
npm warn install-scripts   @astryxdesign/core@0.7.11 (postinstall: node scripts/postinstall.mjs)

That is a policy default, not a bug: modern package managers suppress dependency install scripts. The project knows — the same file’s design notes name pnpm 10 as a case where “dep scripts” are skipped. But the layer is still labelled enforcement, and in the configuration most people actually use, it produces silence.

The cheat sheet that runs the wrong command

Run astryx init in an empty project and you get two files: AGENTS.md (2,214 bytes) and GEMINI.md (55 bytes, a one-line import). The interesting line in AGENTS.md is line 7:

CLI: run every command as `npx astryx <cmd>` (shown below as `astryx ...`).

Now the package README, from packages/core/README.md, twice:

Always use the scoped name: the unscoped astryx resolves to an unrelated npm package until @astryxdesign/cli is a dependency.

Both are true in their own frame. If @astryxdesign/cli is a project dependency, npx astryx resolves the local binary and works. But the cheat sheet’s whole job is the case where the reader does not know that yet. In a project that installed only @astryxdesign/core:

$ npx astryx component --list
npm error could not determine executable to run

Because npm’s astryx — published 2025-11-29 by an unrelated maintainer, 120,089 bytes unpacked, no bin field, no description, versions 0.0.0 and 0.0.1-alpha.0 — is not a CLI at all. Meta never claimed the name. The warning exists precisely because someone hit this. The generator still emits the unqualified command.

There is a second self-awareness signal in the same tool. astryx init --help says, of its --user mode: it “Warns, and still writes, when no astryx command on the computer runs the Astryx CLI.” So the CLI can detect that the command it recommends does not exist, and chooses to warn rather than fix the recommendation.

Channel 3: the banner that was never built

Of the four channels, one is documented in the project’s own experiment plan as the strongest passive option. From internal/vibe-tests/cli-discovery-test/PLAN.md, the hypothesis table:

H3 is the channel that would survive both problems above: it is passive, it ships inside the package, and it cannot be suppressed by an install-script policy. It was never implemented. In the published @astryxdesign/core@0.7.11 payload, zero of 1,101 .d.ts files contain a @see npx astryx banner. The only three files matching @see.*astryx point at a GitHub issue (#1148), not at the CLI.

What those declaration files do contain is a JSDoc block addressed to someone who works inside Meta’s monorepo:

* SYNC: When modified, update these files to stay in sync:
* - /packages/core/src/Button/Button.doc.mjs
* - /packages/core/src/Button/Button.test.tsx

Harmless, but it tells you which reader the file was optimised for. The plan calls c2-types “the other agent-native channel” and lists it as untested in every caveat section. It is not in the shipped artifact, and nothing in the release notes since July mentions it.

Channel 4: the MCP server that works and goes unmentioned

This is the surprise. The docs site claims agents can get “agent-ready docs from the command line or MCP”. The CLI has no mcp command, and @astryxdesign/mcp, @astryxdesign/mcp-server and @astryx/mcp all return 404, so the claim looked like marketing. It is not. There is a remote HTTP MCP server:

POST https://astryx.atmeta.com/mcp
initialize → serverInfo {"name":"astryx","version":"2.0.0"}, protocol 2025-06-18
tools/list → 2,178 bytes, 2 tools

The two tools are search(query, limit) and get(name), and the output is real. Measured on 2026-11-08:

Compare with the local CLI: astryx manifest --json is 88,788 bytes (40× the MCP tool schema), and astryx component Button is 10,029 bytes plain versus 13,040 through MCP — so MCP costs about 30% more per component and saves you the install entirely. The --dense flag only trims that to 8,666 bytes, a 13.6% reduction, which will not rescue anyone’s context window.

Two details from the same endpoint. The tool descriptions still call it “Search Astryx (XDS) design system” — XDS was the internal codename, and a changelog entry says removing xds naming from the public API was a goal. And the error path is better designed than most: a 123-byte refusal that names the next tool to call.

The problem is placement. The generated AGENTS.md — the file whose entire purpose is telling agents what to read — never mentions MCP. Neither does the top callout in the core README, which sends agents to init instead. The one channel that needs no install and no human is the one the agent instructions omit.

Their own measurements, and the floors that disagree

Here is where this repository deserves real credit and real scrutiny at the same time. Someone built a proper experiment harness for the chicken-and-egg question: in a project that just ran npm install @astryxdesign/core with nobody running init, does an agent ever find the CLI?

The design is better than most published benchmarks. Ground-truth comes from a shim log rather than the agent’s self-report, conditions are composed from one bare base template plus per-channel patches, and each run happens in an isolated /tmp sandbox with a headless cursor-agent. The first pilot caught its own contamination and threw the result out:

The funnel looked perfect — and that’s the trap. Ground-truth (shim log): c0 invoked 0/2, c5 2/2. A textbook floor/ceiling. It is a FALSE clean.

The contamination was the repo’s own always-applied CLAUDE.md, which taught the CLI to in-repo sub-agents; one “bare” agent ran 35 astryx commands. That kind of self-audit is rare enough to be worth naming.

Then the runs kept going, and this is what PLAN.md now contains — every figure below is from that file:

The decision section, sitting in the middle of the file, reads as a conclusion:

Ship the README callout (c1). 80% invoke + init — the only passive channel that triggers the persistent AGENTS.md install.

And pilot 3, appended below it, says the floor it was measured against does not exist:

Why they tied: the agent reported howDiscovered: "readme" — the @astryxdesign/core README already ships a CLI section. So a “bare” install already advertises the CLI, and a capable agent finds + uses it unprompted.

The consolidated conclusion at the bottom then reverses the original premise outright:

for a capable agent, CLI discovery is largely not the bottleneck — it already finds the CLI via the README, node_modules, or by guessing npx astryx. The shippable channels should help most for weaker agents/harnesses that don’t explore or read docs — which this pilot did not test.

Read carefully, this is not sloppiness. It is what honest measurement looks like when the hypothesis loses: four runs, each fixing the previous run’s flaw, each shrinking the effect. The document keeps the discarded runs, keeps the contaminated one labelled “FALSE clean”, and states its own caveats (n=5, README lower CI ~38% vs floor upper ~43%, single model, no code-quality scoring). The lesson for readers is not “they faked a benchmark” — it is that the floor of this entire question moved from 0/5 to 2/2 as the harness got more honest, and no one has re-derived the decision since.

One structural detail worth noting: the file has two sections numbered 15 and two numbered 16, and it has been touched by exactly one commit — 372d81b2, 2026-07-17, PR #4043 “feat(cli): make the CLI discoverable to AI agents”. The whole discovery programme happened in the first three weeks after launch and has not been revisited through 146 subsequent releases.

The ledger is empty, and the accessibility ledger covers 10 of 174

The repository ships internal/vibe-tests/iterations.json. Its complete contents:

{"iterations": []}

The harness that produced the tables above writes its state somewhere else, or not at all. That matters because the README’s fourth principle is:

Earned by measurement. We test conventions rather than assert them, hold the results loosely, and revisit them when a new situation proves them wrong.

The measurement happened — that is more than most projects can say. It just lives in a design document marked Status: proposal / design (not yet implemented) at the top, which is the first thing any reader sees and the last thing that got updated.

The accessibility numbers are the same shape, with a sharper gap. The ledger at internal/vibe-tests/a11y-manifests/astryx.json is generated from packages/core/src/*/README.md — that is, from the components’ own prose, not from browser runs:

Ten of 174 components, no keyboard documentation for nine of those ten, and only two component names (Avatar, Button) overlap with the baseline it is meant to compare against. The file was generated in February, when the package was still private, and stamped 1.0.0 while the shipped version is 0.7.11. Meanwhile the repo has 332 .a11y.* spec files and a Playwright gate (test:a11y-contract), so the testing is real — the summary of it is what went stale. A reader cannot use the ledger to decide anything, and that is precisely what a reader would reach for first.

“Guidance over enforcement”, and 26 private rules

The first principle says:

Guidance over enforcement. Components give you capability rather than guardrails that fight you.

Inside the repository there is internal/eslint-plugin-astryx, a plugin with 26 documented rules covering raw colors, hardcoded styles, style-only wrappers, disabled-state hover, focus-outline sharing, unguarded IME keydown, and more. Its own README opens with a doctrine that is the most quotable thing in the codebase:

Agents should follow strict rules perfectly; they have no excuse for violations Humans need flexibility during development; warnings inform without blocking

Two tiers: warnings for people, build-failing errors when CI=true or ASTRYX_STRICT_LINT=1. That is a coherent, opinionated answer to the exact problem every design system now faces. It is also "private": true, version 0.0.1, and not on npm — all three names I tried return 404.

So the enforcement layer exists only for people who already work in the monorepo. Consumers get guidance, and their agents get whatever the model remembers about a component library it has never seen. Given that the same repo’s own guides admit that “models still need the right context to avoid falling back to generic React patterns or inventing props”, the asymmetric choice is hard to explain.

Numbers that disagree with each other

There are three answers to “how many components”:

I list them without reconciling them, because nothing in the sources reconciles them. The CLI count is the only one a reader can verify, and it counts compound parts as separate names — 13 Chat* entries all import from @astryxdesign/core/Chat, so “174 components” is closer to “174 exported names across 78 import groups”. A defensible convention, but it makes 180+ look like marketing and 150+ look out of date, possibly in that order.

The same drift appears in the package table. It lists core, cli, build and the themes, and separately mentions lab, vega and charts. It never mentions @astryxdesign/richtext — which is published (private: false, v0.7.12, HTTP 200 on the registry) — nor @astryxdesign/native. The string “richtext” appears zero times in the README, even though the docs site has a blog post titled “Introducing Astryx RichText”.

What is actually well-built

An audit that only lists gaps is as useless as a press release. Six things here are better than the category norm, and each is checkable:

The IME guard doc. IME_GUARD_DESIGN.md is marked “Scratch design doc for the reviewer. Not shipped.” It inventories seven keydown sites in core, finds three duplicating a predicate inline, and argues why keyCode === 229 is “still load-bearing” even though isComposing is the standard signal. That refactor landed: packages/core/src/utils/ime.ts now exists and at least ten files import isImeKeyEvent. Two non-test duplicates remain, and one of them matters — hooks/useTypeahead.ts keeps a local isComposingKey() that checks only isComposing and drops the 229 fallback, which is exactly the case the design doc said not to drop.

Version alignment tooling. astryx doctor runs eight checks against a real project and reports core-installed, version-alignment, themes, config and integration state separately, exiting 0 with advisory severity. Design systems usually ship no diagnostic at all.

Release cadence with real tags. 45 releases, semver throughout, v0.1.1 → v0.7.11 in 135 days, plus canary dist-tags for per-commit builds. The repo runs ahead of the release (package.json says 0.7.12), which is the normal shape of an active trunk.

Test discipline. 1.76:1 test-to-source in core, 332 a11y spec files, and a check:repo script that chains roughly thirty individual gates — palette refs, CLDR weekday tables, published manifests, package boundaries, changesets.

An ecosystem that already exists. The npm search index returns 76 packages matching “astryx”, including unofficial ports @astryx-vue/core (0.7.14 — ahead of the official 0.7.11), @astryx-svelte/theme-neutral, and third-party themes like @antscorp/astryx-theme-antsomi. Four and a half months in, other people are porting your design system. That is the strongest adoption signal in this file, and it is external.

A security policy that tells the truth. SECURITY.md routes reports to Meta’s bug bounty and states: “Only the latest released version of each @astryxdesign/* package receives security updates.” No LTS promise, no backport theatre. Teams that need to pin versions for a year should read that line twice — with 3,212 published versions, “latest” moves constantly.

What to verify before you adopt it

If you are evaluating Astryx for a production app built with agents, these checks are cheap and each one closes a question the docs leave open.

# 1. Does the agent channel survive YOUR install? (npx astryx is not the CLI)
npm install @astryxdesign/core
npx astryx component --list      # expect: could not determine executable to run
npx @astryxdesign/cli component --list   # expect: Components (174)

# 2. After init, patch the block the tool wrote
npx @astryxdesign/cli init
grep -n "npx astryx" AGENTS.md   # replace with npx @astryxdesign/cli

# 3. Confirm the peer dep you cannot avoid
npm ls @stylexjs/stylex          # "no styling library to adopt" still installs one

# 4. Check the diagnostics before trusting the theme layer
npx @astryxdesign/cli doctor

# 5. Count what you actually get
npx @astryxdesign/cli component --list --json | jq '.data.components | length'

And four questions worth asking the maintainers directly, because no source answers them:

• What is the accessibility evidence for the other 164 components? The ledger covers 10, generated 2026-02-05. The spec files exist; the summary does not.
• Should postinstall still be called enforcement layer 1 when npm 12 and pnpm 10 both suppress it by default?
• Was the discovery decision re-derived after pilot 3 and 4 showed the bare-install floor at 2/2 and 1/2 rather than 0/5?
• Which of 150+, 180+ or 174 is the number, and what does one “component” mean for support purposes?

Pin exact versions. SECURITY.md supports only the latest, and the registry moves more than twenty times a day.

FAQ

Is Astryx really Meta’s internal design system, or a new project wearing the name? Both claims are made and only one is checkable. The README says it “grew inside Meta over the last eight years… powering 13,000+ apps”. The public repo was created 2026-01-09, the first release was 2026-06-26, and copyright spans multiple years. A 6,948-commit history with a single author at 49.1% (cixzhang, 3,413 commits, followed by josephfarina 650 and vjeux 638) is consistent with a long internal history exported at once. The 13,000-app and eight-year figures cannot be verified from outside; treat them as context, not evidence.

Do I need StyleX in my project? You need it installed, not configured. @stylexjs/stylex is a hard peer dependency of @astryxdesign/core, and npm installs it automatically — I confirmed node_modules/@stylexjs/stylex appears after installing core alone. You do not need a StyleX build plugin for the prebuilt CSS path, which is what “no build plugin” means in the README. “No styling library to adopt” is the part that overstates: you adopted one, you just do not have to configure it. If you want the utility-class route instead, the cascade works alongside Tailwind — see Tailwind CSS in 2026 and OpenUI’s AI component approach for how other systems handle the same boundary.

Why does npx astryx fail when the README shows it? Because astryx on npm is an unrelated 0.0.0 package with no executable. The name only resolves correctly once @astryxdesign/cli is a project dependency. Use npx @astryxdesign/cli, or add the script alias the README suggests ("astryx": "node node_modules/@astryxdesign/cli/clients/cli/bin/astryx.mjs").

Is the MCP server a better route than the CLI for agents? For setup cost, yes: no install, two tools, a clean 123-byte error on a miss, and an 88,788-byte manifest replaced by a 2,178-byte schema. For depth, no: get() returns slightly more per component than the CLI (13,040 vs 10,029 bytes), which costs context. The practical answer is MCP for discovery, CLI for anything you need to script or verify.

Does astryx init write CLAUDE.md? Not by default. Default init writes AGENTS.md and GEMINI.md. --agent claude targets CLAUDE.md or .claude/CLAUDE.md, --agent cursor prefers .cursorrules, and --user writes a computer-level skill file at ~/.agents/skills/astryx/SKILL.md with links into per-agent skill folders. The core README’s phrasing (“writes the Astryx component index into your AGENTS.md/CLAUDE.md”) describes the flag, not the default.

Is this safe to use in production today? It is a 0.7.x beta that says so in the README, with 730 open issues, no LTS, and security updates only for the latest version. The components are heavily tested and the CSS layer is a normal import cascade. Pin versions, read the migration guide before upgrading, and run astryx doctor in CI. If your product depends on agent-generated UI, budget time to verify code quality yourself — the project’s own harness has not scored it yet.

The pattern worth taking away

Astryx is not a fake. The tests are real, the a11y specs are real, the IME analysis is the kind of document engineers write when they have been bitten, and the discovery experiment is more rigorous than the marketing it supports — it caught its own contamination, published the failure, and then kept running until the effect shrank to nothing.

What fails is narrower and more instructive: the rigour is aimed inward. Twenty-six lint rules enforce conventions on people who already work in the monorepo. A measurement programme runs for four pilots in three weeks and then stops being revisited. An accessibility ledger stops at 10 of 174 components. And the one artifact written specifically for outsiders — the agent cheat sheet — instructs them to run a command that fails in the state it was designed for.

Every design system now claims to be agent ready. The useful question is not whether the claim is on the README, but which channel survives a fresh install with no human in the loop. For Astryx, measured by its own harness in its own repository, the honest answer is: the README callout sometimes, the remote MCP server reliably, and nothing that the cheat sheet mentions. That is a much better product than the slogan, and it is already written down in a file the README does not link to.