Independent audits of the open-source projects everyone is starring.

We clone the repo, rerun its benchmarks, read its licences and count its stars, then publish what holds up and what doesn't. Alongside the audits: practical guides to AI tools, software engineering and search.

378 articles and 328 trend notes, in English and Spanish.

Latest 18 min read

Codex Security Ships a 2,737-Case Accuracy Benchmark and Publishes No Score: 22,234 Semgrep Alerts Bucketed Into 2,737 Labels, 29.4% of the Evidence Is Vendored Code, and a Cost-Stopped Scan Is Sealed as Completed

Codex Security ships 2,737 SastBench cases, 89.1% of them machine-labelled negatives, publishes no score, and seals a cost-stopped scan as completed.

Read the article

Recent articles

All articles

  1. AWS Emulator Measured: 56.46% of Its Own Fidelity Tests Pass, 8,261 of 40,130 Operations Answer 501, and 306 Return Success Where AWS Returns an Error

    AWS Emulator's own ledger: 24,243 of 42,937 fidelity units pass (56.46%), 8,261 of 40,130 operations answer 501, and 306 return 2xx where AWS returns 4xx.

    Developer Tools

  2. MuJoCo 4.0 Measured: 2,959× Better Integration, One Silent Actuator Change, and a Load Error That Never Says Why

    MuJoCo 4.0.0 has 61 changelog items, 13 in breaking boxes. Run side by side, RK4 error fell 2,959× and an armature-free actuator whipped to 447 rad/s.

    Open Source

  3. OpenMausBot Audited: 18,250 Stars, 1.07M Lines, and a Sandbox That Switches On Only in the Cloud

    OpenMausBot hit 18,250 stars in 89 days. Its bubblewrap sandbox is Apache-2.0, yet it defaults on only in the paid cloud; the self-host image lacks bwrap.

    AI

  4. Astryx Audited: Meta's 14,522-Star Design System Ran Its Own Agent Test Four Times, and the Floors Disagree

    Meta's Astryx has 14,522 stars and 6,948 commits, yet its own agent-discovery pilots measure bare installs at 0/5, 2/2 and 1/2. What ships, what does not.

    Developer Tools

  5. Auto-Ontology Audited: 9,026 Stars, 46 Watchers, and a SQLite Connector That Returns Ada Twice While Reporting No Error

    Auto-Ontology has 9,026 stars, 46 watchers, no GitHub releases, and a SQLite connector that returns Ada twice while reporting no error.

    AI

  6. agent-desktop Audited: 16,842 Stars, 42 Watchers, and 44 Commands That Answer PERM_DENIED Where No Permission Exists

    agent-desktop has 16,842 stars and 42 watchers. On a headless Linux box, 44 of its 60 commands returned PERM_DENIED for a missing D-Bus bus.

    Developer Tools

  7. gstack Audited: 139,339 Stars for 'All Markdown' Hides 689,039 Lines of TypeScript, a 122-Version Lag Behind the Last Tag, and 20,616 Tokens Before /review Reads Your Diff

    gstack has 139,339 stars but 766 watchers. I cloned it: 689,039 TypeScript lines behind 'all Markdown' and 20,616 tokens before /review reads a diff.

    Developer Tools

  8. Worktrunk Audited: 153 Releases in 351 Days, a post-start Hook That Fails While wt switch Exits 0, and hash_port Ports That Collide Across Repositories

    Worktrunk has 10,479 stars and 153 releases in 351 days. I ran the binary: a failing post-start hook exits 0, merge deletes your cwd, hash_port collides.

    Developer Tools

  9. Scrapling Audited: 92,848 Stars, 1.9 Issues per 1,000 Stars, and an Adaptive Matcher That Scores 47% on the Wrong Element

    Scrapling has 92,848 stars but 179 issues. I ran its benchmark (broken), its adaptive matcher (61.9% right, 47.08% wrong) and its fetchers. Here is what holds.

    Developer Tools

  10. Lightpanda Measured: 40,121 Stars, 98 Watchers, and 62% of Its WPT Passes Come From One Encoding Suite

    Lightpanda passes 1,841,066 WPT subtests, but 62.5% come from one encoding suite. Measured on 10 real pages: 5.5s vs 12.8s, one page per process.

    Developer Tools

  11. dsh-superagent: DeepSeek's 6,447-Star Prototype Compiles Its Philosophy Into Character Limits

    dsh-superagent: 6,447 stars, 2 commits, 34 watchers — and a philosophy of conserving attention enforced as hard caps: 100-word acks, 4,000-char pages.

    AI

  12. CVE-2026-98432: A Two-Statement Kernel Fix Scored 9.8, 8.1 and High by Five Vendors

    Two kfree() calls became kfree_rcu(). The scores never converged: CNA 9.8, Oracle and Amazon 8.1, SUSE 8.1/9.2, Ubuntu High. Trackers read 1 Nov.

    Infrastructure