<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Benchmark Audit on SoloSoft</title><link>https://www.solosoft.dev/tags/benchmark-audit/</link><description>Recent content in Benchmark Audit on SoloSoft</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Thu, 22 Oct 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://www.solosoft.dev/tags/benchmark-audit/index.xml" rel="self" type="application/rss+xml"/><item><title>Headroom at 77,253 Stars: 300 Log Lines Compressed to Zero, and Four Different Savings Numbers for the Same Tool</title><link>https://www.solosoft.dev/post/headroom-77253-stars-log-compression-audit/</link><pubDate>Thu, 22 Oct 2026 00:00:00 +0000</pubDate><guid>https://www.solosoft.dev/post/headroom-77253-stars-log-compression-audit/</guid><description>&lt;p&gt;Headroom compresses what your agent reads before the model sees it. The GitHub description states the pitch in one line: &amp;ldquo;20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers.&amp;rdquo; The PyPI page says &amp;ldquo;Cut costs by 50-90%&amp;rdquo;. The README proof table says 21%, 57%, 42% and 30%.&lt;/p&gt;
&lt;p&gt;Those are four different claims on four surfaces the same team controls. This article measures what the library actually does to seven kinds of payload, and then compares every published number against every other published number, because the compression tool&amp;rsquo;s own repository contains the evidence that its front-page figures were not reproducible — &amp;ldquo;by anyone, including us&amp;rdquo;, in the words of a script it ships.&lt;/p&gt;</description></item><item><title>Ponytail at 169,651 Stars: a 2,721-Byte Ruleset, and the 26% Saving That Shrinks to 1.7% Once the Code Is Checked</title><link>https://www.solosoft.dev/post/ponytail-169651-stars-cost-savings-audit/</link><pubDate>Wed, 21 Oct 2026 00:00:00 +0000</pubDate><guid>https://www.solosoft.dev/post/ponytail-169651-stars-cost-savings-audit/</guid><description>&lt;p&gt;Ponytail is one markdown file. The project says so itself, in a section called &lt;em&gt;The prompt&lt;/em&gt;: &amp;ldquo;Ponytail is one prompt: &lt;code&gt;skills/ponytail/SKILL.md&lt;/code&gt;. The compact version, for agents that read a rules file, is &lt;code&gt;AGENTS.md&lt;/code&gt;. Everything else in this repo loads that prompt into different agents.&amp;rdquo;&lt;/p&gt;
&lt;p&gt;That is 2,721 bytes for &lt;code&gt;AGENTS.md&lt;/code&gt;. Around it: 214 files, 12,066,966 bytes, and 169,651 GitHub stars. The ratio is 4,435 to 1.&lt;/p&gt;
&lt;p&gt;This article is about the number that gets quoted — &lt;em&gt;53% less code, 41% faster, 26% cheaper, 45% fewer tokens&lt;/em&gt; — and what happens when you open the per-task table behind it and sort it by which tasks have a correctness check. Two statistics of mine reproduce the vendor&amp;rsquo;s headline to the decimal, so the table is the one they used. Sorted differently, it says something the headline does not.&lt;/p&gt;</description></item></channel></rss>