<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Small Language Models on SoloSoft</title><link>https://www.solosoft.dev/tags/small-language-models/</link><description>Recent content in Small Language Models on SoloSoft</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Sat, 21 Nov 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://www.solosoft.dev/tags/small-language-models/index.xml" rel="self" type="application/rss+xml"/><item><title>MiniMind Audited: 63,912,192 Parameters, a 52,762-Row 'LoRA Sample' That Is 99.25% of the CEval Test Split, and Four Definitions of 'Two Hours'</title><link>https://www.solosoft.dev/post/minimind-64m-2-hours-audit/</link><pubDate>Sat, 21 Nov 2026 00:00:00 +0000</pubDate><guid>https://www.solosoft.dev/post/minimind-64m-2-hours-audit/</guid><description>&lt;p&gt;MiniMind has 67,682 stars and a one-line pitch that fits in a screenshot: train a 64M-parameter language model from scratch, on one RTX 3090, in two hours, for about three yuan. The repo is Apache-2.0, the data is public on Hugging Face, and the training scripts are 4,695 lines of plain PyTorch. It is the rare project where almost every claim can be checked from a filesystem — so I checked them.&lt;/p&gt;
&lt;p&gt;Three claims survived contact with the files. Several did not, and the interesting part is that the failures are not the usual ones. The parameter count is exact to the digit. The compute budget, once you convert it to FLOPs, is &lt;em&gt;plausible&lt;/em&gt; — it implies a 3090 running at 44% of its bf16 tensor peak, which is a real number a real person could hit. What doesn&amp;rsquo;t hold together is the bookkeeping around that number, and the description of the dataset used to produce the project&amp;rsquo;s most flattering benchmark row.&lt;/p&gt;</description></item></channel></rss>