Strangers pretrained a language model with HF PRs and a cron job

By somevyn · 2026-08-02 · 1 points · 0 comments

Last week a few of us pretrained a 15M-parameter language model from scratch — past its Chinchilla-optimal token budget — with no cluster, no server, and no budget. The training coordinator is a GitHub Actions cron job. The gradients are pull requests. By the end, someone we'd n…

Open the full discussion on BetterNews