Sapient Intelligence just open-sourced HRM-Text, a 1-billion-parameter text generation model that dramatically cuts the cost of pretraining. The key is a novel hierarchical reasoning architecture that reduces compute requirements by a factor of 130 to 600.
The model was pretrained on just 40 billion structured tokens — roughly 0.1% of what a comparable model would typically need. In benchmark tests, training the 1B version from scratch took about 46 hours on two 8-GPU H100 servers, costing roughly $1,472. The 0.6B variant needed only a single node running for 50 hours, at a hardware cost of around $800. Sapient has also open-sourced the full engineering pipeline, including data extraction, sequence packing, and PyTorch distributed training.
The efficiency comes from a dual-timescale recurrent design, with two sets of Transformer modules — a fast (low-level) and a slow (high-level) — that iterate over the same input in alternating fashion, exchanging information through state addition. This lets the model scale its computational depth dynamically by increasing the number of cycles, without adding physical parameters.
By slashing the pretraining barrier, Sapient opens the door for researchers to test theories that were previously too expensive to validate. But the released weights are raw, unaligned pretrained parameters — the model can only do prefix continuation tasks, not act as a question-answering assistant.