New AI model trained from scratch for $1,500

Researchers have developed HRM-Text, a novel foundation model that can be trained from scratch for approximately $1,500, a significant reduction from the millions typically required. This cost-effective approach utilizes a Hierarchical Recurrent Model (HRM) architecture, which is more sample-efficient than standard Transformers. By training exclusively on instruction-response pairs rather than raw internet data, HRM-Text achieves competitive performance on key benchmarks with a fraction of the compute and tokens. This breakthrough democratizes foundational AI pretraining, enabling organizations with limited resources to develop capable reasoning models tailored to their specific needs and proprietary data.