OpenAI's upcoming Astra model is using a recurrent-depth architecture — also called a looped transformer. A normal Transformer passes each generated token through a fixed stack of layers. Astra can route the same information through the same set of layers multiple times, looping a few extra rounds before producing output.
ByteDance already went public with this approach last year. Its Seed team released Ouro, a Looped Language Model that runs a group of Transformer layers in a loop, pushing more computation inside the model itself. Instead of generating ever-longer chains of thought, the model can increase its reasoning compute. Smaller models can therefore spend more compute and get close to the performance of much larger models.
The architecture also introduces a safety issue. Some of the reasoning happens inside hidden state, where there's no full text record for humans to inspect — making it harder to check via chain-of-thought (CoT) whether the model crossed a line. OpenAI is therefore limiting how much Astra uses recurrent depth, making sure it still keeps a readable chain of thought, and plans to add extra CoT monitoring.
Source: source