OpenAI chief scientist Jakub Pachocki has a warning: chain-of-thought (CoT) monitoring — reading the reasoning a model writes down, looking for hidden goals or misaligned intentions — is becoming unreliable.
A few days ago, outside observers linked Astra's recurrent depth architecture to the deterioration. Pachocki pushed back, saying the architecture wasn't the cause, and promised to explain in detail. Five days later, he made good on that promise, laying out three reasons:
- Agent reasoning increasingly happens inside tool calls or in conversations with humans and other AI systems, all of which require their own supervision. Looking only at CoT makes it harder to see the complete reasoning process.
- Models are getting better at controlling their own reasoning.
- Stronger pre-training makes models smarter even when they don't write out their reasoning at all.
OpenAI has long treated CoT monitoring as a central safety technique. One key reason o1-preview never got a fully public chain of thought was to avoid direct supervision of CoT — the kind of oversight that can teach models to hide their true intentions. OpenAI's own evaluations now show that approach is becoming less reliable.
Pachocki is blunt about the bigger issue: no lab has solved alignment and monitoring well enough to keep scaling models at full speed indefinitely. He wants labs to slow down voluntarily until shared safety thresholds are established. And if necessary, OpenAI will stop scaling too.
Dongcha Beating Telegram channel · Dongcha Beating discussion group