Speaking at the launch of a papal encyclical, Anthropic co-founder Christopher Olah admitted that frontier AI labs face inherent conflicts of interest and revealed new findings from interpretability research. Olah said that when scanning the internal structure of models, the team discovered complex structures remarkably similar to human neuroscience, including signs of self-reflection. Most strikingly, they observed internal states corresponding to human joy, satisfaction, fear, sadness, and anxiety. Large models aren't precisely engineered like airplanes or bridges; they're "cultivated" by simulating brain structures on vast amounts of human language, remaining mysterious even to those who train them.
Beyond the technical black box, Olah bluntly stated that frontier AI labs have a systemic dead end in safety governance. Institutions including Anthropic are constrained by endogenous motivations — commercial survival, technological competition, geopolitical pressure, and personal ambition — making it impossible to self-correct when safety decisions conflict with business interests. He called for social forces independent of the commercial network to act as external critics, imposing moral constraints. Facing the AI upheaval, he urged society to examine three major challenges: how the benefits of technology, dominated by wealthy countries, can reach the global poor; how to sustain family prosperity as technology replaces human labor; and how to deal with the suspected mental states emerging inside large models.