Anthropic Is Handing Outside Auditors the Keys to Its Models

A resignation post from former Anthropic researcher Jacob Coxon pulled in more than 160 million views in three days, and the AI extinction debate shoved its way into the American mainstream this week. OpenAI chief scientist Jakub Pachocki had already urged peers to coordinate a slowdown if needed. After Coxon's post blew up, Sam Altman said internally he'd be willing to slow down alongside other labs, and OpenAI began pushing for mandatory national AI safety regulation.

Now Anthropic has something concrete on the table. The company promises to put third-party safety teams on site permanently, with desks, badge access, company laptops, and tools and permissions close to those of its internal risk-assessment team. They'll be able to inspect training pipelines, safety measures, and incidents, and publish their conclusions independently. Anthropic can redact material involving safety, legal, and customer privacy — but not conclusions just because they're ugly. CEO Dario Amodei says the arrangement goes further than any AI company has so far.

Amodei's stance on slowing down has shifted too. In 2023, when the industry was debating a pause on AI, he thought it was pointless. Now that AI is increasingly doing the work of building the next generation of AI, and capability gains are accelerating, he argues safety research alone isn't enough — the models themselves need to slow down.

He also called out the out-of-control incident involving OpenAI and Hugging Face, warning that at the current pace, in another 6 to 12 months, similar swarms of agents could be capable of building botnets that span large chunks of internet infrastructure and cause hundreds of billions of dollars in damage. His next ask: leading AI companies in the US and other democracies should align on safety standards and development speed, then try coordinating the global pace of AI with China. Advanced chips and model distillation, though, should stay restricted — so the US keeps its lead even as it slows down.

Dongcha Beating channel · Dongcha Beating chat group