The AI industry just had its loudest internal argument in years — and it played out entirely in public.
Over the past 72 hours, a chain of events turned AI safety from a background policy debate into the single biggest story in tech. A researcher publicly resigned from Anthropic over existential risk. Anthropic's own alignment lead escalated things with a viral post. And then CEO Dario Amodei responded with a formal plan to slow down frontier AI development.
Here's a full breakdown of what happened, who said what, and what it means for the developers and companies actually building on these models.
The Timeline: How the Firestorm Unfolded
1. Jacob Coxon resigns from Anthropic
The current discussion began when AI researcher Jacob Coxon — who has also worked at OpenAI — announced his resignation from Anthropic. His stated reason: he believes the leading AI companies are "gambling with our lives" by racing to develop increasingly capable frontier models.
What made Coxon's resignation notable, as TechCrunch's Equity podcast pointed out, is that he's in a rare camp. When CEOs like Sam Altman or Dario Amodei talk about existential risk, the obvious follow-up is: "If you really believe that, why are you still building it?" Coxon answered that question with his career — he stopped.
2. Anthropic's alignment lead escalates — with an unfortunate exclamation mark
Coxon's resignation post was immediately amplified on X by Anthropic's alignment science lead, who added: "We really do earnestly believe AI could kill all humans!" — putting his personal estimate of that outcome at "more than 10% within the next decade."
That exclamation mark may go down as one of the most-memed punctuation choices in tech history. The post immediately went viral, and the "P(doom)" concept — the probability of catastrophic outcomes from AI — jumped from obscure rationalist forums into mainstream coverage.
3. The skeptics push back
Not everyone bought it. Commentators raised three main counterpoints:
- The "we" problem — Who exactly is "we"? The AI research community is far from a monolith, and a ">10%" figure that isn't derived from any model or calculation is, as one critic put it, "a made-up number."
- The flexing theory — Is every warning about AI agents "breaking through" safeguards secretly a way to advertise how capable your models are? If your AI weren't so advanced, there would be nothing to warn about. Warnings and marketing can look uncomfortably similar.
- The IPO question — With Anthropic reportedly preparing for a public listing, some observers wondered aloud whether its S-1 filing would need to state that the company believes there is a greater than 10% chance it develops something that could "eradicate all of humanity" — and that this "would be materially bad for our business."
4. Dario Amodei responds with a pacing plan
Rather than walking anything back, Anthropic CEO Dario Amodei escalated in the other direction: he published a formal proposal for pacing frontier AI development — a framework for coordinating how fast the leading labs ship new frontier capability. (We covered the full plan in yesterday's article: Anthropic CEO outlines plan to slow AI development.)
The message from Anthropic is now internally consistent: senior leadership genuinely believes the risk is real, and they're proposing structural changes to address it.
Why This Flared Up Right Now
Three forces converged this week:
- Rapid capability jumps. OpenAI's GPT-6 Astra shipped a few weeks ago, and Anthropic's latest Claude frontier models keep setting new benchmarks. Stronger models make both safety warnings and the "flexing" critique more credible.
- Real security incidents. A recent Hugging Face compromise tied to OpenAI's internal models and ongoing supply-chain attacks on AI developer tooling gave the doomers concrete examples to point at.
- Public-market pressure. Both OpenAI and Anthropic are rumored to be eyeing IPOs. Sam Altman said over the weekend it would be "ill-advised" for OpenAI to go public in 2026. Safety positions and public-company disclosure requirements are on a collision course.
What It Means for Developers (Practically, Not Philosophically)
If you build on AI APIs, here's the realistic impact:
- Expect safety-tier model variants. Labs are already shipping models with extra safeguards (Anthropic's enterprise frontier safeguards, Claude's biology-related restrictions). Your use case may get routed or gated differently.
- Expect more text watermarks and provenance features. Anthropic's new Claude text watermark is a sign of where attribution tooling is heading.
- Pricing pressure continues regardless. Even as the safety debate rages, the economic trend is unchanged: open-weights and wholesale compute marketplaces keep pushing inference prices down. Frontier drama doesn't change the fact that a flagship-quality model on Qubax can cost under $1 per million output tokens.
The Honest Take
Both camps have a point. The skeptics are right that ">10%" numbers thrown out without methodology are rhetoric, not analysis, and that safety warnings conveniently double as capability marketing. But Coxon's resignation is genuinely different from CEO doom-talking — he acted on his stated beliefs at real personal cost.
The most likely outcome: this debate keeps raging while the actual frontier keeps moving, and the binding constraint on AI progress turns out to be neither doom nor hype, but economics — who can afford to train, who can afford to run, and how fast marketplaces can compress prices.
The safety debate is worth having. Just don't expect it to slow your API bill down.
Want to build with frontier models without frontier prices? Compare 200+ models side by side with real per-token pricing at qubax.ai/models, or read the Qubax docs to get started in minutes.
FAQ
What is P(doom)?
P(doom) is shorthand for an individual's estimated probability that advanced AI causes a catastrophic or existential outcome. It's a subjective probability, not a scientifically derived figure — which is exactly why the ">10%" claim sparked so much pushback.
Did Anthropic actually confirm Coxon's resignation?
Coxon publicly announced his resignation and Anthropic's alignment lead acknowledged and amplified it. Anthropic has not issued a detailed formal statement beyond leadership's broader safety positions.
Does this mean AI models will get more expensive?
No — the opposite, historically. The safety debate happens at the frontier, while inference prices keep falling due to competition, distillation, and open-weight models. Wholesale marketplaces like Qubax routinely offer flagship models at 70–90% below official retail API pricing.
Where can I read Dario Amodei's pacing proposal?
It's published on Anthropic's newsroom, alongside their related posts on frontier safeguards and open-weights positions.
