Anthropic researchers warn AI could cause human extinction
Archived — this story has rotated out of today’s deck. It is kept here in full.
The gist
Anthropic researcher Jacob Coxon resigned Tuesday, warning AI could kill all humans by 2030.
Colleagues agree privately; the safety-focused firm is preparing a $1 trillion IPO.
Background
Anthropic has built its reputation as the safety-minded AI company, but several of its own current and former researchers now say the industry's race toward self-improving superintelligence is dangerously reckless. The warnings came just days after top OpenAI leaders, including CEO Sam Altman, said AI is speeding into a scary, uncontrollable phase. The posts also land as Anthropic gears up for an initial public offering that analysts say could value the company at more than $1 trillion.
How it unfolded
- Jun 2026Anthropic notes in a blog post that full recursive self-improvement might increase the risks of humans losing control over AI systems.
- Jul 2026More than 1,300 employees at frontier AI companies sign an open letter urging the U.S. government to support an international effort to pace automated AI development.
- Sep 8, 2026Jacob Coxon announces his resignation from Anthropic, saying he spent the past three years doing pre-training research at Anthropic and OpenAI.
- Sep 9, 2026Coxon's warning draws responses from Anthropic colleagues including Evan Hubinger and Samuel Marks, and coverage from Axios, The Hill, CNBC, NBC News and others.
Who’s saying what
- Party
- Jacob Coxon says neither Anthropic nor OpenAI is acting responsibly, that they are racing straight to self-improving superintelligence and gambling with our lives, and that no other human activity poses this level of danger.
- Party
- Anthropic alignment-science lead Evan Hubinger says Coxon is correct, that he personally thinks it is more than 10% within the next decade, and that Anthropic is trying its best but has no plan yet to solve alignment for superintelligence.
- Party
- Anthropic scalable-oversight lead Samuel Marks says developers continue due to a mixture of commercial incentives and a belief they are in a race with less responsible developers, and that the more senior the employee, the more concerned they are.
- Public
- The warnings erupted across the tech world and reignited fears about the AI race, according to Washington Post and PBS coverage.
Still unverified
Coxon's claim that executives and senior researchers express fear privately while using measured language in the press is his own account; OpenAI and Anthropic did not respond to requests for comment from Business Insider.