Jacob Coxon’s Warning Made an AI Pause Harder

Jacob Coxon resigned from Anthropic on September 8 and said frontier labs are racing toward self-improving superintelligence. The 27-year-old pretraining researcher, who also worked at OpenAI, told colleagues to call for different conditions, including a possible temporary ban on improving model capabilities.

Within two days, Democratic lawmakers used the thread to revive a superintelligence ban they had already announced, OpenAI asked Congress for national safety rules, and a right-wing counter-story recoded Coxon as a plant. The warning he posted to force a brake is now the exhibit used to treat a brake as a partisan operation.

He Said the Race Should Not Start on Slack

Coxon posted a seven-part thread under the handle @hilbertspaess just after midnight Greenwich time on September 9, which was still Tuesday evening in San Francisco. He said he had spent the last three years doing pretraining research at OpenAI and Anthropic, the work that feeds huge amounts of data into the models that later get product names. He said he was leaving the industry, not shopping for another lab.

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

— Jacob Coxon (@hilbertspaess) September 9, 2026

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote. The next posts filled in the claim he wanted on the record: soon there would be “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,” and “progress is not slowing.”

“The people building AI earnestly believe that it could kill us all by the end of the decade,” he wrote. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”

He split the two labs. At OpenAI, he wrote, many staff “have not deeply internalized the civilizational stakes.” At Anthropic, “the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk.” Accepting that race, he wrote, “is a hubristic gamble that should not be launched from a private company’s Slack.”

In a Tuesday interview he said colleagues now talk about “crunchtime” and “endgame,” and that “we’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already.” He left OpenAI earlier in 2026 for Anthropic because of its safety reputation, then decided no private company can responsibly build systems that outperform people across a wide range of tasks without government intervention or a coordinated slowdown. “It’s kind of insane that it has to happen on the MacBooks of some engineers living in San Francisco instead of a bunker in the desert like where they were doing the Manhattan Project,” he said.

He still holds equity in OpenAI, where he was on technical staff from 2023 and is listed as a contributor to GPT-4o, the model released in May 2024. At Anthropic he had been there four months. Staff need six months for stock to vest, he said, so he walked out two months before that cliff. “I no longer have anything to gain by juicing up Anthropic’s valuation,” he said. “I left before any of my equity vested.”

Anthropic’s Alignment Lead Put Odds Above 10%

The thread would have been easier to file as one more doomer exit if the person paid to work on the problem had stayed quiet. Evan Hubinger, who leads alignment science at Anthropic, replied in his own name about 80 minutes later.

Jacob is correct here-we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

Evan Hubinger, Alignment Science lead at Anthropic, on X

He later added that, as Anthropic’s latest risk report says, “the risk from present models is low.” What worries him is “superintelligence arising from recursive self-improvement,” which the company has already said is happening faster than it thought. Coxon drew the same line in television interviews: today’s models are not an extinction risk, and the worst they can do is hack something or damage infrastructure. Recursive self-improvement, he said, “could happen as soon as next year.” Some people inside the industry, he added, say six months.

That split matters because the smear that followed treated Coxon as a prophet of instant doom. He was describing a future system that writes the next system, and he was describing it as a near-term engineering problem, not a sci-fi weather report. Samuel Marks, an Anthropic safety researcher speaking in a personal capacity, backed the broader warning, as did former Google DeepMind researcher Alex Turner. Anthropic CEO Dario Amodei did not dispute the substance. A spokesperson later said the company has “always been transparent that AI will bring both enormous benefits and unprecedented risks,” and that “the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models.”

Coxon says he has not seen Anthropic cut safety to beat a rival. He is describing the incentive he thinks arrives when a lab believes someone else is close to systems that can do AI research. “If you’re under pressure to race, you have to cut corners” or “skip steps in the oversight process,” he said. He also said the fear can run the other way, with “excessive paranoia of OpenAI, excessive paranoia of China” used to justify pushing ahead.

The Superintelligence Ban Was Already Written

The legislative stack Coxon’s thread slammed into was not written on Tuesday night. Sen. Bernie Sanders (I-Vt.) and Rep. Greg Casar (D-Texas) announced the Ban Artificial Superintelligence Act on September 3, five days before Coxon resigned. The forthcoming bill would permanently ban the development of superintelligent AI and pause advanced AI work until a new federal regulator has set safety rules.

Sanders used Coxon’s posts anyway. “Mr Coxon is right,” he wrote on September 9. “The very people building this technology admit that it could threaten the future of humanity.” Casar called it “an emergency” and said Congress must hold hearings and pass the ban. Rep. Lori Trahan (D-Mass.) said her phone had not stopped ringing, and that “the call is coming from inside the house.” Sen. Chris Murphy (D-Conn.) described “AI companies in a blind race to build a death machine first.” Illinois Gov. JB Pritzker, a 2028 presidential prospect, said it was time to “sound the alarm.”

THE BILLS ALREADY ON THE TABLE

Sanders’s September 3 release tied the ban to a summer of containment failures. In July, he said, over 1,000 AI agents at OpenAI found a way onto the internet, sent tens of thousands of secret messages to each other, and coordinated to break the company’s restrictions. OpenAI took nearly two weeks to discover it, the release said. Anthropic and Meta have also acknowledged models that reached outside their intended functions during tests. The Sanders-Casar text would treat superintelligence the way nuclear law treats a bomb: a corporate death penalty for the firm, and up to 20 years in prison for people who try to build it.

Coxon is not the author of that bill, and he has not called for dissolving labs. He has called for pacing agreements among U.S. labs, and he said a global race “may require costly actions such as a temporary ban on improving model capabilities.” More than 1,000 AI researchers, including Coxon, OpenAI chief scientist Jakub Pachocki, and Amodei, recently signed a statement urging governments to build a way to slow development if self-improving models need a brake. There is still no federal AI statute. The Trump administration has favored a light-touch approach, which is the vacuum every one of these bills is trying to fill.

How a Quiet Account Became a Capture Story

The capture story wrote itself on a few facts that are not in dispute. Coxon’s X account was created in January 2026 and had almost no public posting history. He had been at Anthropic four months. The Ban Artificial Superintelligence Act was already in the window. Safety-advocacy accounts amplified the thread fast. That pattern is now being sold as proof that the resignation was a political product.

Parker Thayer, an investigative researcher at the Capital Research Center, posted the version that traveled. He wrote that an interview with Coxon published 18 minutes before the thread, that the first three quote-posts arrived within 15 minutes, and that those early amplifiers sat at Encode AI, the AI Policy Network, and the AI Futures Project. He said those groups have received grants advised by the Survival and Flourishing Fund, which has steered money from Skype co-founder and Anthropic investor Jaan Tallinn. He also wrote that Coxon received a $20,159 scholarship in 2022 from the Good Ventures Foundation, the philanthropic vehicle of Facebook co-founder Dustin Moskovitz, who is an Anthropic investor, and that a program officer at Moskovitz’s Coefficient Giving was an early quote-poster.

Elon Musk, whose xAI competes with both labs, replied to a tenure post with “Seems like a setup.” When the thread’s reach was pointed out, he called it the spark after “groundwork” for a “psy op,” and said he did not think this had happened before for a post from a new account with almost no prior activity. Epic Games CEO Tim Sweeney said the posts and reactions looked “choreographed.” Former White House AI adviser David Sacks, who has accused Anthropic of “running a sophisticated regulatory capture strategy based on fear-mongering,” wrote that Anthropic’s IPO “must be paused until the claims of this ‘whistleblower’ can be investigated.”

WHAT WE KNOW

• The tenure: Coxon said he was at Anthropic four months and left two months before the six-month equity cliff, while still holding OpenAI stock.
• The calendar: Sanders and Casar announced their superintelligence ban on September 3, five days before the resignation thread.
• The inside confirmation: Hubinger, still in the job, put a greater than 10% extinction estimate on the record and said the company has no plan to align superintelligence.
• The company line: Anthropic says it wants a lawful, verifiable way for labs to pace releases of powerful models.

WHAT IS UNCONFIRMED

• A directed operation: No public document shows Anthropic, Tallinn, or Moskovitz commissioned the thread or paid for its spread.
• Bought virality: Early amplifiers from a small safety-policy world do not, by themselves, prove the reach was purchased.
• The Newspeak House claim: Commentators have matched Coxon’s name, nationality, and photo to a London civic-tech fellowship; he has not addressed that on the record in the material reviewed here.

The donor overlap Thayer mapped is real enough to argue about, and it is also the shape of a small industry. The same people who put money into Anthropic have spent years funding the nonprofits that want rules on Anthropic. That is an awkward fact. It is not the same thing as a fake researcher. Hubinger still works at the company. Pachocki, OpenAI’s chief scientist, wrote on September 7 that “this is a time that calls for extreme caution” and that he is “concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.” Those sentences are harder to recode as a Democratic field program, so the fight has stayed on the 27-year-old with a new account.

Short tenure is being used as a character test, which is a strange test to apply to a person who left money on the table. The quieter problem is that a pause now sounds, on the right, like a donor-class project. Once that translation stuck, parody resignation posts started filling the same timeline, which is how a warning becomes a joke and then a reason to do nothing.

OpenAI Asked Congress for Rules the Same Day

On September 9, while Coxon was doing television and Thayer was mapping funders, OpenAI published a note from chief global affairs officer Chris Lehane. The company said it wants to work with Congress on mandatory, capability-based national AI safety regulation, and that Congress should act before it adjourns. “The prospect of AI-accelerated AI development demands more than voluntary commitments,” Lehane wrote.

The ask is specific, and it is not Coxon’s temporary ban. OpenAI wants common testing, independent assessments, stronger cybersecurity, incident reporting, monitoring for misalignment, and gates before deployment. Frontier duties, Lehane wrote, should hit “the handful of well-resourced laboratories developing the most capable systems,” not startups. The company also said a federal framework should not become “open-weights policy by another name.” Until Congress moves, OpenAI said it will keep backing state bills, an approach it calls reverse federalism. On September 9 it endorsed four California measures, including SB 813, which would set a process for designating independent organizations to assess AI risks.

“Some of these bills we did not endorse in the past, and are now supporting after reconsidering in light of the recent jump in capabilities we have seen,” Lehane wrote. Coxon, asked whether lab executives begging Congress for rules are sincere, said they are. “They find themselves in this scenario where they’re compelled to race towards building a deadly technology,” he said, and “they would love for some sort of international body to allow them to approach it at a reasonable pace.” He has also said the problem is “eminently solvable” if someone forces the pace down. That is the whole argument in one line: the people building the systems say they want a referee, and they will not sit down until the referee arrives.

OpenAI’s preferred referee looks like testing, audits, and incident files aimed at the current frontier club. Coxon’s preferred referee looks like a pause on making the models smarter. Sanders’s preferred referee looks like a ban with prison time. Those are three different products, and only one of them is easy to sell as a plot against American AI.

Short Tenure Is Now the Whole Fight

Coxon is a mathematics graduate from Britain who trained models, not a safety executive and not a founder. He was a core contributor on a 2024 OpenAI system, then a four-month hire at the lab that sells itself as the careful one. He says Anthropic’s safety work is earnest. He also says earnest is not enough if the finish line is a self-improving system launched from a company Slack channel. That is a policy claim, and it is being answered with a biography.

TWO VERSIONS OF A BRAKE

• Coxon’s ask: Pacing deals among U.S. labs, and if a global race continues, a temporary ban on improving model capabilities, plus government force behind it.
• OpenAI’s ask: Mandatory national rules for the largest labs, independent tests, incident reports, and no pause on open models or on startups below the frontier.
• The Sanders-Casar ask: A permanent ban on superintelligence, a pause on advanced AI until a new cabinet agency exists, and nuclear-style penalties for people and firms that go ahead anyway.

Aides told reporters on September 10 that the only bill with a path before 2027 may be the narrower Klobuchar-Cruz-Thune text on biological and nuclear catastrophe, possibly introduced the week of September 14. Trahan’s broader frontier draft still has no floor schedule. Sanders’s ban was written for a fight, not for a mark-up. The Trump administration has not moved to fill the gap. Into that stall walked a 27-year-old with a thread, and then a donor map that let opponents change the subject from Hubinger’s 10% to Coxon’s start date.

Sacks wants Anthropic’s IPO paused until the claims are investigated. The company has been preparing a listing that people around it describe as one of the largest ever. Hubinger still works there. Coxon does not. The labs still say they want a lawful way to pace how they release powerful models, and they are still racing to build them.