Srinagar, Sep 10: Former Anthropic and OpenAI researcher Jacob Coxon’s warning that people building advanced artificial intelligence “earnestly believe that it could kill us all by the end of the decade” has been joined by six other current and former researchers from leading AI laboratories, with seven people in all voicing serious concerns about the risks of AI over the past four days.
Coxon, who announced his resignation from Anthropic two days ago after three years of pretraining research at OpenAI and Anthropic, warned that the companies developing increasingly powerful AI systems were “racing straight to self-improving superintelligence and gambling with our lives”.
Since then, or around the same four-day period, researchers and former researchers from Anthropic, OpenAI and Google DeepMind have made their own stark assessments, ranging from the possibility of human extinction to fears that AI development is advancing at a pace that leaves humanity little room for error.
Evan Hubinger, Anthropic’s alignment science lead, directly endorsed Coxon’s warning.
“Jacob is correct here—we really do earnestly believe AI could kill all humans!” Hubinger wrote.
Samuel Marks, Anthropic’s scalable oversight lead, offered another stark assessment of the risks perceived by people working in AI.
“AI developers believe their technology could cause human extinction (or similarly bad outcomes),” Marks wrote. “This could happen in the next few years.”
Former Google DeepMind research scientist Alex Turner also backed Coxon’s assessment.
“I left Google DeepMind in June. Jacob is right: many researchers believe they are building something that could kill everyone on the planet,” Turner wrote.
“It was literally my day job to think about how to stop that.”
The warnings are not confined to former employees or researchers who have left the companies.
OpenAI chief scientist Jakub Pachocki called for “extreme caution” as machine intelligence continues to advance.
“This is a time that calls for extreme caution,” Pachocki wrote. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.”
OpenAI researcher Jason Wolfe similarly warned of a range of potentially disastrous outcomes, while stopping short of assigning a probability to extinction.
“I don’t know what my probabilities are on literal extinction, but I think there are a number of ways AI could go poorly for humanity,” Wolfe wrote.
“At the current frankly terrifying pace humanity will be quite lucky if we manage to find and stay on the narrow path between all the bad outcomes.”
Former Google DeepMind senior research scientist Jonathan Richard Schwarz raised concerns about another consequence of the AI race — the concentration of technological power.
“I left DeepMind after 7 years (and since rejected offers to join the other two) due to severe concerns about the concentration of power these labs represent,” Schwarz wrote.
“What is currently happening in this field is deeply unhealthy for society.”
The seven warnings, including Coxon’s, have emerged over a period of just four days from people with direct experience of some of the world’s leading frontier AI laboratories.
Their concerns are not identical. Coxon has focused on the possibility of self-improving superintelligence and an AI race that he believes is moving too quickly. Hubinger and Marks have explicitly raised the prospect of extinction, while Pachocki and Wolfe have warned about the consequences of rapid advances in machine intelligence. Schwarz has highlighted the concentration of power within the industry.
