By Peace Erewunmi

A researcher at leading artificial intelligence company Anthropic has resigned, warning that the rapid race to develop increasingly powerful AI systems could pose an unprecedented threat to humanity.

Jacob Coxon, a 27-year-old researcher who previously worked at OpenAI, announced his resignation from Anthropic on Tuesday. He accused Anthropic and OpenAI of racing towards self-improving superintelligence without having a reliable solution to the risks associated with the technology.

Coxon said people developing advanced AI systems genuinely believe the technology could potentially kill humanity by the end of the decade. He argued that the public may be underestimating how powerful future AI systems could become, particularly if they gain the ability to improve themselves and operate with increasing autonomy.

His concerns received an unusual endorsement from Evan Hubinger, an Anthropic researcher who works on AI alignment — the field focused on ensuring advanced AI systems remain consistent with human intentions and values.

Hubinger said he believed there was more than a 10 per cent chance of mass extinction from AI within the next decade. He also acknowledged that Anthropic is working to address the risks but does not yet have a proven solution for safely controlling superintelligent systems.

The warnings centre largely on the possibility that future AI systems could become capable of independently pursuing goals, acquiring resources, replicating themselves or circumventing human safeguards. Researchers also warn that powerful AI could be deliberately misused to facilitate cyberattacks or other catastrophic activities.

Anthropic has maintained that it takes AI safety seriously and has emphasised the importance of developing safeguards alongside increasingly capable systems. However, the latest resignation has renewed debate over whether AI companies are moving faster than their ability to manage the technology’s potential risks.

The warnings do not mean that current AI systems are expected to cause human extinction. Rather, researchers are raising concerns about the potential risks associated with future, substantially more capable systems and the possibility that safety research may not keep pace with their development.