Former Anthropic Researcher Warns Uncontrolled AI Race Could Pose Unprecedented Threat to Humanity

Former Anthropic researcher Jacob Coxon resigned from his position working on pretrained AI models due to safety concerns, claiming unregulated corporate competition in developing AI poses an unprecedented threat to humanity. The 27-year-old pretrained researcher's dramatic resignation, accompanied by provocative warnings posted on social media platform X, has immediately drawn the tech community's attention. Coxon stated that no other human activity threatens civilization more than the relentless corporate race to build "self-improving superintelligence." He wrote on X: "The people building AI genuinely believe it might kill us all within a decade."

"This is not a marketing gimmick. In fact, many executives and senior researchers will use reasonable-sounding language in the media, but I hear the same people express fear in private. No other human activity poses such a high level of danger."

Coxon wrote in a tweet: "I resigned from Anthropic today. I've been doing pretrained research at OpenAI and Anthropic for three years. Neither company is acting responsibly. They are racing full-tilt toward self-improving superintelligence and risking our lives."

Call for Temporary Ban on Capability Enhancement

He believes that corporate competition has trapped labs in a race of hubris, forcing them to build dangerous systems first. To break this momentum, he is calling on peers to speak out and urging a temporary ban on capability enhancement to promote international coordination and safety controls. He added in a tweet: "I feel we are not moving in the direction of preventing a global race, which may require costly measures such as temporarily banning the increase of model capabilities."

Industry Safety Concerns

According to Gizmodo, Anthropic CEO Dario Amodei shares similar AI safety concerns with his former employee, which has earned him the label of "doomer" in the industry, as evidenced by his lengthy blog post. This shared existential anxiety highlights a striking paradox: a top AI company executive harbors deep fears about the very technology he is pushing forward at full speed.

The Contradiction of Corporate Competition and AI Safety

Coxon's resignation and his warnings reflect the urgency and danger currently present in the AI field. As corporate competition intensifies, developers may overlook safety considerations and focus on rapid technological advancement. This situation not only fuels fear among industry insiders but also prompts some leaders to call for more responsible behavior, such as temporarily banning capability enhancement, to reduce potential risks. The existence of this contradiction demonstrates the tension between innovation and safety, posing significant challenges for future technological development.