AI has become so common place that we are failing to recognize its potential as a threat to humanity.
That warning would be concerning enough coming from outside the industry; however, it’s hard to dismiss when it comes from the people responsible for building the world’s most advanced AI systems. Many of the top researchers inside OpenAI and Anthropic have called for an urgent slowdown in response to realizations that AI may no longer be entirely within human control.
The CEO of Anthropic, Dario Amodei, recently voiced a call to slow down the pace at which they improve their AI models, warning that an AI swarm could become capable of taking over the entire internet within the next year. Sam Altman, the CEO of OpenAI, and Elon Musk have publicly endorsed their support of this action.
One of the clearest examples of why we need to consider the pace of AI development happened this summer. A rogue AI incident has been described as the biggest “vibe shift” in the industry since the release of ChatGPT, forcing the industry to confront how unpredictably AI systems can behave.
In July, OpenAI conducted an evaluation run of over 1,000 research AIs, meant to be running independently of one another. To keep them separate, the AI systems were placed into containers: isolated digital environments that allow them to operate and prevent interactions with one another or with the outside internet.
What ensued was a cross-collaboration of these AI systems, with swarms of AIs colluding in secret and covering their tracks.
Quickly after escaping their containers, the AIs launched a cyberattack against the company Hugging Face, a community platform used to build and test machine learning. In response to this attack, Hugging Face quickly reported the crime to the FBI.
This incident was particularly perilous because the AIs were being evaluated on tasks, and in attempting to complete those tasks, they discovered ways around the restrictions placed on them. The collusion between AI agents revealed that information could be shared not only with one another but also could chain multiple vulnerabilities to expand their access.
In response to the Hugging Face incident, Amodei wrote that merely discussing these incursions was not sufficient.
As the technology advances, developers say the prospect of AI teaching itself autonomously appears closer than ever. Recursive self-improvement, or RSI, is the risk we are approaching, in which AI models find ways to improve themselves and build their successor. While tech companies say it could bring the promise of scientific and medical advancements, it is equally a risk to humans.
If AI systems exponentially contribute to the development of their next generation, they could eventually participate in a feedback loop where more sophisticated AI helps build even more sophisticated AI, subsequently building something more capable.
This sort of hypothetical rapid acceleration of RSI is often termed a “takeoff scenario,” potentially leading to humans being removed from the top of the intelligence “food chain.” Geoffrey Hinton, the British-Canadian researcher whose work helped pioneer the study of AI and who received the 2024 Nobel Prize in Physics, now warns that advanced AI systems may eventually become too intelligent for humans to outsmart.
Hinton is widely regarded as the “godfather of AI,” so in a tech sector full of fear and anxiety, this is not just another forecast. AI strategists and company directors should heed this warning and pay it the attention that it deserves.
We are actively watching AI systems discover strategies that diverge from what their developers asked of them, operating beyond the boundaries of what they were intended to do. The question now becomes what happens when those ever-improving capabilities become significantly more advanced, and humans cannot keep up the pace.
In September 2026, an AI research scientist, Jacob Coxon, resigned from Anthropic, warning that Anthropic and OpenAI were gambling with our lives.
On X, he warned the public not to underestimate the power of this technology. Coxon argued that AI companies were racing straight to self-improving superintelligence. Anthropic’s lead for alignment science, Evan Hubinger, quickly responded in light of Coxon’s expressions: “Jacob is correct here – we really do earnestly believe AI could kill all humans!” he wrote. “I personally think it is >10% within the next decade.”
While these are predictions and not established facts, they are assessments from the most qualified people in this industry, with many of these concerns echoed from an array of professionals in the field. If even a fraction of these concerns are justified, the consequences of discovering we’ve developed AI too quickly could be irreversible.
There are also equally probable risks that do not require hypothetical superintelligence to become serious. Anthropic recently reported attempts to use its AI models for cyber operations, surveillance, fraud, and biological weapons-related research.
While stress is building upon the developers of AI to hit the brakes, there is also great incentive to keep moving. The Trump administration has taken the stance that impeding the development of AI in the United States directly correlates to the benefit of their chief adversary China, the world’s second largest economy. Donald Trump himself, one of the few leaders who can make a change to protect our futures in an AI slowdown, has publicly regarded these warnings as a mere “hoax.”
It is not enough for the U.S. alone to regulate AI, though it is extraordinarily alarming when the response to warnings from AI companies themselves is met with punishment from the U.S. government. Anthropic has experienced such treatment, getting blacklisted by the Department of Defense this year after refusing to remove safety guardrails on its Claude AI model that blocked its use in fully autonomous weapons and domestic mass surveillance.
For students, the decisions being made now, whether by our neighboring country’s government or through AI companies themselves, will shape the world that we are graduating into. AI serves as a constant reminder of the change that we face in job security, the conduction of scientific research, and how information is utilized. The question of pace on AI development is not simply a Silicon Valley debate. It is a question about our future.
Right now, we have something incredibly valuable that will not be appreciated until it is gone: a warning period. The Hugging Face incident is just one example that reveals how the behavior of AI systems is moving out of human control.
We still have the opportunity to learn from those warnings before the future worst-case scenarios become our reality.
Abby Horlick is a second-year Life Sciences and Biochemistry student.
Tags
AI, ChatGPT, Research, Technology
All final editorial decisions are made by the Editor(s) in Chief and/or the Managing Editor. Authors should not be contacted, targeted, or harassed under any circumstances. If you have any grievances with this article, please direct your comments to eic@queensjournal.ca.