America's Security Depends on Stopping Superintelligence
The US government needs to treat superintelligent AI as the security risk that it is. America must take action now to ensure that it cannot be developed domestically, or anywhere else in the world.
It’s no secret that AI’s capabilities are growing exponentially and increasingly posing national security threats. Just a few months ago, leading AI company Anthropic withheld the public release of its Mythos model after testing showed it had nation-state level autonomous cyberattack capabilities. Recently, General Joshua Rudd, director of the National Security Agency and the Pentagon’s Cyber Command, confirmed that Mythos was able to break into almost all of his organization’s classified systems in a matter of hours.
Anthropic later deployed a dialed back version of Mythos called Fable 5, but within 72 hours, the US government banned foreign nationals from accessing the models, due to national security risks and potential jailbreaks. In order to comply, Anthropic pulled back access to the models for all users globally.
These AI models’ unprecedented cyberweapon capabilities should be a wake-up call for governments around the world, but the real danger here is not just about autonomous AI hacking tools. It’s about what’s next.
All of the leading AI companies are explicitly racing to build superintelligent AI, AI that is vastly smarter than humans and capable of overpowering and outmaneuvering our national security apparatuses. Top AI researchers, Nobel Prize winning scientists, and even the CEOs of the leading AI companies themselves have repeatedly warned that superintelligent AI would pose an extinction risk to humanity.
Such AI does not currently exist, but experts across industry and academia widely expect that superintelligence will be developed in the next two to five years. At a G7 meeting in mid-June, Anthropic’s CEO assessed that AI models will be better than humans across the board within one to two years, an assessment backed up by OpenAI’s CEO saying at the same meeting that he expects AI “systems with astonishing power” within a year or two. The AI firms at the head of the race are closing in on developing AIs that can autonomously improve themselves, which many in the industry acknowledge as being the point where humanity loses control over AI.
In response, the US national security establishment needs to get pragmatic, and quickly. The US needs to recognize superintelligence as the national and global security risk that it is, and impose the only policy that will actually mitigate the threat: prohibit the development of superintelligent AI domestically and prevent its development anywhere in the world, using the full ladder of deterrence.
This is not about banning all AI development. Many of today’s models have valuable applications, including for defense purposes, but superintelligent AI is in a category of its own. Because of its inherent uncontrollability, it would neither be a tool we could leverage, nor a strategic weapon, like nukes.
Instead, the moment superintelligence is created, we would be simultaneously developing the world’s deadliest weapon and handing over control to the weapon itself. With its ability to outmaneuver and outsmart human organizations, superintelligence would upend governments’ defense apparatuses, and threaten humanity with extinction.
If we have a world containing millions, or even billions, of superintelligent AIs more capable than humanity, competing with each other and pursuing goals we did not set and cannot alter, it is hard to imagine things going well for us. Anthropic CEO Dario Amodei estimates a 25% probability that things will go “catastrophically wrong” with AI, and admits that we only understand maybe 3% of how AI models work. Elon Musk has said that there’s a 10 to 20% chance that AI annihilates humanity, and OpenAI’s Altman reconfirmed in 2025 his long-held view that superintelligent AI is “the greatest threat to the continued existence of humanity.”
The good news is a prohibition on superintelligent AI can be both monitored and enforced. AI capabilities do not emerge out of thin air. Not unlike the large, traceable process required for building nuclear weapons, developing advanced AI systems requires conspicuous physical infrastructure and significant industrial capacity.
The data centers running powerful AIs around the clock can use as much power as a medium-sized city, and satellite imagery makes it easy to track these centers being built. Additionally, the supply chain for developing advanced AI is narrow and mainly controlled by Western countries and Taiwan. Take the cutting-edge chips that power advanced AI models, for example. They are nearly all designed by a single American company, NVIDIA, and produced by the Taiwanese firm TSMC. Similarly, the lithography machines responsible for building the chips are exclusively produced by ASML in the Netherlands.
Prohibiting the development of superintelligence should be the number one security concern across governments. In May, President Trump and President Xi Jinping met in Beijing and discussed working together to place guardrails on AI models, but they are still far away from any kind of agreement on AI risks, let alone on prohibiting superintelligence.
In a conventional arms race, guarding your state’s national security interests involves maintaining a perpetual edge over your adversary. But because superintelligence would be beyond the control of humans and our defense organizations, no country ‘wins’ a race to superintelligence, and no edge will be provided by it.
Protecting America thus requires prohibiting superintelligence’s development before it is ever created. If superintelligence is developed, there will be no opportunity for the US to wrest back control, as it did in response to Fable 5’s release.
The US should act with urgency to recognize superintelligence as the national security threat that it is, understanding that the technology would not confer a strategic advantage over adversaries. It should prohibit the development of superintelligence within the US, including immediate precursors like autonomously self-improving AIs, and immediately set a policy to prevent and deter its creation anywhere in the world, whether by adversaries or non-state actors. This should ultimately be followed by Washington working with other states to establish an international ‘trust but verify’ regime that imposes a global prohibition on superintelligence.
America must take action now to ensure that superintelligent AI cannot be developed domestically, or anywhere else in the world. We still have time to prevent superintelligence, but the clock is ticking.
Please see my post on The Fiduciary Standard for Ai.
If AI is unchecked, it could take over the worry and would be merciless. Is it Hal in 2001 Space Odyssey that predicted this?