A Narrow Path
A plan to ensure the survival and flourishing of humanity against the threat of artificial superintelligence.
A Narrow Path is now available here, and thread here.
As the development of artificial intelligence (AI) accelerates and new capabilities emerge at an unprecedented pace, we must navigate a narrow path to ensure these advancements are controlled and aligned with human flourishing. While most AI developments are beneficial, the rise of superintelligent AI (ASI) threatens humanity with extinction. "A Narrow Path" introduces new institutions, legislation, and policies that countries should implement to address the risks associated with developing ASI systems. It offers a pragmatic approach for policymakers, balancing the encouragement of AI innovation with the imperative to prevent catastrophic outcomes and protect humanity’s future.
Our proposal is divided into three phases, each designed to guide the safe development of AI at both national and international levels. The initial stage, Phase 0 - Safety, proposes a defense in depth approach combining compute-based thresholds with novel measures that tackle directly dangerous AI capabilities. It calls for immediate action by national governments to establish institutions and legislative frameworks that set computational and performance limits while also restricting high-risk capabilities such as autonomous self-improvement and environmental escape. By introducing a licensing regime to monitor and control these advanced AI systems, we ensure that no AI progresses towards general artificial intelligence without strict oversight and comprehensive safety controls. These measures should then be internationalized through a treaty.
The next stage, Phase 1 - Stability, introduces a comprehensive international framework for AI governance designed to prevent a competitive arms race and ensure cooperation. It is composed of three institutions: a central authority responsible for setting regulations, monitoring compliance, and managing global AI safety efforts; a research arm that conducts AI safety research and oversees high-risk AI development under the central authority’s guidelines; and an independent judicial body that resolves disputes and ensures adherence to the AI treaty, thereby fostering sustained international cooperation.
A final stage, Phase 2 - Flourishing, describes what we believe would be required to safely develop and deploy potentially transformative AI technology. It emphasizes the scientific foundations and understanding of intelligence needed to integrate AI into the social fabric as a key driver in healthcare and economic development while remaining within human control. This phase considers both the immense potential benefits of advanced AI as well as the practical challenges involved in ensuring it is safely developed, deployed, and used.
Without appropriate governance mechanisms in place at the domestic and international levels, the risks associated with AI could far outweigh its potential rewards. AI development is currently racing towards the creation of systems that could surpass human intelligence within the next 3–15 years. Given our current inability to predict to a meaningful degree of accuracy what capabilities frontier AI systems will possess when developed, and the potential hazards these may entail, we believe that preventing the emergence of superintelligent AI within the next two decades is an urgent priority. This delay would provide humanity with the necessary time to build a scientific and regulatory infrastructure capable of managing the safe deployment of transformative AI systems.
“A Narrow Path” lays out the essential principles and actionable steps needed to address the extinction risks from AI, enabling the creation of concrete policies and laws that ensure a safe and prosperous future for humanity.
For more detailed information on the proposal and its recommendations, the full document is available here.
You can also email us if you want to get in touch.