ControlAI Weekly Roundup #2
OpenAI launches ChatGPT search, Anthropic publishes a policy proposal for the next 18 months, and Connor Leahy and coauthors release The Compendium — a guide to AGI extinction risks.
Welcome to the ControlAI Weekly Roundup! Each week we provide you with the most important AI safety news, and regular updates on what we’re up to as we work to make sure humanity stays in control of AI.
Join our Discord to continue the conversation.
What we’re reading
OpenAI launches ChatGPT search, competing with Google and Microsoft
Source: CNBC
OpenAI has launched ChatGPT search, positioning the company to compete with search engines: “The feature offers up-to-the-minute sports scores, stock quotes, news, weather and more, powered by real-time web search and partnerships with news and data providers.”
The case for targeted regulation
Source: Anthropic
“Governments should urgently take action on AI policy in the next eighteen months. The window for proactive risk prevention is closing fast.”
In Anthropic’s proposal they say they want companies to be required to provide “Responsible Scaling Policies”, a set of risk evaluations, and a mechanism to determine the accuracy of RSPs and risk evaluations.
They call for incentives on better safety and security practices, and a focus on AI catastrophic risk prevention. They strongly oppose a use-case based approach, since models are highly general.Source: thecompendium.ai
The Compendium aims to present a coherent worldview explaining the race to AGI and extinction risks and what to do about them, in a way that is accessible to non-technical readers who have no prior knowledge about AI.
It covers the ideology driving the race to AGI, why AGI could quickly scale to superintelligence, and why safety is challenging and we are currently not on track. In particular, it cautions that much of the AI safety community is captured by the very actors racing to AGI, leading to ineffective solutions.Inside OpenAI’s multibillion-dollar gambit to become a for-profit company
Source: Vox
Kelsey Piper writes in Vox about OpenAI’s moves to become a for-profit company. Piper writes that you can’t actually just convert a charity into a for-profit, and that nonprofit lawyers told her:what’s almost certainly going on is a complicated and fraught negotiation: the sale of all of the OpenAI nonprofit’s valuable assets to the new for-profit entity, in exchange for the nonprofit continuing to exist and becoming a major investor in the new for-profit entity.
What we’re watching
Andrea Miotti explains A Narrow Path, our plan for humanity to avoid the extinction threat of AI and flourish, on the Future of Life Institute podcast:
Watch the clip in full (Twitter)
What we’re working on
This week we hosted a roundtable with officials and experts to discuss AI policy. We also hosted a dinner at the House of Lords with parliamentarians and renowned organizations to discuss AI risks and UK policy.
If you want to get in touch with us you can do so at hello@controlai.com or join our community on Discord.
See you next week!