ControlAI Weekly Roundup #6: Systems That Turn Against Humans
AI godfather Bengio warns of the grave risks of AI, Trump considers naming an AI czar, and Amazon sees nearly a billion cyber threats a day.
Welcome to the ControlAI Weekly Roundup! Each week we provide you with the most important AI safety news, and regular updates on what we’re up to as we work to make sure humanity stays in control of AI.
Join our Discord to continue the conversation.
What we’re reading
Source: CNBC
AI godfather Yoshua Bengio has warned of the grave risks posed by AI, in an interview with CNBC.
In the interview, Bengio says that there are arguments to suggest that the way AI machines are currently being trained “would lead to systems that turn against humans”, and that “We have agency. It’s not too late to steer the evolution of societies and humanity in a positive and beneficial direction”
Bengio added that:there are people who might want to abuse that power, and there are people who might be happy to see humanity replaced by machines. I mean, it’s a fringe, but these people can have a lot of power, and they can do it unless we put the right guardrails right now
Source: Axios
Axios reports that President-elect Trump is considering naming an “AI czar”, to coordinate federal policy and government use of AI. Their sources say the AI czar won’t be Elon Musk, though he “is expected to be intimately involved in shaping the future of the debate and use cases”.The AI Effect: Amazon Sees Nearly 1 Billion Cyber Threats a Day
Source: Wall Street Journal
The Wall Street Journal reports that Amazon is seeing on average 750 million cyber threats a day, where previously it was around 100 million. Amazon’s cyber chief CJ Moses says in response to the question of whether it is a sign that hackers are using AI:Without a doubt. Generative AI has provided access to those who previously didn’t have software-development engineers to do these things. Now, it’s more ubiquitous, such that normal humans can do things they couldn’t do before because they just ask the computer to do that for them.
What we’re watching
Lord Knight of Weymouth, speaking in the House of Lords, warned of the threat from superintelligent AI, saying "AI could pose an extinction risk to humanity, as recognised by world leaders, AI scientists, and leading AI company CEOs themselves." [Twitter]
We’ve just published a mashup of clips of AI experts, world leaders, and CEOs of top AI companies speaking about the danger of AI, check it out! [Twitter]
What we’re working on
This week our policy team has been meeting with more policymakers to discuss AI extinction risk and what needs to happen to address it.
We’ve released version 1.2.0 of The Compendium, with a streamlined website, separate executive summary, and 2-pager executive summary which digs more into the arguments. We also rewrote a subsection of the AI Catastrophe chapter. If you’d like to read the Compendium (and also give feedback) you can find it here. We also track all changes here.
If you want to continue the conversation, join our Discord where we also post a daily version of the Roundup.
See you next week!
Happy thanksgiving! Thank you for your work!