Grok Shows Governments Are Playing Catch-Up on AI
What the Grok deepfakes scandal reveals about government preparedness for AI, and what should be done to fix it.
Grok, which last year made the news for calling itself “MechaHitler”, has become the subject of another scandal in recent weeks. Developed by Elon Musk’s AI company, xAI, the chatbot has been found to comply with requests to digitally undress photos of women and girls without their consent — known as deepfaking. In one case, a 14-year-old actress was targeted.
If you find this article useful, we encourage you to share it with your friends. If you’re concerned about the threat posed by AI and want to do something about it, we also invite you to contact your lawmakers. We have tools that enable you to do this in as little as 17 seconds.
One analysis which collected 20,000 images generated by Grok between December 25th and January 1st found 53% showed people in “minimal attire”, with 2% showing people appearing to be 18 years of age or younger. 81% of the images depicted women.
While at ControlAI we’re focused on preventing the risk of extinction posed to humanity by superintelligent AIs, the problem of deepfakes is an issue we were early campaigners on. Governments haven’t kept up, and are now scrambling to respond to the Grok deepfake scandal.
In the UK, the online safety regulator (Ofcom) has opened a formal investigation, while the government moved to criminalize the creation of sexual deepfakes. In Canada, Japan, Australia, and the European Union, investigations into Grok and X were opened or expanded. Meanwhile, Brazilian authorities gave xAI 30 days to tackle the problem, while regulators in France and India have issued warnings. In Southeast Asia, Indonesia, Malaysia, and the Philippines temporarily blocked users from accessing Grok. California’s Attorney General sent xAI a cease-and-desist letter, demanding that Grok stop producing sexual deepfakes.
Deepfakes have been a known problem for years, and governments have had time to restrict them. What changed with Grok is that access to AIs that will create them has been handed to anyone that uses Twitter. This scandal shows that governments aren’t keeping up with developments, reacting to address this issue, rather than proactively developing regulations to mitigate problems from AI — before they become an issue of this scale.
If we want to avoid issues like this, and even larger-scale ones that come with ever more powerful AIs — including the risk of human extinction — governments should move to an approach of regulating development of the most powerful AIs to address risks before they become a problem.
🎥 VIDEO: Here’s a Sky News interview of ControlAI’s founder and CEO Andrea Miotti (and coauthor of this newsletter) discussing the Grok deepfakes scandal.
You can watch it here: https://news.sky.com/video/grok-the-creators-dont-know-how-it-works-or-how-to-keep-it-under-control-13495255
Why is Grok doing this in the first place?
xAI didn’t design Grok to act as a deepfake machine, probably. Grok is a multimodal AI system that can do many things, including acting as a chatbot, searching the web, and so on. One capability this system has is the ability to edit images. In order generate these images, Grok’s been trained on vast amounts of images and pieces of text. This will include images of women in bikinis and such, and so the AI learns to produce those too.
When a user provides Grok with a request to answer a question, or produce an image, Grok will evaluate whether it should comply. It seems plausible that in this case the policies xAI set for Grok on what image requests it should comply with are simply too loose, with that being a major contributor to the problem we see now.
The deep problem with modern AI systems
We should also point out that the nature of these modern AI systems is that they are not really controllable to begin with.
Modern AI systems are more grown, like biological organisms, than they are built. Instead of being written by programmers, AIs like Grok are developed by giving a simple learning algorithm access to vast amounts of data and running it on supercomputers for months. This produces a structure of billions of numbers, called the “model weights”, such that when you run this on a computer you have a functioning form of intelligence, an AI.
Unfortunately, while developers know how to implement the process for producing these numbers, they understand almost nothing about what they mean. Somewhere contained within these numbers are the interesting things about an AI that you’d want to control or verify. If we knew what they meant then we might be able to tell what capabilities, drives, goals, and behaviors an AI has without even running it. But we don’t know what they mean, so our ability to get a sense of these is limited to coming up with tests for the AIs and running them — a blunt tool. We also can’t set these things.
This means that controlling these AIs is limited to tools like fine-tuning. Essentially you show an AI how you’d like it to behave, and it learns to imitate that, with varying degrees of reliability. Another such tool is providing it instructions not to do certain things and hoping that it follows them. The ability to get an AI to do things it’s not supposed to is called jailbreaking, and the UK AI Security Institute’s recent frontier AI trends report found ways to universally jailbreak all AIs they tested.
So, even when AI companies try to stop their AIs engaging harmful behaviors, they can’t do so reliably.
This situation with modern AI systems, where we can’t reliably control them or understand them, isn’t just relevant in the case of harms caused by AIs today. Right now, AI companies like xAI, OpenAI, Anthropic, and Google DeepMind are racing each other to build artificial superintelligence — AI vastly smarter than humans.
They also don’t know how to ensure that superintelligent AI would be safe or controllable. And if we lost control of an AI that powerful, that could be catastrophic. Countless AI experts, and even the CEOs of these companies themselves, have warned that it could result in human extinction. We wrote an article about how it could happen here:
How Could Superintelligence Wipe Us Out?
There’s growing agreement among experts that the development of artificial superintelligence poses a significant risk of human extinction, perhaps best illustrated by the 2023 joint statement by AI CEOs, godfathers of the field, and hundreds more experts:
That’s what we’re focused on preventing at ControlAI. So far, we’ve got over 100 UK politicians to support our campaign for binding regulation on the most powerful AI systems, acknowledging the risk of extinction posed by superintelligent AI.
As part of our campaign, we’ve developed contact tools that enable people to get in touch with their representatives in the US, UK, and beyond, within seconds. So far, tens of thousands of people have done so, and many MPs who’ve joined our campaign did so after being contacted.
As a member of the public, this is one clear action you can take to help solve this problem and prevent the worst from happening. You can check out our tools here:
https://campaign.controlai.com/take-action
We also have a Discord you can join if you want to connect with others working on helping keep humanity in control.
This is spot on about reactive vs proactive regulation. The pattern here is basically the same we saw with social media in the 2010s - tech moves fast, regulaters scramble after the damage is done. My sister works in digital policy and she's been saying for like 2 years that deepfake regulation was comming way too slow. Now we've got a situation where millions of users have access to tools that can be weaponised against anyone with a public photo online.
This nails the enforcement gap: the issue isn’t only rules—it’s whether regulators can act faster than the harm can scale.