Canada’s Privacy Commissioner says X’s Grok AI chatbot violated law with sexualized deepfakes

The Office of the Privacy Commissioner of Canada (OPC) says X’s (Twitter) Grok chatbot violated privacy law by not having sufficient safeguards for its AI image generation feature at launch.

In a news release, the OPC wrote that these “lack of protections allowed users around the globe to create and share non-consensual, sexualized deepfakes, many targeting women and children.” In January, the Elon Musk-owned X came under fire internationally for allowing people to ask Grok to produce images of women and children in various states of undress — often in bikinis but sometimes in the nude — without their consent.

Per his investigation, Privacy Commissioner Philippe Dufresne said Grok was at one point “generating well over 6,000 [sexualized] images per hour,” which led to “millions” of sexual deepfakes being created in total. When this news originally surfaced in January, Canadian AI Minister Evan Solomon condemned X for allowing it to happen but ruled out banning the platform. MPs ultimately amended a proposed bill in April to criminalize deepfakes that include “nearly nude” imagery.

Despite all of this, neither X nor the terminally online Musk ever took accountability. Musk, for his part, claimed he was aware of “literally zero” instances of sexualized images being created by Grok, while X argued in an official statement that the onus was on users for making the inappropriate Grok queries in the first place. The social media platform also moved to restrict this feature to paying X Premium members — in other words, directly profiting from lewd and even nude imagery of women and minors.

In the new OPC report, Dufresne noted that X has since “introduced new measures, including safeguards to reduce the risk that their tool will be misused to produce sexualized deepfakes, and proactive sweeps to detect and remove this harmful content on their platforms.” He also noted that X has committed to introducing various safety measures for users, including quarterly reports and third-party audits on their improvements to safeguards.

However, he said he has “recommended additional action by the companies to further improve these measures and demonstrate their effectiveness in mitigating this issue.” He added that there needs to be “modern privacy laws that are designed for a modern world and include administrative monetary penalties and the power to make orders to bring companies into compliance.”

It should be noted, however, that there are still reports of Grok producing sexualized imagery of women. In a new Wired investigation published just this week, the outlet found that dozens of “nudified” images of celebrities, including at least one politician, are still be created and distributed on X.

The OPC didn’t address these reports of recurring deepfakes but said it will continue to monitor how X handles its commitment to implementing these increased safety measures. The federal government, meanwhile, just proposed a bill to ban social media for youth under 16 and place restrictions on AI chatbots.

Source: Office of the Privacy Commissioner of Canada