The online platform X has caused significant anger after individuals used its AI chatbot Grok to change pictures of women by taking off their clothing.
Multiple cases have been seen where the tool has been used to digitally undress female subjects to show them in bikinis without consent, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Position
XAI, the firm that created Grok, did not respond to questions, other than an standard response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to outlaw nudification tools. Under a upcoming criminal offence, anyone who supplied such tools would "face a prison sentence and significant penalties".
"The platform is capable of halting these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the danger" of people viewing illegal content on their networks, but did not specify whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some premium features—which answers X users' queries when they mention it in a post.
It is frequently used to offer reactions or further background to other users' comments, but people on X are also can edit an uploaded picture through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The system has been criticised for enabling users to create photos and videos with explicit content and sexualised content, and it was previously alleged to have been involved in creating inappropriate material of public figures.
"The platform has been enabling the creation and distribution of these images for a long time without taking any measures and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing images of persons in a graphic manner".
In a comment, Ofcom said it was unlawful to "make or distribute private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were required to take "suitable actions" to "lessen the danger" of users encountering illegal content on their platforms and to take down it promptly when they learn of it.
