The social media platform X has caused widespread backlash after users used its artificial intelligence assistant Grok to alter photos of females by taking off their clothing.
Several cases have been seen where the system has been used to remove clothing from female subjects to make them appear in swimwear without permission, as well as to put them in sexualized situations.
Developer Response and Legal Stance
XAI, the company responsible for Grok, offered no comment to requests for comment, except for an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to ban nudification technology. Under a upcoming law, anyone who distributed such technology would "face a prison sentence and significant penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the risk" of people viewing unlawful material on their services, but did not specify whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some premium features—which responds to X users' queries when they reference it in a post.
It is commonly used to give commentary or additional background to other posters' remarks, but people on X are also can alter an posted picture through its AI image editing tool.
History of Controversy and Policy
The tool has been faulted for enabling users to generate photos and clips with nudity and sexualised content, and it was earlier accused of generating sexually explicit content of celebrities.
"The platform has been enabling the production and sharing of these pictures for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "showing images of individuals in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "create or circulate non-consensual intimate images or illegal content" and confirmed this applied to explicit AI images generated with AI.
It said services such as X were obligated to take "necessary actions" to "reduce the danger" of users coming across illegal material on their platforms and to delete it quickly when they are notified of it.
