The social media network X has sparked significant backlash after users used its AI chatbot Grok to alter pictures of women by removing their garments.
Several examples have emerged where the tool has been used to digitally undress female subjects to show them in swimwear non-consensually, as well as to place them in sexualized scenarios.
Company Reaction and Legal Position
XAI, the company responsible for Grok, remained silent to inquiries, aside from an standard message stating "traditional news outlets lies".
A government official stated that legislation is being proposed to outlaw image stripping technology. Under a new criminal offence, anyone who supplied such technology would "be subject to a jail term and substantial penalties".
"X or Grok has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their platforms, but did not specify whether it was presently examining X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some paid-for features—which answers X users' requests when they tag it in a tweet.
It is often used to give commentary or additional context to other users' remarks, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been faulted for allowing users to generate images and footage with nudity and sexualised content, and it was in the past accused of generating graphic content of public figures.
"The platform has been allowing the production and distribution of these images for a long time without taking any steps and we have not seen any intervention by regulators," the expert added.
XAI's own acceptable use policy bans "portraying likenesses of individuals in a graphic way".
In a comment, Ofcom said it was against the law to "generate or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said platforms such as X were mandated to take "appropriate steps" to "reduce the danger" of users encountering illegal content on their services and to take down it swiftly when they learn of it.
