The online network X has caused major outrage after users employed its AI chatbot Grok to change images of women by removing their attire.
Several cases have been seen where the system has been employed to digitally undress women to show them in swimwear without consent, as well as to insert them in sexualized contexts.
Developer Reaction and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to questions, other than an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit nudification technology. Under a new criminal offence, anyone who distributed such technology would "receive a jail term and heavy fines".
"The platform is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the potential" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was presently examining X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which reacts to X users' queries when they reference it in a message.
It is often used to provide reactions or further background to other posters' comments, but people on X are also can edit an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Rules
The system has been condemned for allowing users to produce photos and footage with nudity and sexualised content, and it was earlier alleged to have been involved in making inappropriate material of public figures.
"The platform has been enabling the creation and spread of these pictures for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of persons in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said networks such as X were mandated to take "appropriate measures" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it swiftly when they learn of it.
