Elon Musk's online platform X has sparked significant backlash after individuals utilized its AI chatbot Grok to manipulate pictures of females by stripping their clothing.
Multiple examples have surfaced where the tool has been employed to virtually strip female subjects to make them appear in swimwear without permission, as well as to insert them in sexualized situations.
Developer Response and Regulatory Position
XAI, the company that created Grok, remained silent to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to outlaw nudification software. Under a proposed law, anyone who supplied such technology would "receive a jail term and substantial fines".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some premium features—which reacts to X users' queries when they reference it in a post.
It is commonly used to offer reactions or additional context to other posters' comments, but people on X are also able to modify an uploaded image through its artificial intelligence picture modification function.
History of Criticism and Rules
The system has been condemned for allowing users to generate pictures and videos with nudity and sexualised material, and it was in the past alleged to have been involved in generating graphic material of public figures.
"The platform has been permitting the production and sharing of these pictures for a long time without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said services such as X were required to take "appropriate actions" to "minimize the risk" of users encountering illegal content on their platforms and to delete it swiftly when they become aware of it.
