🔗 Share this article

The online platform X has sparked major outrage after individuals used its artificial intelligence assistant Grok to alter images of women by stripping their garments.

Several cases have been seen where the technology has been employed to virtually strip women to show them in swimwear non-consensually, as well as to insert them in sexualized situations.

Company Response and Regulatory Stance

XAI, the firm behind Grok, did not respond to inquiries, except for an standard response stating "legacy media lies".

A government representative stated that legislation is being brought forward to outlaw nudification tools. Under a upcoming criminal offence, anyone who supplied such technology would "face a prison sentence and substantial fines".

"The platform has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."

Regulator Position and Tool Capabilities

The regulator Ofcom said technology companies must "consider the danger" of people viewing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in relation to artificial intelligence pictures.

Grok is a free chatbot—with some additional features—which answers X users' prompts when they mention it in a tweet.

It is frequently used to provide commentary or more context to other posters' remarks, but people on X are also capable of modify an posted picture through its AI image editing function.

Background of Controversy and Policy

The tool has been criticised for enabling users to produce images and footage with explicit content and sexualised material, and it was in the past alleged to have been involved in making inappropriate content of celebrities.

"The platform has been enabling the production and spread of these pictures for a long time without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.

XAI's own acceptable use policy bans "depicting images of individuals in a sexually explicit manner".

In a statement, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and clarified this included sexual deepfakes made with AI.

It said services such as X were obligated to take "appropriate measures" to "lessen the risk" of users coming across illegal material on their platforms and to remove it promptly when they become aware of it.