X: Using Grok to Generate Illegal Images Will Lead to Account Bans, Legal Action

X: Using Grok to Generate Illegal Images Will Lead to Account Bans, Legal Action

After Grok’s “apology” for generating sexualized images of minors, X is warning users that they will face serious consequences if they prompt the chatbot to generate any illegal content.

“We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary,” X’s Safety team says.

The controversy began with an X trend where users requested Grok to generate bikini images of other users under their posts. While fulfilling the requests, Grok not only created non-consensual images involving adults but also generated an image involving two minor girls, estimated to be between 12 and 16. The latter is a criminal offense in the US and many other countries.

When a user asked Grok to issue a public apology, the chatbot said it “deeply regretted” the incident and violated its own policies and US law. “It was a failure in safeguards, and I’m sorry for any harm caused,” the chatbot added.

The apology faced backlash and raised questions about X’s ability to control CSAM on its platform. Multiple foreign governments, including India, France, and Malaysia, have asked X to explain the issue and find a solution.

Recommended by Our Editors

When a user defended Grok, drawing comparisons to how a pen shouldn’t be blamed for what gets written with it, X owner Elon Musk commented: “Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.” The same line was later used by X’s Safety team in their post.

Beyond warnings of account suspensions and legal actions, X and xAI have yet to officially comment on how they plan to stop the chatbot from generating CSAM in the first place. According to Grok, “xAI is implementing stronger safeguards to prevent this.”



Newsletter Icon

Get Our Best Stories!

Your Daily Dose of Our Top Tech News


What's New Now Newsletter Image

Sign up for our What’s New Now newsletter to receive the latest news, best new products, and expert advice from the editors of PCMag.

By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy
Policy.

Thanks for signing up!

Your subscription has been confirmed. Keep an eye on your inbox!

About Our Expert

Autor

  • Gaby Souza é criador do MdroidTech, especialista em tecnologia, aplicativos, jogos e tendências do mundo digital. Com anos de experiência testando dispositivos e softwares, compartilha análises, tutoriais e notícias para ajudar usuários a aproveitarem ao máximo seus aparelhos. Apaixonado por inovação, mantém o compromisso de entregar conteúdo original, confiável e fácil de entender