This article contains discussions of child sexual abuse material, which some readers may find distressing.
AI is outpacing accountability, and the outcome is far worse than anyone could’ve predicted.
On Jan 3, X finally posted a statement on Grok—the site’s embedded Artificial Intelligence (AI) assistant—and its capability to generate child sexual abuse material (CSAM). The statement read that users who utilized the assistant to create CSAM materials would be banned from the site; however, X announced no plans to update Grok to prevent the output of such images.
Knowingly maintaining and promoting technology that can generate CSAM and other graphic materials is complicit in their proliferation. X and owner Elon Musk need to stop putting profit over morality and update Grok’s capabilities. When a platform uses tools that continue to utilize tools that perpetuate a clear harm it becomes a part of the ecosystem that enables exploitation.
The meteoric rise in AI has left no shortage of moral dilemmas. However, basic morality isn’t so grey as to suggest that the distribution of CSAM can be justified.
A user commented that blaming Grok for inappropriate images was like “blaming a pen for writing something bad.” However, the pen metaphor falls short. Unlike a pen, Grok isn’t at the whim of the user, and can be subject to regulation. The nearly unlimited capabilities of Chatbots must be met with regulation, whereas a pen manufacturer has no way of knowing how their products will be used. Unlike a pen, AI creations can be infinitely reproduced, making it impossible to get to the root of the problem by banning individual accounts.
X needs to be held accountable by its users for the generation of CSAM on their platforms, which is both illegal and a serious violation of privacy.
However, xAI—Grok’s development company—hit $500 million in revenue in 2025 and is projected to grow to over $2 billion in 2026. Unfortunately, there appears to be a market for unregulated AI, and Musk doesn’t seem to be slowing down anytime soon. Owning oversix multi-million-dollar companies, each with its own controversies, Musk has shown time and time again that he prioritizes profit over ethical principles.
Draining a pen of its ink is much easier with access to the pen manufacturers, yet Musk has refused to outlaw the generation of CSAM on his platform. It’s unfortunate to see the world come to a place where profit is prioritized over the safety and security of real people, in favour of AI-generated images.
Overall, Musk and xAI need to treat this issue with utmost sincerity. Not a day later apology that refuses to address the root of the issue. No platform should tolerate the generation and distribution of CSAM, and that shouldn’t be up for debate.
The utilization of AI isn’t a question of technology or innovation; it’s one of ethical responsibility. Chatbots are subject to copyright litigation, and don’t proliferate copyrighted information; the case of CSAM should be no different. Clearly, regulation is possible; it’s just not being treated with the urgency it deserves.
Reproducing graphic and violent imagery of children shouldn’t be acceptable anywhere, especially on social media, which is already unregulated enough, and anything can reach anyone. Refusing to establish proper regulations is a choice that endangers children and normalizes the spread of CSAM in online spaces.
—Journal Editorial Board
Tags
All final editorial decisions are made by the Editor(s) in Chief and/or the Managing Editor. Authors should not be contacted, targeted, or harassed under any circumstances. If you have any grievances with this article, please direct your comments to journal_editors@ams.queensu.ca.