How xAI’s Grok is Linked to CSAM Creation—Now Facing User Lawsuits

It’s important to highlight that any CSAM detected by xAI will be reported directly to NCMEC.

In its complaint, xAI maintains that Harwood is solely accountable for his actions, claiming he “grossly violated” xAI’s guidelines and “went to great lengths to circumvent” Grok’s technical safeguards.

xAI asserts that Harwood used “misleading prompts” to achieve this, and the company alleges he failed to monitor his own actions even after recognizing he was potentially generating illegal content. The company believes Harwood should have known he was banned from using Grok after his initial reliance on the chatbot to create prohibited content. However, xAI does not provide evidence that Mr. Harwood was informed about the risk of penalties for his account.

Rather, xAI argues that Mr. Harwood “continued to use Grok during the relevant period despite violating xAI’s Terms of Use.” According to xAI, “the Terms of Use he agreed to explicitly prohibited his use after a previous violation.”

xAI anticipates a ruling from a U.S. district court that Harwood breached xAI’s terms and conditions, along with the contractual agreement he entered into. More significantly, Musk seeks to uphold an indemnity clause stating that only users, not xAI, are accountable for the CSAM and NCII generated by Grok. xAI contends that users are responsible for all content created while using Grok, encompassing both inputs and outputs.

The outcome of whether courts will hold users accountable for AI-generated output remains uncertain. Notably, the Copyright Office does not classify AI output as human-created. Any challenges courts face in determining the legality of AI-generated child sexual images—compared to other images that may not be credited—could expose vulnerabilities in xAI’s position.

If xAI prevails in this matter, Harwood could face substantial damages, including “any actual damages to third parties,” xAI’s “exposure to potential third-party claims or litigation,” and “damage to xAI’s reputation,” as outlined in the complaint.