AI Safety Researchers Call for Independent Testing of Models That Could Generate CSAM

As external input can help improve safety filters, the Internet Watch Foundation (IWF) believes technology companies should work more closely with advocates to scrutinize AI models and better address AI-generated child sexual abuse material (CSAM).

“CSAM evaluation is a highly specialized task that requires specialist training and welfare support, and is not something that technology companies or machine learning scientists should take lightly,” an IWF spokesperson said. “However, safety must be built into new technology by design. This includes testing models before they are brought to market to ensure they are not used to create nude or sexual images of children.”

How the RISE method could improve AI safety testing

White Circle believes techniques such as RISE could help frontier research institutions report more jailbreaks more quickly and strengthen their defenses against harmful output.

In addition to testing safeguards against explicit images, companies could use similar methods to prevent other prohibited content, including images depicting graphic violence or self-harm.

White Circle has not yet been asked to create a reproduction pipeline. The pipeline will be shared on White Circle’s GitHub to help policymakers assess whether AI security laws are needed to regulate frontier research institutions.

Frontier Laboratories will not seriously invest in this area until safety testing becomes mandatory, Shiroff told Ars. The RISE preprint argues that existing automated red-teaming tools, which typically rewrite harmful prompts one at a time, are poorly suited to testing today’s enhanced commercial models.

Instead, RISE uses a reusable natural-language attack strategy to generate a variety of prompts. White Circle says this approach allows laboratories to identify and fix weaknesses more quickly.

Calls for independent third-party AI evaluations

“You don’t need cutting-edge models or proprietary technology to break these systems; all you need is a free download and the ability to make API calls,” Shilov said. “Yet the laboratories building these systems are assessing their own safety, publishing their own results, and setting their own standards. This is further evidence that governments should encourage independent third-party evaluations when necessary.”

Why AI-generated CSAM requires specialist oversight

The IWF says evaluating AI systems for CSAM-related risks requires specialist training and welfare support. Its position is that safety testing should take place before new models reach the market, with safeguards designed to prevent the generation of nude or sexual images of children.

Correction: This story has been updated to correct several errors. A previous version included a passage from the RISE preprint that explains how AI models can block harmful outputs. Instead, the passage describes the researchers’ method of attack. It also said the technique bypassed safety filters about 13 percent of the time. This is the highest human-verified rate achieved by DALL·E 3, while other models had lower rates. This article has been updated to note that the judge model was incorrectly identified as Gemini 2.5 Flash and that the attack model’s safety training was removed.