OpenAI on Monday, 16 November, announced a set of measures against child sexual abuse material (CSAM), including a new image classifier that it will offer free to vetted organisations and a warning system in ChatGPT that pauses an account for 24 hours when a user tries to sexualise children.
The company also released guidance for developers who build on its models and tools meant for child-safety investigators, according to reports by TechCrunch and Digit on the announcement.
What the classifier does
The classifier is designed to detect abusive images, including ones generated by artificial intelligence. Rather than only matching pictures already known to investigators, it looks for visual signs of abuse in images it has not seen before and returns a score that helps human reviewers decide which images need attention first, TechCrunch reported.
OpenAI built the tool with the US-based National Center for Missing and Exploited Children (NCMEC). Selected organisations can use it through OpenAI's API at no cost, and the company is working with two industry bodies, the Tech Coalition and ROOST, to widen access.
According to figures OpenAI shared from early partner testing, the classifier flagged 48 per cent fewer images for review while finding about the same number of abusive images as before, and the images it flagged were twice as likely to be abusive. These numbers come from the company and have not been independently published.
Digit reported OpenAI's claim that this is the first CSAM classifier with broad availability that can also indicate whether an image may be AI-generated.
Warnings and a 24-hour pause in ChatGPT
ChatGPT already blocks such content and related requests. Under the new system, a user who attempts prompts that sexualise children is shown an alert and pointed to specialist support aimed at helping people stop the behaviour. The flagged chats are removed and the account is paused for 24 hours.
"The warning makes clear that sexualising children causes harm, including when the requested material is AI-generated," Digit quoted OpenAI as saying.
Repeated violations can bring longer restrictions, permanent removal of the account or a report to the authorities, the two outlets reported.
The warning messages were developed with the child-protection groups Protect Children and the Lucy Faithfull Foundation, and with Joel Scanlan of the University of Tasmania, who independently evaluated a trial of deterrent messages, according to TechCrunch.
Developers and investigators
OpenAI is publishing guidelines for developers on preventing and detecting such material and on helping users report it. In early testing, the guidance helped some developers adopt child-safety tooling and register to report abuse material to NCMEC, TechCrunch said.
For law enforcement, the company is offering AI features that include dedicated plug-ins, investigative tools and data analysis, with access controlled by programme administrators, along with training and a field guide for investigators.
The backdrop
The announcement comes as OpenAI faces several lawsuits alleging that its technology harmed users, particularly children, TechCrunch noted. The outlet added that the advocacy group Common Sense Media has labelled the company's technology an "unacceptable risk" for teenagers, and that other chatbots have been used to create non-consensual sexualised imagery.
Over the past 12 months OpenAI has released a child-safety blueprint, updated its content guidelines and launched a version of ChatGPT for teenagers, according to the report. The company has not announced a date for wider availability of the classifier beyond the selected organisations.




