EnforcementTuesday 17 November 2026via Business Standard

OpenAI steps up child safety with new image classifier and ChatGPT warnings

Guardii Analysis

OpenAI announced on November 17, 2026, the introduction of an AI image classifier to detect child sexual abuse material, developed with assistance from the National Center for Missing and Exploited Children, alongside new warnings and temporary account restrictions in ChatGPT to discourage attempts to access or generate abusive material involving children. The company made the classifier tool available to vetted organisations through its API at no cost and introduced interventions within ChatGPT to prevent generation of CSAM.

Platform-side classifiers that scan for known or generated CSAM address supply-side distribution but do not intercept the demand-side contact where an offender approaches a child, shares generated imagery to normalize abuse, or uses AI-generated content as leverage for sextortion. Anti-CSAM detection operating within the messaging layer flags both the sharing of AI-generated child sexual abuse material and the surrounding conversation pattern, identifying coercion, escalation and the use of synthetic imagery as a grooming or extortion tool. This approach, deployed by a world-leading AI online-safety platform across Instagram, Discord, Roblox and other environments, detects the harmful contact in context rather than relying solely on the content itself, enabling intervention at the point where an offender attempts to weaponize AI-generated material against a child.

Related Articles