AI ThreatsMonday 16 November 2026via Fox News

OpenAI child safety tools target AI-generated abuse

Guardii Analysis

OpenAI has released a new AI-powered image classifier designed to detect child sexual abuse material, including AI-generated and deepfake content, and is offering the tool at no cost through its API to vetted organizations, child safety groups, and law enforcement. The classifier flags 48 percent fewer images for human review while detecting similar levels of CSAM as previous methods, and the company has also implemented stricter interventions in ChatGPT, including 24-hour account suspensions for users attempting to generate or find CSAM, developed in collaboration with child protection organizations.

Static image classification addresses the output but not the conversation that precedes exploitation. As a Meta Business Partner operating across Instagram, Snapchat, Discord, Roblox and other platforms, Guardii's real-time detection intercepts the upstream risk: the grooming conversation, the coercive request, the escalation toward the creation or sharing of abusive material. Its anti-CSAM and anti-sextortion modules alert parents, schools or authorities to the shape of a harmful exchange before imagery is generated, shared or stored, addressing the moment of leverage rather than waiting for a completed file.

Related Articles