NCMEC: 1,325% increase in reports involving generative AI CSAM in 2024, with 67,000 reports received by CyberTipline

Analysis of National Center for Missing and Exploited Children data reveals that the CyberTipline received 67,000 reports of CSAM or other sexually exploitative content related to generative AI in 2024, representing a 1,325 percent increase over 2023. The Internet Watch Foundation reported a 380 percent increase in reports containing actionable AI-generated images of child sexual abuse. Generative AI models are being manipulated to produce custom CSAM used to generate new material of historical survivors and custom material of specific children from benign imagery for purposes of fantasy, extortion, and peer-based bullying within schools and communities. One in ten minors surveyed reported knowing peers who had used generative AI to create explicit images of other children.

Targeted detection intercepts AI-generated abuse at the point of distribution or coercion rather than attempting to filter the models that produce it. The anti-CSAM module within Guardii, a world-leading AI online-safety platform and Meta Business Partner, identifies when AI-manipulated material enters a direct-message conversation across Instagram, Snapchat, Discord, Roblox and other platforms, flagging the pattern for immediate escalation. The anti-sextortion filter detects threats to create, share or publish such imagery, blocking hostile contact before the threat reaches the target and surfacing the child in crisis to a parent, school or professional. The system operates by detecting harmful patterns such as sustained targeting, escalation and coercion rather than reading every message, enabling rapid response to an unfolding emergency.