Stanford Internet Observatory reports 300 percent increase in AI-generated child abuse imagery

Researchers at the Stanford Internet Observatory documented a 300 percent year-on-year increase in reports of suspected AI-generated child sexual abuse material to the National Center for Missing and Exploited Children in a study published on 14 October 2024, finding that generative AI tools are being deliberately misused to create synthetic CSAM and that such imagery is circulating on mainstream social media and messaging platforms. The report identifies significant challenges for existing hash-matching detection systems, which rely on databases of known abuse material and cannot recognise novel AI-generated images, and warns that the volume and accessibility of synthetic CSAM is outpacing the capacity of traditional content moderation.

Detection technology purpose-built to recognise AI-generated and manipulated child sexual abuse material closes the gap that hash-based systems leave open. Guardii's anti-CSAM module, deployed in real time within direct messaging on platforms including Instagram, Snapchat, Discord and Roblox, identifies both known and synthetic abuse imagery at the point of exchange, blocking distribution before the material reaches a child or wider network and escalating the incident to parents, safeguarding teams or law enforcement with the context needed for rapid response. Where offenders exploit generative AI to create content that evades retrospective detection, pattern recognition and image analysis at the moment of transmission—integrated into the conversation layer rather than applied retrospectively to uploaded content—provides the operational capability that research now identifies as urgent.