The Canadian Centre for Child Protection published research in August 2024 showing a 96 percent increase in reports of AI-generated child sexual abuse material compared to the previous year, with analysts identifying synthetic imagery in submissions to its Cybertip.ca reporting portal. The study found that offenders are using widely available generative AI tools and face-manipulation applications to create realistic abuse material from benign photographs of children, and that such content is increasingly difficult for automated systems and human moderators to distinguish from photographic CSAM. The Centre warned that the ease of production and the scale of distribution pose urgent challenges for law enforcement and technology platforms.
The operational challenge the research identifies is detection at the point of harm: distinguishing synthetic child abuse material from legitimate content and stopping it before it reaches a victim or spreads through a network. Anti-CSAM detection systems built into messaging infrastructure—such as those used by Guardii across Instagram, Snapchat, Discord, Roblox and other platforms—are trained to recognise both photographic and AI-generated abuse material using pattern analysis that does not depend on exact image matching. When the system identifies synthetic CSAM in a direct message, it blocks transmission, flags the attempt for review, and surfaces the threat to a parent, school or safeguarding professional, closing the gap between production and enforcement that the 96 percent increase represents.
