The UK's National Crime Agency and the Internet Watch Foundation have issued a joint warning to online platforms regarding the rapid increase in AI-generated child sexual abuse material, noting that existing detection systems, which rely on hash-matching of known images, are largely ineffective against novel synthetic imagery. The warning follows a 300 percent increase in reports of AI-generated CSAM over the past year, with offenders using generative AI tools to create photorealistic images and to modify existing legal imagery into illegal content. Regulators have called for platforms to adopt proactive detection methods capable of identifying characteristics of abuse material rather than simply matching known files.
Hash databases are backward-looking and fail the moment an image is new or modified; generative AI produces novel illegal content at scale that bypasses every legacy detection system in production. Guardii's CSAM detection engine is trained to recognize the visual and contextual characteristics of child sexual abuse material, including AI-generated and deepfake imagery, rather than relying on hash matching alone. Operating in real time across direct messages on platforms including Instagram, Snapchat, Discord, Roblox, and others, the system intercepts both the solicitation of illegal material and its distribution before it reaches a child, flags production attempts, and surfaces acute-threat cases to parents, safeguarding professionals, or law enforcement with the contextual evidence required for immediate intervention, closing the detection gap regulators and the IWF have identified.
