The Internet Watch Foundation reported on 27 August 2024 that confirmed cases of AI-generated child sexual abuse material increased by 29 per cent in the twelve months to July 2024 compared with the previous year, with analysts identifying more than 11,000 discrete images assessed as synthetic or manipulated using generative AI tools. The charity, which operates the UK hotline for reporting illegal online content, stated that the majority of AI-generated CSAM was distributed via social media direct messages, private forums and encrypted channels, making detection and removal more challenging than material hosted on public websites. IWF chief executive Susie Hargreaves described the trend as an emerging threat that demands new technical countermeasures alongside law enforcement action.
When synthetic abuse material migrates into private messaging, detection must operate within the message stream itself rather than relying on retrospective user reports. Guardii's CSAM detection module, which identifies both photographic and AI-generated child sexual abuse imagery in real time across Instagram, Snapchat, Discord and Roblox, blocks such material before it is delivered to the recipient and enables immediate escalation to the Internet Watch Foundation, the National Crime Agency or the relevant jurisdiction's authority. As a Meta Business Partner with live-deployment capability, the platform closes the operational gap the IWF has identified: it intercepts prohibited content at the point of distribution within direct messages, preventing exposure and generating actionable intelligence while the exchange is active, rather than discovering the material only after it has been shared, saved and potentially re-distributed.
