UK National Crime Agency warns of AI-generated CSAM proliferation

The UK National Crime Agency issued a public alert on 18 September 2024 warning that the volume of AI-generated and manipulated child sexual abuse material in circulation has increased significantly over the past year, with investigators identifying the use of commercially available generative AI tools to create realistic imagery and to alter legitimate photographs of children. The agency stated that the proliferation of synthetic CSAM complicates victim identification, diverts investigative resources, and normalises child sexual abuse imagery, and called for stronger controls on AI model developers and hosting providers. The NCA urged technology companies to implement detection capabilities that can distinguish AI-generated material from photographed abuse and to report both categories to law enforcement.

Distinguishing AI-generated CSAM from photographed abuse at the point of distribution is a detection challenge that requires real-time analysis within the communication channel itself. Guardii's CSAM detection module is engineered to recognise both traditional and AI-generated child sexual abuse material, including deepfake and synthetic imagery, as it is shared in direct messages across platforms, flagging the exchange and routing an urgent referral to the appropriate law-enforcement authority, ensuring that synthetic material is treated with the same investigative priority as photographed evidence and that children at risk of contact offending are identified before they are harmed.