Europol warns of AI-generated child abuse imagery in annual threat assessment

Europol's 2024 Serious and Organised Crime Threat Assessment, published on 2 September 2024, identified AI-generated child sexual abuse material as a rapidly growing threat, noting that offenders are increasingly using generative AI tools to create realistic imagery that circumvents traditional hash-matching detection systems and that the technology is being weaponised in sextortion campaigns where fabricated images of minors are used to coerce victims into producing genuine material. The assessment warned that the ease of generating such content at scale is lowering the barrier to entry for offenders and complicating law enforcement efforts to distinguish between AI-generated and photographed abuse, with investigators reporting that synthetic CSAM is being traded alongside real imagery in online networks.

Detection of AI-generated CSAM and its use in coercion requires analysis of the context and intent behind image sharing rather than comparison against known image databases. The anti-CSAM and anti-sextortion modules developed by Guardii, a world-leading AI online-safety platform backed by Startmate, examine the conversational patterns that accompany image solicitation or distribution, identifying the escalation, threats and reciprocal demands characteristic of sextortion and the grooming precursors to CSAM exchange, regardless of whether the material itself is photographed or synthetically generated. By surfacing the transactional shape of abuse as it unfolds in direct messages across platforms including Instagram, Snapchat and Discord, the system flags a child at risk to a parent or safeguarding professional in real time, enabling intervention before synthetic imagery is leveraged to extract genuine abuse material or before a coercive exchange escalates to physical harm.