FBI warns of dramatic rise in AI-generated child sexual abuse material
Guardii Analysis
The Federal Bureau of Investigation has issued a public warning about a significant increase in the creation and distribution of child sexual abuse material generated or manipulated using artificial intelligence tools. In a statement released in mid-November 2024, the FBI reported that offenders are using generative AI to produce realistic images and videos depicting child sexual abuse, as well as to alter innocent photos of real children into sexualised content, which is then shared on online platforms and used to extort victims. The agency noted that AI-generated CSAM is often difficult to distinguish from photographic images, complicates victim identification, and is being deployed at scale by predators to groom and coerce children.
Traditional hash-matching systems, which rely on databases of known photographic CSAM, cannot identify novel synthetic or manipulated imagery created by generative models. Detection technology purpose-built for AI-generated and deepfake abuse material, such as the module deployed by Guardii across Instagram, Snapchat and other direct-message platforms, analyses image characteristics and exchange context in real time to flag synthetically produced CSAM before it is used as a grooming or sextortion tool. By intercepting these images at the point of transmission rather than waiting for them to be reported and catalogued, Guardii closes the gap the FBI has identified, preventing AI-enabled abuse from reaching a child or being leveraged in coercion.