Stanford and law enforcement research highlights harms of AI-generated child sexual abuse material
Guardii Analysis
Most schools are not talking to students about the risks of AI-generated child sexual abuse material, specifically via nudify apps, nor are they training educators how to respond to incidents of students making and circulating deepfake nudes of other students. While many states have recently criminalized AI CSAM, most fail to address how schools should establish appropriate frameworks for handling child offenders who create or share deepfake nudes. Through 52 interviews conducted between mid-2024 and early 2025 and a review of documents from four public school districts, Stanford researchers find that the prevalence of AI CSAM in schools remains unclear but appears to be not overwhelmingly high at present. AI-generated CSAM has become easier to create thanks to the proliferation of generative AI software programs commonly called nudify, undress or face-swapping apps, which are purpose-built to let unskilled users make pornographic images, and some of those users are children themselves.
Schools lack both detection capability and clear escalation protocols when students weaponize AI-generated CSAM against peers. Guardii's anti-CSAM detection module, developed for deployment across Instagram, Snapchat, Discord, Roblox and other platforms students use, identifies AI-generated and authentic child sexual abuse material in direct-message conversations in real time, flagging the exchange before material circulates further and surfacing the incident to the school safeguarding lead or authority with jurisdiction. The system, backed by Startmate and operating as a Meta Business Partner, enables a school to detect a deepfake-nude incident at the moment of sharing, report to the responsible agency, and activate a trauma-informed response to both the targeted child and the offending student without relying on victim disclosure or teacher awareness of a platform-native event.