EU regulation mandates detection obligations for online child sexual abuse material

The European Parliament approved new rules on 16 October 2024 requiring online platforms to detect, report and remove child sexual abuse material, including AI-generated content, with a new EU Centre on Child Sexual Abuse established to coordinate detection orders, support cross-border investigations and maintain a database of abuse material hashes. The regulation mandates risk assessment by service providers and permits competent authorities to issue binding detection orders where a significant risk of CSAM distribution is identified, balancing detection obligations with privacy safeguards including independent review and time-limited orders.

Guardii's real-time pattern and content detection within direct messaging satisfies the EU's risk-based detection mandate without mass surveillance of all communications. Operating across Instagram, Snapchat, Discord, Roblox and other platforms, the technology deploys anti-grooming, anti-sextortion and anti-CSAM modules—including recognition of synthetic and deepfake imagery—that identify harmful escalation and material exchange as it occurs, reporting the nature and severity of a threat to the appropriate authority rather than collecting or retaining message content at scale. For platforms subject to detection orders, and for schools and safeguarding organisations with a duty of care, the system enables compliance with the regulation's prevention and intervention obligations while preserving the privacy of benign conversations and focusing resources on the contacts that present genuine risk to children.