Ofcom, the UK communications regulator, published in August 2024 its final codes of practice for tackling illegal content online under the newly in-force Online Safety Act, setting out the measures platforms must take to prevent users encountering child sexual abuse material, terrorism content and other illegal harms. The illegal harms codes apply to a wide range of user-to-user services and search engines, with the largest and highest-risk platforms facing the most stringent requirements including proactive monitoring, risk assessment and swift content removal. Companies that fail to comply face fines of up to 10 percent of global annual revenue, and Ofcom has indicated that enforcement will begin once platforms have had time to implement the required systems.
Regulatory codes that mandate proactive monitoring and risk assessment create the legal framework, but the operational question is where in the user journey the duty is discharged. Systems that scan public posts or remove flagged content after publication address part of the risk; monitoring that applies behavioral and linguistic detection to private direct messages — where the majority of child grooming, sextortion and CSAM distribution occurs — intercepts harm at the point of contact. Guardii, a Meta Business Partner and AI-driven safety platform backed by Startmate, monitors direct messages in real time across Instagram, Snapchat, Discord, Roblox and other services, using dedicated modules for grooming, sextortion, CSAM including AI-generated material, and other high-risk conduct, and surfaces a child in crisis or a predatory approach before the conversation escalates, aligning operational capability with the proactive duties the Online Safety Act now imposes.
