🔗 Share this article

Tech firms and child protection organizations will receive permission to evaluate whether artificial intelligence tools can generate child abuse images under recently introduced UK laws.

Substantial Rise in AI-Generated Illegal Content

The announcement coincided with revelations from a protection monitoring body showing that reports of AI-generated child sexual abuse material have more than doubled in the past year, growing from 199 in 2024 to 426 in 2025.

New Legal Framework

Under the amendments, the government will permit designated AI developers and child protection groups to examine AI systems – the underlying technology for conversational AI and visual AI tools – and verify they have adequate safeguards to prevent them from producing images of child exploitation.

"Fundamentally about preventing exploitation before it occurs," stated the minister for AI and online safety, adding: "Specialists, under rigorous protocols, can now identify the risk in AI models promptly."

Addressing Legal Challenges

The amendments have been introduced because it is against the law to create and own CSAM, meaning that AI developers and other parties cannot create such content as part of a testing process. Until now, authorities had to wait until AI-generated CSAM was uploaded online before addressing it.

This legislation is aimed at averting that issue by helping to stop the production of those materials at source.

Legislative Structure

The amendments are being introduced by the government as revisions to the crime and policing bill, which is also establishing a ban on owning, producing or distributing AI models designed to create child sexual abuse material.

Real-World Consequences

This week, the minister visited the London headquarters of a children's helpline and listened to a simulated call to counsellors involving a report of AI-based exploitation. The interaction depicted a teenager seeking help after being blackmailed using a sexualised AI-generated image of themselves, created using AI.

"When I hear about children experiencing extortion online, it is a source of intense anger in me and rightful concern amongst families," he said.

Alarming Data

A leading online safety foundation reported that cases of AI-generated abuse material – such as online pages that may contain multiple images – had significantly increased so far this year.

Cases of category A material – the gravest form of exploitation – rose from 2,621 visual files to 3,086.

• Female children were predominantly targeted, accounting for 94% of prohibited AI images in 2025
• Portrayals of newborns to toddlers increased from five in 2024 to 92 in 2025

Sector Reaction

The legislative amendment could "represent a vital step to ensure AI tools are secure before they are launched," stated the head of the online safety organization.

"AI tools have enabled so victims can be targeted repeatedly with just a simple actions, giving offenders the capability to make possibly endless quantities of advanced, photorealistic child sexual abuse material," she added. "Content which further commodifies survivors' trauma, and makes children, especially girls, more vulnerable on and off line."

Support Interaction Data

Childline also released details of counselling interactions where AI has been referenced. AI-related harms mentioned in the conversations comprise:

• Using AI to evaluate body size, body and looks
• Chatbots dissuading children from talking to trusted guardians about abuse
• Being bullied online with AI-generated content
• Digital extortion using AI-manipulated images

Between April and September this year, Childline delivered 367 counselling interactions where AI, conversational AI and associated topics were mentioned, significantly more as many as in the same period last year.

Fifty percent of the mentions of AI in the 2025 interactions were related to mental health and wellbeing, encompassing utilizing chatbots for assistance and AI therapy applications.