🔗 Share this article

Technology companies and child safety agencies will receive permission to evaluate whether artificial intelligence systems can produce child abuse material under recently introduced British legislation.

Significant Rise in AI-Generated Illegal Material

The declaration came as findings from a protection monitoring body showing that reports of AI-generated CSAM have more than doubled in the last twelve months, rising from 199 in 2024 to 426 in 2025.

New Regulatory Framework

Under the amendments, the government will allow designated AI developers and child protection organizations to inspect AI models – the underlying technology for chatbots and visual AI tools – and ensure they have adequate protective measures to prevent them from producing images of child sexual abuse.

"Ultimately about preventing exploitation before it happens," stated Kanishka Narayan, adding: "Specialists, under rigorous conditions, can now identify the risk in AI models promptly."

Tackling Legal Obstacles

The amendments have been implemented because it is against the law to produce and possess CSAM, meaning that AI developers and other parties cannot create such images as part of a testing regime. Until now, authorities had to wait until AI-generated CSAM was uploaded online before addressing it.

This law is aimed at averting that problem by helping to stop the production of those materials at source.

Legal Framework

The amendments are being introduced by the government as revisions to the criminal justice legislation, which is also establishing a prohibition on owning, creating or sharing AI systems designed to generate exploitative content.

Real-World Impact

This week, the minister visited the London headquarters of a children's helpline and heard a mock-up conversation to counsellors involving a report of AI-based exploitation. The call depicted a adolescent seeking help after being blackmailed using a explicit deepfake of himself, created using AI.

"When I hear about children facing extortion online, it is a source of extreme anger in me and justified anger amongst parents," he stated.

Alarming Data

A leading internet monitoring organization reported that instances of AI-generated abuse material – such as online pages that may include multiple files – had more than doubled so far this year.

Instances of category A content – the gravest form of abuse – increased from 2,621 images or videos to 3,086.

• Girls were overwhelmingly victimized, making up 94% of prohibited AI depictions in 2025
• Depictions of newborns to two-year-olds rose from five in 2024 to 92 in 2025

Industry Response

The law change could "constitute a crucial step to ensure AI products are safe before they are released," commented the head of the internet monitoring organization.

"Artificial intelligence systems have made it so victims can be victimised repeatedly with just a few clicks, giving criminals the capability to make possibly limitless amounts of advanced, photorealistic child sexual abuse material," she added. "Material which additionally commodifies survivors' suffering, and renders children, particularly female children, less safe both online and offline."

Support Session Information

The children's helpline also released details of support sessions where AI has been mentioned. AI-related risks mentioned in the sessions comprise:

• Using AI to evaluate body size, physique and appearance
• AI assistants dissuading children from talking to safe guardians about harm
• Facing harassment online with AI-generated content
• Online blackmail using AI-faked images

During April and September this year, Childline conducted 367 support interactions where AI, chatbots and associated topics were discussed, four times as many as in the equivalent timeframe last year.

Fifty percent of the references of AI in the 2025 sessions were related to mental health and wellness, encompassing utilizing chatbots for support and AI therapy applications.