A new report by the U.K.-based Internet Watch Foundation (IWF) has revealed a troubling increase in AI-generated child sexual abuse material (CSAM) circulating online, underscoring the darker applications of AI technology.
The report, released Monday, documents a significant rise in deep fake videos and images within a 30-day review of a dark web forum dedicated to CSAM. The review identified 3,512 AI-generated CSAM items, marking a 17% increase from a similar study conducted in fall 2023. According to Dan Sexton, chief technology officer at IWF, these AI-generated materials are becoming increasingly realistic and depict more extreme sexual acts.
Sexton expressed concern over the trend, noting that while entirely synthetic videos still appear unrealistic, advancements in AI could soon lead to compelling depictions of child sexual abuse. Currently, perpetrators primarily use AI to enhance existing CSAM with techniques like face-swapping, which perpetuates harm against survivors by recycling old footage.
“The realism and severity of AI-generated CSAM are escalating, posing significant challenges for regulators, tech firms, and law enforcement,” Sexton remarked.
Despite major AI companies’ efforts to adopt ethical guidelines, smaller, less-regulated AI tools continue to proliferate online, facilitating the creation and dissemination of illegal content.
The report underscores the urgent need for enhanced measures to combat the exploitation of AI technology for criminal purposes amid growing concerns over its impact on victims and broader societal safety.
