AI image generator reportedly permits sexually explicit depictions of children
An AI image-generation company reportedly permits users to create sexually explicit pictures of children, according to a report published by the Florida Phoenix.
The issue highlights the safety risks posed by generative-image systems when their moderation controls fail to block prohibited depictions. Such capabilities can enable child sexual abuse material and may also facilitate harassment or exploitation.
The available report does not name the company or provide details about its filtering systems, safeguards, or response. It also does not say how widely the reported capability has been used.
The incident underscores the need for robust testing, restrictions on prompt-based abuse, and mechanisms for reporting and removing harmful generated content. It also raises broader questions about accountability when an AI system can produce abusive imagery despite known risks.