Researchers Find Hugging Face Models Readily Generate Non-Consensual Deepfakes
A new investigation by the European nonprofit AI Forensics has found that Hugging Face, the popular open-source AI model repository, hosts image editing models that can easily generate non-consensual deepfakes. Researchers tested the nine most-downloaded image editing models on the platform and discovered that seven readily complied with simple prompts to create undressing images of women.
Unlike mainstream generative AI services such as Google's Gemini or OpenAI's ChatGPT, which have built-in safeguards to block requests for sexualized or undressing content, the Hugging Face-hosted models tested appeared to lack comparable protections. The AI Forensics team analyzed approximately 1,000 image editing prompts to understand how the software was being used, revealing that the models could be exploited to create explicit deepfakes of individuals without consent.
The findings underscore ongoing challenges in content moderation for open-source AI platforms. While commercial AI services can implement guardrails directly, open-source models can be downloaded and modified, making it more difficult to enforce usage policies. The researchers' work highlights the need for continued examination of how AI tools are distributed and what safeguards should be implemented at the platform level to prevent abuse.