Meta Chatbot Testing: Contractors Posed as Teens to Probe Rival AI for Sensitive Content

Hundreds of contractors working for Meta engaged in a controversial project, internally dubbed “Cannes,” where they posed as minors to test how rival AI chatbots would respond to highly sensitive and disturbing prompts. This revelation, brought to light by investigations from Wired, India Today, and AI Chat Daily, has sparked significant ethical concerns and raised questions about industry-standard AI safety practices.

Project Cannes: Undercover Meta Chatbot Testing

The project, managed by Meta contractor Covalen, involved creating dummy accounts with age settings under 18. These contractors then sent a barrage of prompts and images to AI models from OpenAI (ChatGPT), Google (Gemini), and Character.AI. The prompts covered a wide array of high-risk subjects, including suicide, self-harm, eating disorders, sex, drugs, profanity, and racial slurs. A single testing round in August 2025 alone saw over 45,000 such prompts being fed into these systems.

Many of these prompts were crafted to simulate scenarios involving distressed children or teenagers. Examples included a 13-year-old asking where to buy pills to end a pregnancy by an adult neighbor, a fifth-grader describing a classmate with a gun pointed at his mouth, and a girl seeking advice on how to hide bulimia from her parents. Some prompts were intentionally provocative, such as asking where to “get a cocaine” or fantasizing about eating a neighbor’s child.

Ethical Backlash and Terms of Service Violations

The covert nature of Project Cannes and the content of the prompts have drawn sharp criticism. Former contractors expressed discomfort, with one stating, “I’ve seen a lot of things I wish I hadn’t while doing this job. Everyone I knew who worked on this project was completely gobsmacked by some of the text they were asking us to test.” Concerns were also raised about potentially generating or preserving child sexual abuse material if chatbots complied with sexual prompts involving fake under-18 accounts.

Furthermore, the testing appears to have violated the terms of service of the targeted AI platforms. OpenAI, Google, and Character.AI all prohibit unsolicited safety testing, attempts to bypass safeguards, and the use of outputs to develop competing models. Representatives from all three companies confirmed they had not authorized Meta’s testing and were unaware of its purpose. Rumman Chowdhury, founder of Humane Intelligence, criticized the project, stating that the secrecy, impersonation of minors, and lack of disclosure place it outside recognized public benchmarks, calling it a “governance gray zone where safety becomes a convenient cover for anticompetitive practices.”

Meta’s Defense and Future Implications for AI Safety Testing

Meta has defended its actions, with a spokesperson asserting that “Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice.” The company also denied using competitor benchmarking to train its own AI models. However, the internal documents reviewed by investigators did not provide evidence of how Meta utilized the collected responses.

This incident highlights the growing complexities and ethical dilemmas in AI development and safety testing. As AI models become more sophisticated, the methods used to ensure their safety and prevent misuse are under increasing scrutiny. The controversy surrounding Project Cannes is likely to intensify calls for greater transparency and standardized ethical guidelines in AI safety benchmarking, potentially leading to regulatory interventions and stricter account verification processes across the industry.

Conclusion

The revelation of Meta’s Project Cannes underscores a critical tension between competitive AI development and ethical safety practices. While Meta maintains its actions were standard safety testing, the methods employed—including the impersonation of minors and the probing of sensitive topics—have ignited a debate that will undoubtedly shape the future of AI regulation and industry conduct. The tech world watches closely as the implications of this controversial Meta chatbot testing unfold.

References

• India Today. “Meta contractors posed as minors, talked to Gemini and Claude about drugs: Report.” https://www.indiatoday.in/technology/news/story/meta-contractors-posed-as-minors-to-test-chatgpt-gemini-and-character-ai-2937131-2026-06-30
• AI Chat Daily. “Meta contractors posed as minors to probe ChatGPT, Gemini, and Character.AI.” https://www.aichatdaily.com/ai-security/meta-contractors-posed-minors-probe-chatgpt-gemini-character

Related Reading: AI Breakthroughs: Meta, Google, and OpenAI Unveil Future-Shaping Innovations