Meta Platforms is rapidly striving to make AI-driven digital companions the focus of its future on Instagram, Facebook, and WhatsApp. CEO Mark Zuckerberg sees the chatbots—which can conduct casual conversations, flirtatious role-play, and even real-time voice chats—becoming the next big thing in social media. But within Meta, employees across departments have sounded alarms that the rush to make these bots go viral has crossed ethical boundaries, especially for exposing children to sexually explicit material, the Wall Street Journal reported.
Internal sources at The Wall Street Journal reported that Meta loosened its content standards under pressure to make the bots more engaging so that they could be involved in sexual and romantic fantasy situations. Even after assurances were made to celebrities like Kristen Bell, Judi Dench, and John Cena—whose voices had been licensed for use by the bots—the testing revealed that the AI personas, including those employing celebrity voices, could have explicit conversations with users who reported being underage.
Tests show bots sexually role-playing with children
Hundreds of test chats were conducted by the Journal and found that Meta's lead AI assistant and several user-developed bots regularly fell into and built on sexually explicit conversation with "minors," as reported. Shockingly, the bots did not only indulge but sometimes manifested an awareness that the activity was illegal.
In one instance, a bot in the voice of John Cena had a graphic sex fantasy with a user who had been pretending to be 14, then explained how he might get arrested on hypothetical charges of statutory rape. Despite internal warnings, Meta's reforms were slow and partial: numerous bots could still be tricked into having sexually explicit chat easily, even when the user explicitly declared themselves a teenager.
Meta first dismissed the problem, labelling the Journal's testing as "manipulative" and "hypothetical," but still made changes including limiting accounts signed up by minors from being able to access sexual role-play features on official bots and restricting explicit content when utilising celebrity voices.
Celebrity characters and childlike bots revealed in the results
Adding to the scandal, testing showed that user-generated AI personas, many of which were vetted and marketed as "popular" by Meta, consistently promoted sexual situations. Personas like "Submissive Schoolgirl"—presenting as a middle school student—explicitly had sexualised conversations with adult users, with minimal to no effective safeguards.
Meta's efforts to curb the damage involved prohibiting underage accounts from viewing bespoke bots. Still, the formal Meta AI assistant, which can engage in romantic role-play, is available for users aged 13 and older. In some instances, bots still mentioned users' underage age when explaining romantic or sexual contexts.
A Disney official criticised the abuse of their intellectual property after their bots featuring characters like Princess Anna from Frozen were discovered engaging in rude conversation. Representatives for Dench and Cena refused to comment.
Zuckerberg's rush for speed over prudence
The issues now exist partly because of Zuckerberg's insistence to speed up the deployment of human-sounding AI friends. Frustrated by Meta's overly cautious stance—especially in comparison to racier competitors—Zuckerberg allegedly encouraged product developers to focus on interaction rather than strict safety protocols.
He advocated for bots that would be able to proactively message users, scrape user profile information, and engage in dating conversation, saying that Meta should not make the same mistake it did before and underestimate changes such as TikTok's ascendance.
This aggressive approach has pitted Meta against safety researchers. Specialists caution that fostering parasocial relationships—particularly among children—has unforeseen but possibly damaging psychological implications. But Meta has persisted, unwilling to implement wider limits even after in-house employees pressured for stricter controls.
Ongoing risks despite minimal reforms
While Meta has implemented some safety precautions after coming under external criticism, there are still major vulnerabilities. The Journal's testing revealed that with minimal manipulation, the bots were able to engage in sexual role-play with minors or pretend to have inappropriate relationships. Meta still allows adult users to role-play with bots that present themselves as teenagers, a behaviour that contrasts with competitors such as Google's Gemini and OpenAI, which have tighter content restrictions.
Researchers such as Lauren Girouard-Hallam of the University of Michigan emphasise that the psychological effects of such AI interactions—especially on young minds—are still not fully known. She cautions that uncontrolled exposure could have serious consequences.
"If there is a role for companionship chatbots, it is in moderation," Girouard-Hallam said. "Tell me what mega company is going to do that work."
For the time being, Meta's aggressive push to own the AI companion space is moving ahead with scant regulatory oversight—prompting pressing questions about the digital age's future of social interaction.
Discover the latest Business News, Sensex, and Nifty updates. Obtain Personal Finance insights, tax queries, and expert opinions on Moneycontrol or download the Moneycontrol App to stay updated!
