Meta rolls out AI systems to police scams, abuse, and harmful content

Meta

• Meta to boost AI for moderating Facebook, Instagram content
• AI systems now detect twice as much adult content as humans
• Meta launches AI-powered support assistant for 24/7 user help

Did our AI summary help?

Meta is leaning harder into AI to clean up its platforms, with plans to let automated systems handle a bigger chunk of content enforcement.

The company says it will gradually deploy more advanced AI models across apps like Facebook and Instagram—but only once they consistently outperform its current systems. At the same time, Meta will dial back its reliance on third-party moderation vendors, a move that could reshape how content policing works at scale.

This isn’t about removing humans entirely. Meta is clear that people will still be involved, especially for high-risk decisions like appeals or cases that may require law enforcement. But the company wants AI to take over the repetitive, high-volume work—reviewing graphic content, spotting scams, and keeping up with constantly evolving tactics used by bad actors.

And Meta is backing that shift with early data. According to the company, its AI systems are already detecting twice as much adult solicitation content compared to human review teams, while cutting error rates by over 60%. It also claims improvements in spotting impersonation accounts—particularly those targeting celebrities—and flagging suspicious account activity, such as unusual logins or sudden profile changes.

Scams remain a major focus. Meta says its systems are now identifying and blocking around 5,000 scam attempts every day, many of which try to trick users into sharing login credentials.

The broader pitch is simple: AI can move faster, scale better, and adapt quicker than human teams—especially in areas where tactics keep changing. But the shift also comes with context.

Over the past year, Meta has been loosening parts of its content moderation playbook. It scrapped third-party fact-checking in favour of a Community Notes-style system and eased restrictions around certain types of political content. At the same time, the company—and others in Big Tech—continues to face legal pressure over platform safety, particularly when it comes to younger users.

Alongside the moderation push, Meta is also rolling out a new AI-powered support assistant. The tool will offer 24/7 help within Facebook and Instagram, both on mobile and desktop, giving users another way to resolve issues without waiting on human support.