Meta takes down AI bot after complaints of fake research

(Representative image)

Meta has taken the demonstration for its artificial intelligence (AI) bot Galactica offline after several users complained that it was faking research papers in the latest setback for the Facebook parent after its metaverse fiasco.

Galactica was introduced as a large-scale language model for science that could "summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more".

Introducing Galactica. A large language model for science.

Can summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more. Explore and get weights: https://t.co/jKEP8S7Yflpic.twitter.com/niXmKjSlXW — Papers with Code (@paperswithcode) November 15, 2022

In a statement released to The Daily Beast, the Meta AI team said that the bot was trained using "106 billion tokens of open-access scientific text and data. This includes papers, textbooks, scientific websites, encyclopedias, reference material, knowledge bases, and more.”

Soon after the launch, Galactica seemed to be garnering negative reviews. Turns out, that the bot was generating fake research. Twitter users began sharing some hilarious made-up posts such as one about the Stanford Gaydar AI that was supposedly worked on in secret to create software that could analyse human faces to tell if they were gay or straight.

"Meta AI says computer scientist David Forsyth secretly worked at Stanford developing AI-powered Gaydar system for Facebook"

I'm tempted to publish that just to make sure the PR team at Meta have something to do today. Source: https://t.co/LOugQTqselhttps://t.co/tVrOXBWhJH — Tristan Greene (@mrgreene1977) November 16, 2022

Then there was one about "the benefits of eating crushed glass".

That's my primary concern. (see images) I got it to spit out numerous different formats for "the benefits of eating crushed glass." It hallucinated all kinds of positive statements, including study details, livestock trials, and chemical explanations: pic.twitter.com/jBiJUEBCdJ — Tristan Greene (@mrgreene1977) November 17, 2022

What was worrying was that the bot would create fake studies and attribute them to real researchers. It would also generate extremely convincing citations using the language model. 

Meta took down the demo two days after the launch and the Chief AI scientist at Meta Yann LeCun let his frustrations be known through a tweet. 

Galactica demo is off line for now. It's no longer possible to have some fun by casually misusing it. Happy? https://t.co/K56r2LpvFD — Yann LeCun (@ylecun) November 17, 2022

Meta recently had trouble with metaverse with many in the company complaining about the money assigned to an unproven project. Worse, no one within the company seemed to like the flagship Horizon World's app and the team in charge of the project was placed in a quality lockdown.    

Several top executives from the industry are unimpressed by the direction Meta is taking the app in, with Xbox head Phil Spencer saying that it resembled a "poorly built video game".