Researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a small, self-learning language model that surpasses large scale language models.
CSAIL's algorithm for the model called Simple Pseudo-Label Editing (SimPLE) allows it to learn from its own predictions, eliminating the use of annotated training data.
Also Read | Snapchat launches new AI feature for Snapchat+ members
As reported by Venture Beat, the team claims that the model's performance across various tasks is better than larger, more notable models like OpenAI's GPT-4 or Google's LaMDA.
“Our small model is trained to grasp the core principle of language understanding — contextual entailment, while LLMs do not explicitly learn about it," said Hongyin Luo, postdoctoral associate at MIT CSAIL and lead author on the research.
"With a clear goal of learning contextual entailment, the parameter efficiency of our model is much higher than LLMs, thus achieving good performance on NLU tasks," Luo added.
Also Read | Apple’s iPhone maker expects to more than double AI server sales
Lou said the team's next step was, "employing the entailment models in various language-related tasks".
He said that the model only contained 1/500th of the parameters compared to GPT-3, which would make its deployment easier and faster in comparison.
Discover the latest Business News, Sensex, and Nifty updates. Obtain Personal Finance insights, tax queries, and expert opinions on Moneycontrol or download the Moneycontrol App to stay updated!
