Meta open-sources Llama 2, but with strings attached

The intense AI wars in the tech industry this year have seen giants catch up to and compete with existing large language models (LLMs). And Facebook parent Meta is no different.

The social media giant is open-sourcing its Llama 2 (Large Language Model Meta AI) in direct competition with OpenAI's GPT models. However, the open-sourcing comes with strings attached, primarily to prevent its competitors from benefiting from it.

The race to challenge OpenAI's popular AI chatbot, ChatGPT, has been fierce since its launch in November last year. In February 2023, both Microsoft and Google announced their own AI chatbots. Meta responded with the initial release of a smaller version of LLaMA, which was initially limited to researchers.

Now, with the introduction of Llama 2 and its open-sourcing, Meta aims to level the playing field and invites developers and businesses to build commercial products on top of the model, in an attempt to play catch up and compete with existing players.

The latest version of Llama 2 will be distributed through Microsoft's Azure cloud service, compatible with the Windows operating system. Interestingly, Meta referred to Microsoft as their "preferred partner" for this release.

Also Read | Explained: What powers ChatGPT and Bard? A look at LLMs or large language models

“We believe an open approach is the right one for the development of today’s AI models, especially those in the generative space where the technology is rapidly advancing. By making AI models available openly, they can benefit everyone,” Meta said in a statement.

An open-source model is a tried-and-tested method for tech giants to catch up with each other, but Meta's comes with a catch.

Although Meta has announced that it will open-source its LLM and allow free usage for commercial and research purposes, a closer examination of its community license agreement reveals a different story.

According to Meta's commercial terms, companies with 700 million or more monthly active users (MAU) are required to obtain a license from Meta. This means that Meta's new AI technology is off-limits to some of its social media competitors.

Additionally, another clause in the commercial terms of Llama 2 states that users are prohibited from utilising it to enhance or improve other large language models, aside from Llama 2 itself.

Meta has opened access to its AI model, hoping that a community of developers and researchers will stress test it, identify and solve problems quickly. Meta believes that this approach will help to improve the model and make it more secure.

"By seeing how these tools are used by others, our own teams can learn from them, improve those tools, and fix vulnerabilities," it said.

This is not the first time that a company has open-sourced technology in an effort to catch up to rivals. For example, Google open-sourced its Android mobile operating system in 2007, in an effort to compete with Apple's iPhone. Despite the iPhone's early advantage, Android eventually emerged as the dominant software used in smartphones.

Meta's decision to open access to its AI model aligns with a broader trend in the industry. Abu Dhabi-based Technology Innovation Institute recently released Falcon LLM, making its code freely available this year. Additionally, Mosaic ML, which was acquired by Databricks for approximately $1.3 billion last month, also provides open-source software for training LLMs.

The initial version of LLaMA already proved to be competitive with OpenAI's ChatGPT and Google's Bard chatbot models. However, the latest version of the LLM offers various model sizes, including 7 billion, 13 billion, and 70 billion parameters. In comparison, OpenAI's GPT-3.5 boasts of an impressive 175 billion parameters, while Google's LaMDA 2 is trained on 540 billion parameters.

Parameters in LLMs represent the accumulated knowledge during the model's training phase. More parameters generally lead to more accurate predictions because the model has access to a greater amount of contextual information.

According to Meta, the pre-trained LLM models were trained on a significantly larger dataset, around 40 percent bigger than the one used for LLaMA 1. Additionally, Llama-2-chat models have been trained on over 1 million new human annotations.

Microsoft has forged a notable alliance with OpenAI, the creator of ChatGPT, leveraging this partnership to introduce various products like Bing Chat and Copilot, which are built on the Sam Alman-led AI startup’s technology.

While Microsoft has made a significant investment (over $10 billion) in OpenAI, embracing AI as a strategic focus, the company is signalling that it is not exclusively reliant on OpenAI for advancing this emerging technology.

"Meta and Microsoft share a commitment to democratising AI and its benefits and we are excited that Meta is taking an open approach with Llama 2. We offer developers choice in the types of models they build on, supporting open and frontier models...," the Redmond, Washington-based tech giant said.

As part of this collaboration with Meta, Microsoft will now make Llama 2 accessible through Azure AI and Windows. Additionally, Llama 2 will be available through other providers such as Amazon Web Services and Hugging Face.