Sarvam AI launches Sarvam 1 LLM, trained on 10 Indic languages and English

Apart from Nvidia, the LLM used Yotta’s data centres for compute infrastructure and AI4Bharat’s open-source technology and language resources.

“This combination of high performance and computational efficiency makes Sarvam-1 particularly well-suited for practical applications, including deployment on edge devices,” the AI company wrote in a blog post.

In December 2023, the AI startup launched Open Hathi, India’s first Hindi LLM. The model was built on Meta AI's Llama2-7B architecture, with 48,000 token extensions. On the other hand, Sarvam is developed using a training corpus of 2 trillion tokens.

The LLM has two trillion tokens of synthetic Indic data due to its efficient tokenizer, with the custom data pipeline capable of generating high-quality and diverse text, while maintaining factual accuracy. Sarvam said the latest from its stable matches or beats much larger models like Llama 3.1 8B, apart from being four to six times faster during inference.

In AI, inference refers to the process by which a trained model makes predictions or draws conclusions from new data based on the patterns it learned during training.

Sarvam-2T, the startups’ pretraining corpus, supports two times longer documents, three times higher quality, and eight times more scientific content than existing Indic datasets. Sarvam-2T stores around 2 trillion Indic tokens in total.

The data is almost evenly split between the 10 supported languages, with the exception of Hindi, which comprises about 20 percent of the data.

Also read: India aims to be the AI use-case capital of the world, says Nandan Nilekani