Nvidia is reducing cost of AI by half every year, says CEO Jensen Huang

Nvidia CEO Jensen Huang at the Nvidia AI Summit in Mumbai.

Global chip-making giant Nvidia is reducing the price of artificial intelligence (AI) tokens—or the cost of using generative AI models—by half or at least by a third each year with its newest graphic processing units (GPUs), founder and CEO Jensen Huang said.

This comes at a time when energy costs and carbon footprint of the quick advancing AI technology has been a major concern for the global technology industry. Nvidia is currently in the process of producing and rolling out its Blackwell AI chips by the end of the fourth quarter.

Huang said at Nvidia AI Summit in Mumbai, that the price of AI tokens decreased by 100X in one year.

“The reason for this is because we've invented so many new algorithms for speeding the processing of inference. Secondly, there are many technologies that have been invented for distillation. You take a very large model, and you use that large model as a teacher for small models. And these small models have essentially the same skills as the large model,” Huang said.

“Not as generalizable, not as flexible, not as agile to be able to do many things, but in certain skills, they can be very, very cost-effective. The combination of those two things, and then the third thing, our company's roadmap is so fast” he added.

“From Ampere to Hopper, Hopper to Blackwell, we're introducing new technology every single year. And every year, we double or triple the performance, which is the same as reducing the cost in half or by a third,” Huang said.

Nvidia is ramping up its commitment to AI in India, announcing strategic partnerships with Indian IT firms and the Reliance Industries to build AI Infrastructure at the Nvidia AI Summit on October 24.

Nvidia CEO Jensen Huang emphasised the company's mission to harness India's IT prowess to develop advanced AI solutions for businesses.

“Our job is to help India build and deploy AI, and your job is to take… (India’s) incredible IT capability so that we can create agents and help every single company,” Huang said while addressing the Summit.

According to media reports last week, Nvidia has also proposed a collaboration with India to co-develop an AI chip. The company is in preliminary discussions with the Indian government to tap into this opportunity. Nvidia will also gain access to India’s strong semiconductor design talent and fast-growing market.

The chip-making giant wants to tap into India’s strong semiconductor design talent and fast-growing market as well. Huang initiated the proposal in a meeting with Prime Minister Narendra Modi in the US earlier this year.