Google's DeepMind says its AI can tackle Math Olympiad problems

In the AI industry, where comparison between offerings is difficult, solving math problems has become a key proof point.

Google DeepMind, Alphabet Inc.’s artificial intelligence research division, said it has made strides in solving complex math problems, an area that remains challenging for today’s AI programs.

On Thursday, Google rolled out AlphaProof, which specializes in math reasoning, and AlphaGeometry 2, an updated version of a model focused on geometry that the company debuted earlier this year. The programs aced four of the six problems featured in the International Mathematical Olympiad, an annual competition in which students tackle topics such as algebra and geometry, Google said in a blog post.

In the AI industry, where comparison between offerings is difficult, solving math problems has become a key proof point. That’s because large language models, which are trained on vast amounts of written text, tend to be biased towards linguistic rather than mathematical intelligence. While computers are good at numbers and traditional calculations, word-based math problems fall outside of these norms and require more sophisticated reasoning skills.

That’s why Google’s researchers think its AI’s performance at the prestigious math competition is significant. “This is great progress in the field of machine learning and AI,” said Pushmeet Kohli, Google DeepMind’s vice president of research in AI for science, in a press briefing. “No such system had been developed which could solve problems at this success rate.”

While AI tools are becoming more proficient at chatting naturally or producing images, they often struggle with problems that require planning or take multiple steps to solve. But Google and its competitors haven’t given up. The company’s biggest rival, OpenAI, has also been working on new reasoning technology, Bloomberg has reported.

AlphaProof evolved from Google AI programs that have excelled at complex strategy games such as chess, shogi and Go, Google said. A DeepMind program famously beat one of the world’s top Go players in 2016.

Large language models have a tendency to hallucinate, or deliver incorrect information in convincing fashion. Google said it sidestepped that challenge by using its AI to translate math problems into technical statements, or what it called “formal language.”

Another issue for AI systems in mathematics is the lack of available training data, unlike chatbots, which can glean information from vast troves of text online. As Google’s AlphaProof model successfully solves problems, its code is updated, allowing it to tackle ever more difficult challenges, the company said.

The company also released an improved version of its AlphaGeometry AI model, which it said was able to solve 83% of all historical geometry problems included in the International Mathematical Olympiad, spanning the last 25 years.

But Google’s researchers also said that AI is far from being able to replace human mathematicians with its problem-solving capabilities. “Even in the fullest ambition of what we’re trying to do, I think we are aiming to provide a system that can prove anything,” said David Silver, Google DeepMind’s vice president of reinforcement learning. “But that’s not the end of what mathematicians do.”

Silver said DeepMind’s AI models are more akin to slide rules or calculators: powerful computational tools that might one day help humans come up with mathematical proofs. But what the AI systems lack is imagination. “Mathematicians pose interesting problems,” he said.