DeepMind, Google's ai research and development lab, believes the key to more capable ai systems could lie in discovering new ways to solve challenging geometric problems.
To that end, DeepMind today introduced AlphaGeometry, a system that the lab says can solve as many geometry problems as the average International Mathematics Olympiad gold medalist. AlphaGeometry, whose code was open sourced this morning, solves 25 Olympiad geometry problems within the standard time limit, surpassing the 10 of the previous state-of-the-art system.
“Solving Olympiad-level geometry problems is an important milestone in the development of deep mathematical reasoning on the path to more advanced and general ai systems,” write Trieu Trinh and Thang Luong, ai research scientists at Google, in a blog post published this morning. “(We hope) that…AlphaGeometry will help open up new possibilities in mathematics, science and artificial intelligence.”
Why focus on geometry? DeepMind claims that proving mathematical theorems, or logically explaining why a theorem (e.g., the Pythagorean Theorem) is true, requires both reasoning and the ability to choose from a variety of possible steps toward a solution. This problem-solving approach could, if DeepMind is right, one day prove useful in general-purpose ai systems.
“Proving a particular conjecture to be true or false expands the capabilities of even today's most advanced ai systems,” reads DeepMind press materials shared with TechCrunch. “Toward that goal, being able to prove mathematical theorems… is an important milestone as it shows mastery of logical reasoning and the ability to discover new knowledge.”
But training an ai system to solve geometric problems poses unique challenges.
Due to the complexities of translating tests into a format that machines can understand, there is a shortage of usable geometry training data. And many of today's cutting-edge generative ai models, while exceptional at identifying patterns and relationships in data, lack the ability to reason logically using theorems.
DeepMind's solution was two-fold.
Image credits: deep mind
In designing AlphaGeometry, the lab combined a “neural language” model (a model architecturally similar to ChatGPT) with a “symbolic deduction engine,” an engine that leverages rules (e.g., mathematical rules) to infer solutions to problems. Symbolic engines can be inflexible and slow, especially when dealing with large or complicated data sets. But DeepMind mitigated these problems by having the neural model “guide” the deduction engine through possible answers to given geometry problems.
Instead of training data, DeepMind created its own synthetic data, generating 100 million “synthetic theorems” and proofs of varying complexity. The lab then trained AlphaGeometry from scratch on the synthetic data and evaluated it on Olympiad geometry problems.
Olympic geometry problems are based on diagrams to which “constructs” need to be added before they can be solved, such as points, lines or circles. Applied to these problems, AlphaGeometry's neural model predicts which constructs might be useful to add: predictions that AlphaGeometry's symbolic engine uses to make inferences about the diagrams to identify similar solutions.
“With so many examples of how these constructions led to proofs, AlphaGeometry's language model is able to make good suggestions for new constructions when presented with Olympiad geometry problems,” write Trinh and Luong. “One system provides quick, 'intuitive' insights and the other more deliberate and rational decision-making.”
AlphaGeometry problem-solving results, published in a study in the journal Nature This week, they are likely to fuel the long-running debate over whether artificial intelligence systems should be based on symbol manipulation (that is, manipulating symbols that represent knowledge using rules) or on seemingly more brain-like neural networks.
Proponents of the neural network approach argue that intelligent behavior (from speech recognition to image generation) can emerge from nothing more than massive amounts of data and computing. opposite to Symbolic systems, which solve tasks by defining sets of symbol manipulation rules dedicated to particular jobs (such as editing a line in word processing software), neural networks attempt to solve tasks using statistical approaches and learning from examples.
Neural networks are the cornerstone of powerful artificial intelligence systems such as DALL-E 3 and GPT-4 from OpenAI. But, supporters of symbolic ai say, they are not the end all be all; Symbolic ai could be better positioned to efficiently encode knowledge of the world, reason through complex scenarios, and “explain” how they arrived at an answer, these supporters argue.
As a hybrid symbolic-neural network system similar to DeepMind's AlphaFold 2 and AlphaGo, AlphaGeometry perhaps demonstrates that the two approaches (symbol manipulation and neural networks) set is the best path forward in the search for generalizable ai. Maybe.
“Our long-term goal remains to build ai systems that can generalize across mathematical fields, developing the sophisticated problem solving and reasoning that general ai systems will depend on, and at the same time expanding the frontiers of human knowledge,” write Trinh and Luong. “This approach could shape the way ai systems of the future will discover new knowledge, in mathematics and beyond.”
