<img src="https://news.mit.edu/sites/default/files/styles/news_article__cover_image__original/public/images/202409/ai-improving-simulations.jpg?itok=Ma8YMY8F” />
Imagine that you are tasked with sending a team of soccer players to a field to evaluate the condition of the grass (a likely task for them, of course). If you choose their positions at random, they may cluster in some areas and completely neglect others. But if you give them a strategy, such as spreading out evenly across the field, you may get a much more accurate picture of the state of the grass.
Now, let's imagine that we need to expand not just in two dimensions, but in tens or even hundreds. That's the challenge facing researchers at MIT's Computer Science and artificial intelligence Laboratory (CSAIL). They have developed an ai-powered approach to “low discrepancy sampling,” a method that improves simulation accuracy by distributing data points more evenly in space.
A key novelty lies in the use of graph neural networks (GNN), which allow points to “communicate” and self-optimize for better uniformity. Their approach marks a fundamental improvement for simulations in fields such as robotics, finance, and computational science, particularly in handling complex, multidimensional problems critical for accurate simulations and numerical calculations.
“In many problems, the more evenly the points can be distributed, the more accurately complex systems can be simulated,” says T. Konstantin Rusch, lead author of the new paper and a postdoc at MIT CSAIL. “We have developed a method called Message-Passing Monte Carlo (MPMC) to generate evenly spaced points, using geometric deep learning techniques. This further allows us to generate points that emphasize dimensions that are particularly important for a problem at hand, a property that is very important in many applications. “The model's underlying graph neural networks allow points to 'talk' to each other, achieving much better uniformity than previous methods.”
His work was published in the September issue of the Proceedings of the National Academy of Sciences.
Take me to Monte Carlo
The idea of Monte Carlo methods is to learn about a system by simulating it with random sampling. Sampling is the selection of a subset of a population to estimate the characteristics of the entire population. Historically, it was already used in the 18th century, when the mathematician Pierre-Simon Laplace used it to estimate the population of France without having to count each individual.
Low-discrepancy sequences, which are sequences with low discrepancy, that is, high uniformity, such as Sobol', Halton and Niederreiter, have long been the gold standard for quasi-random sampling, which trades random sampling with low-discrepancy sampling. discrepancy. They are widely used in fields such as computer graphics and computational finance, for everything from pricing options to risk assessment, where filling spaces evenly with dots can lead to more accurate results.
The MPMC framework suggested by the team transforms random samples into points with high uniformity. This is done by processing the random samples with a GNN that minimizes a specific discrepancy measure.
A big challenge of using ai to generate highly uniform points is that the usual way of measuring point uniformity is very slow to calculate and difficult to work with. To resolve this, the team switched to a faster and more flexible measure of uniformity called L2 discrepancy. For high-dimensional problems, where this method is not sufficient on its own, they use a novel technique that focuses on important projections of lower-dimensional points. This way, they can create point sets that are best suited for specific applications.
The implications extend far beyond academia, the team says. In computational finance, for example, simulations depend heavily on the quality of the sampling points. “With these types of methods, random points are often inefficient, but our GNN-generated low-discrepancy points lead to higher accuracy,” says Rusch. “For example, we considered a classic computational finance problem in 32 dimensions, where our MPMC points outperformed previous state-of-the-art quasi-random sampling methods by a factor of four to 24.”
Robots in Monte Carlo
In robotics, trajectory and motion planning often relies on sampling-based algorithms, which guide robots through decision-making processes in real time. MPMC's improved uniformity could lead to more efficient robotic navigation and real-time adaptations for things like autonomous driving or drone technology. “In fact, in a recent preprint, we showed that our MPMC points achieve four times the improvement over previous low-discrepancy methods when applied to real-world robotic motion planning problems,” says Rusch.
“Traditional low-discrepancy sequences were a breakthrough in their time, but the world has become more complex and the problems we are solving now often exist in 10-, 20-, or even 100-dimensional spaces,” says Daniela Rus, CSAIL. . director and professor of electrical and computer engineering at MIT. “We needed something smarter, something that adapts as dimensionality grows. GNNs are a paradigm shift in the way we generate low discrepancy point sets. Unlike traditional methods, where points are generated independently, GNNs allow points to 'converse' with each other so that the network learns to place points in a way that reduces clustering and gaps, common problems with typical approaches”.
In the future, the team plans to make MPMC points even more accessible to everyone, addressing the current limitation of training a new GNN for each fixed number of points and dimensions.
“Much of applied mathematics uses continuously varying quantities, but calculus typically allows us to use only a finite number of points,” says Art B. Owen, a professor of statistics at Stanford University, who was not involved in the research. “The discrepancy field, which is more than a century old, uses abstract algebra and number theory to define effective sampling points. This article uses graph neural networks to find entry points with low discrepancy compared to a continuous distribution. That approach already comes very close to the best-known low-discrepancy point sets on small problems and shows great promise for a 32-dimensional integral from computational finance. “We can expect this to be the first of many efforts to use neural methods to find good entry points for numerical computation.”
Rusch and Rus co-wrote the paper with University of Waterloo researcher Nathan Kirk, University of Oxford ai DeepMind Professor and former CSAIL affiliate Michael Bronstein, and University of Waterloo Professor of Statistics and Actuarial Sciences , Christiane Lemieux. Their research was supported, in part, by the AI2050 program from Schmidt Futures, Boeing, the United States Air Force Research Laboratory and the United States Air Force artificial intelligence Accelerator, the Swiss National Foundation of Sciences, Natural Sciences and Engineering Research Council of Canada. and a world-leading ai research fellowship EPSRC Turing.
