In fields such as physics and engineering, partial differential equations (PDEs) are used to model complex physical processes and generate information about how some of the world's most complicated physical and natural systems work.
To solve these difficult equations, researchers use high-fidelity numerical solvers, which can be time-consuming and computationally expensive to execute. Current simplified, data-driven alternative models calculate the target property of a solution for PDEs rather than the full solution. These are trained on a set of data generated by the high-fidelity solver, to predict the output of the PDEs for new inputs. This is data-intensive and expensive because complex physical systems require a large number of simulations to generate enough data.
In a new article, “Physics-Enhanced Deep Substitutes for Partial Differential Equations”, published in December in Nature Machine Intelligencea new method is proposed to develop data-driven surrogate models for complex physical systems in fields such as mechanics, optics, thermal transport, fluid dynamics, physical chemistry, and climate models.
The article was written by MIT applied mathematics professor. Steven G. Johnson With Payel Das and Youssef Mroueh from the MIT-IBM Watson ai Lab and IBM Research; Chris Rackauckas of Julia Laboratory; and Rafael Pestourie, a former MIT postdoc now at Georgia tech. The authors call their method “physics-enhanced deep surrogate” (PEDS), which combines a low-fidelity explainable physics simulator with a neural network generator. The neural network generator is trained end-to-end to match the output of the high-fidelity numerical solver.
“My aspiration is to replace the inefficient process of trial and error with systematic computer-aided simulation and optimization,” Pestourie says. “Recent advances in ai, such as ChatGPT's large language model, rely on hundreds of billions of parameters and require large amounts of resources to train and evaluate them. “In contrast, PEDS is affordable for everyone because it is incredibly compute-efficient and has a very low barrier in terms of the infrastructure needed to use it.”
In the paper, they show that PEDS surrogates can be up to three times more accurate than an ensemble of feedforward neural networks with limited data (about 1000 training points) and reduce the training data needed by at least a factor of 100 to achieve a target error of 5 percent. Developed using MIT design julia programming languageThis scientific method of machine learning is efficient in both computing and data.
The authors also report that PEDS provides a general data-driven strategy for bridging the gap between a wide range of simplified physical models with corresponding brute force numerical solvers that model complex systems. This technique offers precision, speed, data efficiency and physical information about the process.
Says Pestourie: “Since the 2000s, as computing capabilities have improved, the trend in scientific models has been to increase the number of parameters to better fit the data, sometimes at the cost of lower predictive accuracy. PEDS does the opposite, intelligently choosing its parameters. It leverages automatic differentiation technology to train a neural network that makes a model with few parameters accurate.”
“The main challenge preventing surrogate models from becoming more widely used in engineering is the curse of dimensionality: the fact that the data needed to train a model increases exponentially with the number of variables in the model,” Pestourie says. “PEDS reduces this curse by incorporating insights from data and field knowledge in the form of a low-fidelity model solver.”
The researchers say PEDS has the potential to revive an entire body of pre-2000 literature devoted to minimal models: intuitive models that PEDS could make more accurate while still being predictive for surrogate model applications.
“The application of the PEDS framework goes beyond what we show in this study,” says Das. “Complex physical systems governed by PDEs are ubiquitous, from climate modeling to seismic modeling and beyond. Our fast and explainable, physics-inspired surrogate models will be of great use in those applications and play a complementary role to other techniques.” emerging, such as foundation models”.
The research was supported by the MIT-IBM Watson ai Laboratory and the US Army Research Office through the Soldier Nanotechnologies Institute.
