ChatGPT and other deep generative models are proving to be amazing imitators. These ai supermodels can produce poems, finish symphonies, and create new videos and images by automatically learning from millions of examples of previous work. These enormously powerful and versatile tools excel at generating new content that is unlike anything you’ve seen before.
But as MIT engineers say in a new study, similarity is not enough if you really want to innovate in engineering tasks.
“Deep generative models (DGMs) are very promising, but also inherently flawed,” says study author Lyle Regenwetter, a mechanical engineering graduate student at MIT. “The goal of these models is to imitate a set of data. But as engineers and designers, we often don’t want to create a design that already exists.”
He and his colleagues argue that if mechanical engineers want help from ai to generate novel ideas and designs, they will first have to reorient those models beyond “statistical similarity.”
“The performance of many of these models is explicitly tied to how statistically similar a generated sample is to what the model has already seen,” says co-author Faez Ahmed, an assistant professor of mechanical engineering at MIT. “But in design, being different can be important if you want to innovate.”
In their study, Ahmed and Regenwetter reveal the dangers of deep generative models when tasked with solving engineering design problems. In a case study of bicycle frame design, the team shows that these models end up generating new frames that mimic previous designs but fail in performance and engineering requirements.
When researchers presented the same bicycle frame problem to DGMs they designed specifically with engineering-focused goals, rather than just statistical similarity, these models produced more innovative, higher-performing frames.
The team’s results show that similarity-focused ai models don’t quite translate when applied to engineering problems. But, as the researchers also highlight in their study, with careful planning of the appropriate metrics for the task, ai models could be an effective design “co-pilot.”
“It’s about how ai can help engineers create better and faster innovative products,” says Ahmed. “To do that, we first have to understand the requirements. “This is a step in that direction.”
the equipment is new study recently appeared online and will be in the December print issue of the magazine Desing assisted by computer. The research is a collaboration between computer scientists at the MIT-IBM Watson ai Lab and mechanical engineers at MIT’s DeCoDe Lab. Co-authors of the study include Akash Srivastava and Dan Gutreund of the MIT-IBM Watson ai Lab.
Frame a problem
As Ahmed and Regenwetter write, DEMs are “powerful learners with an unrivaled ability” to process enormous amounts of data. DGM is a broad term for any machine learning model that is trained to learn the distribution of data and then use it to generate new, statistically similar content. The hugely popular ChatGPT is a type of deep generative model known as a large language model, or LLM, which incorporates natural language processing capabilities into the model to allow the application to generate realistic images and speech in response to conversational queries. Other popular models for imaging include DALL-E and Stable Diffusion.
Due to their ability to learn from data and generate realistic samples, DGMs have been increasingly applied in multiple engineering domains. Designers have used deep generative models to design new aircraft structures, metamaterial designs, and optimal geometries for bridges and automobiles. But for the most part, the models have imitated existing designs, without improving the performance of existing designs.
“Designers working with DGM are missing this cherry, which is adjusting the model training objective to focus on the design requirements,” says Regenwetter. “So people end up generating designs that are very similar to the data set.”
In the new study, he describes the main obstacles when applying DGMs to engineering tasks and shows that the fundamental objective of standard DGMs does not take into account specific design requirements. To illustrate this, the team invokes a simple case of bicycle frame design and demonstrates that problems can arise already in the initial learning phase. As a model learns from thousands of existing bicycle frames of various sizes and shapes, it might consider two frames of similar dimensions to have similar performance, when in reality a small disconnect in one frame is too small to register as a significant difference. in statistical similarity. metrics: makes the framework much weaker than the other visually similar framework.
Beyond “vanilla”
Credit: Courtesy of the researchers.
The researchers continued with the bicycle example to see what designs a DGM would actually generate after having learned from existing designs. They first tested a conventional “vanilla” generative adversarial network, or GAN, a model that has been widely used in image and text synthesis, and is simply tuned to generate statistically similar content. They trained the model on a data set of thousands of bicycle frames, including commercially manufactured designs and less conventional one-off frames designed by hobbyists.
Once the model learned from the data, the researchers asked it to generate hundreds of new bicycle frames. The model produced realistic designs that resembled existing frames. But none of the designs showed a significant improvement in performance, and some were even slightly inferior, with heavier, less structurally sound frames.
The team then carried out the same test with two other DGMs designed specifically for engineering tasks. The first model is one that Ahmed previously developed to generate high-performance airfoil designs. He built this model to prioritize statistical similarity and functional performance. When applied to the bicycle frame task, this model generated realistic designs that were also lighter and stronger than existing designs. But it also produced physically “invalid” frameworks, with components that didn’t quite fit together or overlapped in physically impossible ways.
“We saw designs that were significantly better than the data set, but also designs that were geometrically incompatible because the model was not focused on meeting the design constraints,” Regenwetter says.
The last model the team tested was one that Regenwetter built to generate new geometric structures. This model was designed with the same priorities as previous models, with the added ingredient of design constraints and prioritizing physically viable frames, for example, without disconnects or overlapping bars. This latter model produced the highest performing designs, which were also physically viable.
“We found that when a model goes beyond statistical similarity, it can generate designs that are better than those that already exist,” says Ahmed. “It’s proof of what ai can do, if explicitly trained on a design task.”
For example, if DGMs can be built with other priorities in mind, such as performance, design constraints, and novelty, Ahmed predicts that “numerous fields of engineering, such as molecular design and civil infrastructure, would benefit greatly. By shedding light on the potential dangers of relying solely on statistical similarity, we hope to inspire new paths and strategies in generative ai applications outside of multimedia.”
