ETH Zurich researchers and Max Plank propose HOOD: a new method that leverages graphical neural networks, multi-level message passing, and unsupervised training to enable efficient prediction of realistic clothing dynamics

Telepresence, virtual proofing, video games, and many more applications that rely on high-fidelity digital humans require the ability to simulate attractive and realistic clothing behavior. The use of simulations based on physical laws is a popular method to produce natural dynamic movements. Although physical simulation can provide amazing results, it is expensive to compute, sensitive to initial circumstances, and requires experienced animators; State-of-the-art methods are not designed to meet the rigorous computational budgets required for real-time applications. Techniques based on deep learning are beginning to produce high-quality and efficient results.

However, several restrictions have, until now, prevented these methods from realizing their full potential. First, current techniques calculate garment deformations largely as a function of body posture and are based on linear blend skinning. While skinning-based plans can provide impressive results for form-fitting garments like shirts and sportswear, they need help with dresses, skirts, and other loose-fitting clothing that doesn’t accurately mimic the movement of the body. It is important to note that many learning-based cutting-edge techniques are garment-specific and can only predict deformities for the specific outfit they were clipped into. The application is limited by the requirement to retrain these techniques for each garment.

Researchers from ETH Zurich and the Max Planck Institute for Intelligent Systems in this study provide a unique method for predicting graphical neural networks (GNN) of dynamic garment deformations. Through logical inference about the relationship between local deformations, pressures, and accelerations, his approach learns to anticipate the behavior of physically realistic tissues. His approach generalizes directly to arbitrary body shapes and movements due to their location, independent of the overall structure and shape of the garment. Although GNNs have shown promise in replacing physics-based simulation, applying this idea to clothing simulation produces unsatisfactory results. The feature vectors of a given mesh for the vertices and their single-ring neighborhood are locally transformed by GNN (implemented as MLP).

The messages from each transformation are then used to update the feature vectors. The recurrence of this procedure allows the signals to spread throughout the mesh. However, a predetermined number of message passing stages limits the transmission of the signal to a certain radius. In clothing modeling, where elastic waves caused by stretching flow rapidly through the material, this results in near-global and instantaneous long-range coupling between vertices. There are too few steps, which slows down signal transmission and causes uncomfortable overstretch artifacts, giving garments an unnatural rubbery look. Increased computing time is the price of stupidly increasing iterations.

The fact that the maximum size and resolution of the simulation meshes are unknown a priori, which would allow a conservative and appropriately high number of iterations to be chosen, only exacerbates this problem. They suggest a system for passing messages through a hierarchical network that intersperses propagation phases at various degrees of resolution to solve this problem. This allows effective treatment of fast-moving waves resulting from rigid stretch modes at coarse sizes, while providing the key needed to describe local details, such as folds and wrinkles, at finer scales. Through tests, they demonstrate how their graphical representation improves predictions for comparable IT budgets both qualitatively and quantitatively.

By adopting an incremental potential for the implicit time step as a loss function, they combine the ideas of graph-based neural networks with different simulations to increase the generalizability potential of their method. Due to this formulation, they no longer require ground truth (GT) annotations. This allows your network to be trained completely unattended while simultaneously learning multi-scale clothing dynamics, the influence of material parameters, collision reaction, and frictional contact with the underlying body. The formulation of the graph also allows us to simulate the unbuttoning of a moving shirt and clothing with variable and changing topologies.

Graphical neural networks, multi-level message forwarding, and unsupervised training are combined in his HOOD approach, enabling real-time prediction of realistic clothing dynamics for various clothing styles and body types. They show experimentally that, compared to state-of-the-art methods, their method offers strategic advantages in terms of flexibility and generality. In particular, they show that a single trained network:

1. Effectively predict physically realistic dynamic motion for a wide range of garments.
2. It generalizes to new types of garments and shapes that are not seen during training.
3. Allows runtime changes to material properties and garment sizes.
4. Supports dynamic topology changes like opening zippers or unbuttoning shirts.

The models and code are available for research on GitHub.

review the project page, GitHub linkand Paper. Don’t forget to join our 25k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]

Featured Tools:

🚀 Check out 100 AI tools at AI Tools Club