artificial intelligence is entering almost all industries. Creating natural human movement from a story has the power to completely transform the animation, video game, and film industries. One of the most difficult tasks is Story-to-Motion, which arises when characters must move through different areas and perform certain actions. Based on a comprehensive written description, this task requires a seamless integration between high-level semantic control of motion and low-level control dealing with trajectories.
Although much effort has been put into studying text-to-motion conversion and character control, a suitable solution has not yet been found. Existing character control approaches have many limitations as they cannot handle textual descriptions. Even current text-to-motion conversion approaches need more positional constraints, leading to the generation of unstable motions.
To overcome all these challenges, a team of researchers has introduced a unique approach that is very effective in producing trajectories and generating controlled, infinitely long movements that are in line with the input text. The proposed approach has three main components, which are as follows.
- Text-based motion programming: Modern large language models take a sequence of text, position, and duration pairs from long textual descriptions and use them as text-based motion schedulers. This stage ensures that the moves generated are based on the story and also includes details about the location and duration of each action.
- Text-based motion retrieval system: Motion matching and constraints on trajectories and motion semantics have been combined to create a comprehensive motion retrieval system. This guarantees that the generated movements meet, in addition to the textual description, the expected semantic and positional properties.
- Progressive Mask Transformer: A progressive mask transformer has been designed to address common artifacts in transitional movements, such as foot slippage and unusual postures. This element is essential to improve the quality of generated movements and produce animations with smoother transitions and a more realistic appearance.
The team shared that the approach was tested on three different subtasks: motion matching, temporal action composition, and trajectory tracking. The evaluation has shown improved performance in all areas compared to previous motion synthesis techniques. The researchers have summarized their main contributions as follows.
- Trajectory and semantics have been introduced to generate comprehensive motion from extensive textual descriptions, thus solving the Story-to-Motion problem.
- A new method called Text-based Motion Matching has been suggested, which uses extensive text input to provide accurate and customizable motion synthesis.
- The approach outperforms state-of-the-art techniques in trajectory tracking, temporal action composition, and motion matching subtasks, as demonstrated by experiments performed on benchmark datasets.
In conclusion, the system is definitely a big step forward in the synthesis of human movements from textual narratives. Provides a complete answer to problems associated with Story-to-Motion jobs. It will surely have a revolutionary influence on the animation, gaming and film sectors.
Review the Paper and Project. All credit for this research goes to the researchers of this project. Also, don’t forget to join. our 33k+ ML SubReddit, 41k+ Facebook community, Discord channel, and Electronic newsletterwhere we share the latest news on ai research, interesting ai projects and more.
If you like our work, you’ll love our newsletter.
Tanya Malhotra is a final year student of University of Petroleum and Energy Studies, Dehradun, pursuing BTech in Computer Engineering with specialization in artificial intelligence and Machine Learning.
She is a Data Science enthusiast with good analytical and critical thinking, along with a burning interest in acquiring new skills, leading groups and managing work in an organized manner.
<!– ai CONTENT END 2 –>
