Children love to draw and express their thoughts and ideas with the help of doodles and pictures. This is how people, from a very young age, portray their emotions along with creativity. Children put their abstract ideas into a drawing, which helps their cognitive development. The field of artificial intelligence has made notable advances after the growing popularity of large language models such as ChatGPT and DALL-E. With a new research paper and model being published almost every day, now comes a new AI model that can turn a doodle into an animation.
Traditional AI models, trained on images of real-life objects, often have difficulty detecting and recognizing an abstract or unrealistic drawing. To overcome the limitations of these AI models, a team of Meta researchers developed a research demo of the AI system that can bring artwork to life through animation.
The doodles are converted to animation in four main steps. In the first step, the system detects the human figure in the photograph of the drawing. In the second step, the system uses a segmentation mask to separate the figure from the background. The third step consists of the system estimating the pose and rigging of the figure, enabling its animation. Finally, in the fourth and final step, the system animates the figure using motion capture data, which is redirected to the character in a unique and engaging way.
The team has developed a dataset of 178,166 annotated hobbyist drawings. This abstract image dataset can help other researchers and AI creators to further innovate. To create this dataset, the researchers launched a cartoon demo in 2021 and invited people to contribute their drawings to the dataset. People could upload images, check or correct annotation predictions, and receive a short animation of their human character within their drawing with the help of the browser-based demo. More than 3.2 million people around the world visited the site and 6.7 million images were uploaded. The images people chose to share with the team were filtered by human reviewers.
The researchers also implemented privacy safeguards to ensure the anonymity of the participants and the quality of the data set. The team shared the animation code for the model and adjusted model weights for drawn human figure detection and pose estimation, which can be accessed here.
Anyone can use the open source code and dataset to extend their analysis methods and augment hobbyist drawings. This can unlock new forms of storytelling and greater accessibility in art. The system could have applications in animation and game development, as well as in educational settings, where it could be used to engage children in creative activities. The system is fast, intuitive and robust and it is definitely a great development in AI to satisfy human creativity.
review the Paper, Github, Project, and Blog. Don’t forget to join our 18k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]
🚀 Check out 100 AI tools at AI Tools Club
Tanya Malhotra is a final year student at the University of Petroleum and Power Studies, Dehradun, studying BTech in Computer Engineering with a specialization in Artificial Intelligence and Machine Learning.
She is a data science enthusiast with good analytical and critical thinking, along with a keen interest in acquiring new skills, leading groups, and managing work in an organized manner.
