Can you imagine seeing yourself on your favorite show even though you have never acted in real life? Or is it possible to change the beginning and end as you like?
Fable Studios, a San Francisco startup, has just released its SHOW-1 AI technology that can write, produce, direct, animate, and even voice entirely new episodes of TV shows. Fable Studios did this using different broadcast models. These work on a simple method of adding and removing random noise from the data over time that can generate and reconstruct the output. You can start with an image as random noise and gradually transform it into the required output.
Fable Studios trained their broadcast models using a data set comprising 1,200 characters and 600 background images from the television show South Park. Your first model task was to generate individual characters against a background color. Autonomous characters can be generated in the show based on each one’s characteristic appearance, writing style, and voice. Character diffusion models allow you to create South Park characters based on your own look through stable image-to-image diffusion.
The task of the second model was to generate a clean background that can act as a stage to allow the characters to interact, thus allowing multiple scenes and settings to be designed. The only limitation of this model was that they produced low resolution images. The team addressed this using AI scaling techniques that improved image quality. Produce vector-based graphics as they do not lose their resolution when scaling is changed.
Fable Studios redefined an episode of the TV show by changing the dialogue sequence in specific places and the running time to match the episode’s original length. Using simulation data as a chain of cues, they created a story system that runs parallel to the showrunner’s system to monitor the sequence of actions and dialogue to keep the audience engaged. Each character’s voice has been cloned beforehand, and voice clips are generated for each new dialogue.
The data produced by the simulation acts as a creative dictionary for both the individual writing the initial message and the generating story system. It’s common for even seasoned story writers to get bogged down when writing dialogue; these issues can be overcome as the simulation provides context and data points before starting the prompt chain.
The story generation process is shared in proportions between the user, the simulation and GPT-4. The simulation produces a fundamental context such as history, emotions and events for the characters. It acts as an initial creative context. GPT 4 acts as the main generative engine, fusing cutscenes and dialogue based on feedback from the user and from the simulation.
Finally, the integration of the strengths of the simulation, the user and the AI model creates a richer and more interactive storytelling experience. On the contrary, personalizing the shows will mean the loss of jobs. With AI-powered tools now being able to perform tasks like video editing and music composition that were once done by human professionals, this will raise concerns about the future of jobs in the entertainment industry.
review the Project and Reddit post. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 26k+ ML SubReddit, discord channel, and electronic newsletterwhere we share the latest AI research news, exciting AI projects, and more.
🚀 Check out over 900 AI tools at AI Tools Club
Arshad is an intern at MarktechPost. He is currently pursuing his Int. Physics Master’s degree from the Indian Institute of Technology, Kharagpur. Understanding things down to the fundamental level leads to new discoveries that lead to the advancement of technology. He is passionate about understanding nature fundamentally with the help of tools like mathematical models, ML models, and AI.
