Runway AI, a company specializing in artificial intelligence for video generation, unveiled a new feature on Tuesday called Act-One. Integrated within the company’s latest Gen-3 Alpha large language model (LLM), Act-One is designed to accurately replicate facial expressions from a source video onto AI-generated characters, addressing a core challenge in video generation technology: converting real individuals into AI avatars while maintaining realistic expressions.
Introduction of Runway’s Act-One Capability in Gen-3 Alpha
In a blog entry, Runway elaborated on the capabilities of this new tool. The Act-One feature allows users to produce both live-action and animated content by using video and voice performances as inputs, ultimately enhancing character expressiveness in AI-generated videos.
Introducing, Act-One. A new way to generate expressive character performances inside Gen-3 Alpha using a single driving video and character image. No motion capture or rigging required.
Learn more about Act-One below.
(1/7) pic.twitter.com/p1Q8lR8K7G
— Runway (@runwayml) October 22, 2024
The emergence of AI-generated videos has significantly transformed the landscape of video content creation, allowing users to generate videos based on natural language text prompts. However, certain limitations still hinder the broader adaptation of this technology. One such limitation has been the inability to control character expressions effectively or enhance performance quality regarding dialogue delivery, gestures, and eye movements.
With the introduction of Act-One, Runway aims to address these issues. Exclusively compatible with the Gen-3 Alpha model, this tool streamlines the facial animation process—traditionally complex and multi-tiered. Current methodologies often require capturing video from multiple perspectives, manual face rigging, and separate recording of facial motion.
Runway claims that Act-One simplifies this process, transitioning it into a two-step operation. Users can record a video of themselves or an actor using a single camera angle—such as a smartphone—and select an AI character. Once these steps are completed, the tool purportedly captures not only facial expressions but also fine details such as eye movements and micro-expressions.
The company emphasized the capabilities of Act-One in its blog post, stating, “The model preserves realistic facial expressions and accurately translates performances into characters, even those with proportions different from the original source video. This versatility opens up new possibilities for innovative character design and animation.”
One of the models strengths is producing cinematic and realistic outputs across a robust number of camera angles and focal lengths. Allowing you generate emotional performances with previously impossible character depth opening new avenues for creative expression.
(4/7) pic.twitter.com/JG1Fvj8OUm
— Runway (@runwayml) October 22, 2024
Importantly, Act-One is capable of generating performances for both animated and live-action characters within cinematic sequences. The tool can effectively capture facial details even if the actor’s facial angle differs from that of the AI character.
This innovative feature is currently being gradually rolled out to users. However, since it requires the Gen-3 Alpha model, users on the free plan will have access to a limited number of tokens to produce videos using this tool.
