Adobe previews its upcoming text-to-video generative AI tools

Adobe’s new text-to-video and image-to-video AI features will be available in beta later this year. | Image: Adobe

Adobe has teased some of its upcoming generative AI video tools, including a new feature that can produce video clips from still images. This latest preview builds on the in-development Firefly video model that the software giant demonstrated in April, which is set to power AI video and audio editing features across Adobe’s Creative Cloud applications.

The new promotional teaser shows footage produced by Firefly’s text-to-video capabilities that Adobe announced (but didn’t demonstrate) earlier this year. The tool allows users to generate video clips using text descriptions and adjust the results using a variety of “camera controls” that simulate camera angles, motion, and shooting distance. An image-to-video feature was also demonstrated for the Firefly video model that can generate clips using specific reference images. Adobe suggests this could be useful for making additional B-roll footage or to patch gaps in production timelines.

Image: Adobe

Adobe’s new AI video tool will allow users to choose preset filming styles to emulate, alongside describing their desired footage.

If the example footage is any indication of the final release, the generated video quality looks on par with what we’ve seen from OpenAI’s Sora model so far, which Adobe is also “exploring” as a third-party integration for its Premiere Pro video software. Duration is limited, though, with Adobe’s VP of generative AI, Alexandru Costin, telling The Verge that videos produced by the text-to-video and image-to-video features have a maximum length of five seconds.

One advantage Adobe’s own model may have against Sora is its promise that Firefly is “commercially safe” due to being trained on openly licensed, public domain, and Adobe Stock content, which could reduce some concerns about copyright infringement.

GIF: Adobe

Here’s an AI-generated example clip of realistic camera footage produced by Adobe’s Firefly video model.

The text-to-video and image-to-video features will both initially be available in beta as a standalone Firefly application sometime later this year. Adobe says the new Firefly video model will eventually be integrated into its Creative Cloud, Experience Cloud, and Adobe Express applications.

The company also showed off some additional clips of the upcoming “Generative Extend” feature for Premiere Pro that can extend the length of existing video footage, similar to Photoshop’s Generative Expand tool for image backgrounds. Adobe says this will also be arriving on an unspecified date “later this year.”

Posted from: this blog via Microsoft Power Automate.