Meta will launch Meta Movie Gen

Meta will launch Meta Movie Gen, an AI-generation multimedia tool, offering creators tools to generate, customize, and edit videos and produce audio, all in high quality and with text descriptions. Specifically, Meta Movie Gen is the third generation of Meta's foundational models, which the company targets content creators and filmmakers as a tool to help drive their creativity rather than replace it.

As the company explains on its official blog, the models it includes have been trained with licensed and other publicly available datasets to offer four capabilities: video generation, custom video generation, precise video editing, and audio generation.

Regarding video generation, the model uses a 30 billion parameter transformer to produce high-quality videos of up to 16 seconds at a rate of 16 frames per second from a text description.

As Meta notes, “the model can reason about object motion, subject-object interactions, and camera motion, and can learn plausible motions for a wide variety of concepts.” Meta Movie Gen models also generate personalized videos based on an image of a person and a description. This is coupled with the ability to edit with localized or more general actions and accurate results.

Meta Movie wants to be a multimedia ‘total solution’ 

Meta Movie Gen also generates audio; it is Meta’s vision to turn this into a full multimedia ‘total solution’. Specifically, it uses a 13 billion parameter model that produces high-quality video from either a base video or a text description. The output is up to 45 seconds long and includes ambient sound, sound effects, and instrumental background music. This model can also generate coherent audio for videos of arbitrary lengths.

In addition, the company stresses that as they continue to improve their models and move closer to a possible future release, “we will work closely with filmmakers and creators to integrate their feedback.” “By taking a collaborative approach, we want to make sure we create tools that help people enhance their inherent creativity in new ways they never dreamed possible. Imagine animating a 'day in the life' video to share on Reels and editing it using text messaging, or creating a personalized animated birthday greeting for a friend and sending it to them via WhatsApp,” adds Meta.