Yandex's neural network has reached new heights: it is now capable of creating immersive videos in Masterpieceum

On the Russian market, Yandex has become the first company to offer users the technology of creating videos using neural networks. This new feature is available in the Masterpiece app, which generates short videos in response to user requests. Such videos can be used to create music videos, animations, dynamic avatars in social networks and other creative ideas.

The videos generated by Masterpiece are four seconds long and 24 frames per second. Once the videos are created, you can share them with friends or save them in MP4 format. This feature is currently in testing mode. Video generation can be used by active users of Masterpiece, and all users of the app can view the results in the feed.

In order to create a Masterpiece video, you need to describe in text exactly what you want to see. The app will then offer you four variations of the first frame and a set of animation effects that you can use to create motion. The neural network uses your chosen image as a base, generates several modified versions of that image and combines them with the selected effect. Seven effects are now available: zoom (zoom in), timelapse (fast-forward), fly, panorama, rotate, lift, and morph (gradual change).

The cascade diffusion method is used to create the frames of the future video. Using this technology, Masterpiece generates individual images. First, the neural network creates images according to your request, and then gradually increases their resolution by adding details.

Video generation is one of the features of Masterpiece, which complements the creation of images and text posts with a title and illustration. If you want to use content from Masterpiece for commercial purposes, you can submit a request via the feedback form and a response will come within 5 business days.

Masterpiece was introduced by Yandex in April 2023. Immediately after its launch, the app hit the top App Store and Google Play lists in Russia, and currently has over 5.8 million users on iOS and Android. Masterpiece's neural network contains 5 billion parameters and has been trained on 330 million example pairs of pictures with text descriptions.