Shotcut Tutorial 5: How to make high-resolution foveated 3D VR180 video

8 days ago

Learn how to make high-resolution foveated 3D VR180 video, using the free open-source VR video editing software, Shotcut.

This is a guest post by VR creator XR Stereo Video - most videos on his DeoVR channel are rendered in Shotcut. Check out his other Shotcut tutorials:

How to make high-resolution foveated 3D VR180 video with Shotcut

This is an example video of high-resolution foveated 3D VR180 video created using Shotcut:

Equipment and video required

You will need a fairly powerful computer with a powerful CPU and plenty of RAM (32 GB or more).

You need a 3D VR180 video, from either a standalone 3D VR180 camera or DIY rig.

Important: The left and right videos should have the same field of view. In this video, I had to zoom in on one of the phones to match the FOV of the other. The fact that the FOVs didn’t easily match would make it hard to use such a setup on a constant basis. So you will either want to use two devices that have the same FOV - such as the same model of the same device, with the same lens or a 3D camera. This will make the workflow much faster as you will just add new videos to the timeline in a new Shotcut window and just reuse the filters from the videos in the template window.

How to make foveated video in Shotcut

This involves overlaying two conventional rectangular videos over the centres of the 2 halves of a 3D VR180 video. This guide has a lot in common with the tutorial for adding text but there are some differences.

Make a template

You should start Shotcut in the resolution you want to output or set it in the Settings/ Video Mode menu.

Make this a template first so that we will have correctly set up overlay positioning filters. Then the next time we render a foveated video, we can just copy the filters from one Shotcut window to another, so we don’t need to do it e very time.

Have a short 3D VR180 background video and two left and right foveation videos of similar length prepared, filmed with the devices that you will use for this foveated video and store them in a folder.

First, add the 3D VR180 video to the timeline. Your video should have the standard 2:1 ratio of 3D VR180 video and has to have been through the camera’s included software for conversion to the standard VR format (equirectangular).
Rename the video track to “My 3D VR180 video”, by clicking where it says “V1”.

Create two new video tracks:
You can either use the shortcut Ctrl+I or right-click in the “Output” column of the Timeline, select “Track Operations,” and then “Add video Track.”

Select the 2 new video tracks one by one by clicking in the Output column on the left, in the “Timeline” sub-window.
In the Output column, where it says V2 or V3 click on the text and rename one of the 2 new video tracks to “Left eye” and the other to “Right eye”.

Add your left and right-eye videos to the corresponding tracks.

Morph the videos to equirectangular

Now we want to morph this video correctly for 360/ VR180. This can also be used for adding 2D images/ photos, not just video (in the case of foveated 3D photos you would save a frame, not a video).

Click on the video in the left video track to select it, and then go to the “Filters” tab/ sub-window.
In this “Filters” subwindow/ tab, click on the “+” sign to add a new filter to the list, search “360” in the Search box and select “360: Rectilinear to Equirectangular”.

Now the video will have a more curved shape and it will be smaller in size.

Overlay the videos on their proper positions

You have the options “Horizontal” and “Vertical” with which you set the field of view for your video in degrees. You will want to match the FOV of your devices and have proper overlay on the 3D VR180 video, so adjust the FOV so that you get your best overlay.

Move this video to the center of the left eye in the 3D VR180 video.
Click on the “+” sign in the Filters tab/ subwindow and add the “Size, Position & Rotate” filter

Unlike in the tutorial for adding text, here we will have to rely less on pre-established numbers and overlap the videos manually so that the content is overlaid as close as possible to pixel level. We will basically want to use a sort of pixel mapping for future use, when we will overlay other videos.

We will want to use the eye-shaped button on the left side of the video tracks to hide and reveal the overlayed video to see the accuracy of the overlay as we fine-tune the position settings.

WE WILL ENTER SOME APPROXIMATE NUMBERS BUT THEN WE WILL HAVE TO MATCH THE PIXELS MANUALLY.

Set position to MINUS one-quarter of your video width comma zero, in this case -2048 , 0 (then manually adjust it)
Set Size to the resolution of the output video, in this case 8192 x 4096 (then manually adjust it)
For now, set the next parameters like this:
Zoom at 100% (then manually adjust it)
Size mode “Fill”
Horizontal fit “Center”
Vertical fit “Middle” (then manually adjust it)
Rotation 0.0 degrees

Move the right eye video into the correct position and fine-tune the overlay.
Go to the “Left eye” track, select the video by clicking on it, then go to the “Filters” tab/sub window and press on the “Copy” icon to copy the filters and their settings.
Next, select the right eye video in the “Right eye” track, go to the “Filters” tab/sub window and press on the “paste” icon to paste the filters and their settings into the “right eye” video. This will paste all the filters with the settings that we’ve applied but the video will still show into only 1 eye because we haven’t moved it yet to the other eye.
Select the text clip in the “Right eye” track.
Now, let’s move the text for the right eye in the right eye.
Go to the “Size position & Rotate” filter and delete the MINUS sign at Position, so turn your negative number into a positive one, in this case turn -2048 into 2048 (then manually adjust it). Now the text will appear for the right eye as well.

Align/ sync/ genlock the videos (optional)

You will want to sync the videos preferably with a precision of one frame. You can use Shotcut’s zoom button in the preview window to zoom in on the pixels. You can drag the videos to realign them in the Timeline or you can move the cursor and “snap” them to the cursor by having the “Snap” icon that looks like an inverted magnet activated. Trim the ends of the videos if necessary so that all the videos on all the tracks have an equal length.

If any video still doesn’t display in the other eye, SAVE your project as an “mlt” file and then reload it.
Now render the full foveated video by going to the “Export” subwindow/ tab. After selecting your encoding settings by pressing the “Advanced” button, click on the “Export File” button to render your video.
Check in the headset that everything is OK about the video.
Save the project as an “mlt” file for this particular video so that you save your current project.

Save the foveated video template

Now let’s save the template for future usage on another video so that we don’t do all this work all over again and every time for a similar result.

Save this template in a place you’ll remember, to be able to easily use it the next time by just copying the filters from the existing videos to videos in the new video. Go to the menu on the top, select “File” then “Save as” and save the project as a *.mlt file, for example “3D VR180 foveated video template.mlt”.

While in this template the genlock isn’t very important because it’s just a template, a sort of pixel mapping/ overlay, when shooting future videos intended for use in foveated video, you will want to start and stop all the recordings as close to simultaneously as possible, in order to easily sync the videos.

The next time, open the template in one Shotcut instance window, then open Shotcut again in another instance window, make 3 tracks there as well, add and align 3 videos in similar fashion and then click on the video of each track one by one, go to the template Shotcut instance window to the Filters tab/ sub-window and press the “copy” button to copy the filters. It asks whether to copy to clipboard or as a preset. It works either way, now head over to the other Shotcut instance, click on the corresponding video track, go to the Filters tab/ sub-window and paste to paste the filters with their settings.

The positioning of the overlay should be right if you did everything right but you might still have to fine tune the exposure with the settings in the “Color grading” filter, the white balance or what ever else there might be.

Be aware that you can also copy and paste the filters several times on the same track as well.

Now you can have foveated high resolution video in VR, with cheap devices and with free editing in Shotcut!

Join the discussion at the DeoVR forum, Facebook and Reddit.