1. How to configure servers

The easiest way to stream video on demand (VoD) is to use progressive HTTP streaming which doesn't require any additional or special web server configuration. DeoVR player will download video header and based on this information will send HTTP Range requests to the web server in order to download necessary parts of the video file.

HTTPS is required to ensure compatibility with all Android versions.

Just upload your video files to the public folder of your Web Server and if you encoded your video right it should work without any additional server configuration.

There are 2 downsides of this method:

  1. No seamless quality change based on a user internet speed
  2. Users will be able to download your videos using a direct link.


2. Video encoding basics

There are quite a few different video and audio encodings available but not all of them you can use to play high-resolution VR video. All VR-ready PCs have hardware video decoding acceleration and it is a good idea to make use of it, more about it below. Set “Key Frame Distance” to 1-2 seconds, this will ensure good compromise between file size and smooth seeking.

There are simple rules you can follow to make sure that your customers will be able to watch your videos:

  1. The most supported video codec today is H.264/MPEG-4 AVC, any VR-compatible PC can play it. Use it if you want maximum compatibility, you can use the same video file to stream your videos directly in users browsers. Select AAC audio encoding with a bitrate from 128k to 384k depending on your case and MP4 container format. It has limitations:
    1. Maximum resolution hardware decoders can handle is 4096x4096 @ 30 FPS or 4096x2048 @ 60 FPS, do not exceed these constraints to ensure smooth playback for your users.
    2. It requires more bitrate (internet bandwidth) to get the same visual quality than modern video codecs listed below.
  2. More advanced video codec is H.265/HEVC, it provides the same visual quality at a lower bitrate than H.264 and NVidia GeForce GTX 10 Sires video cards can decode up to 8192x8192 @ 30 FPS with a built-in hardware video decoder and some mobile Exynos SoCs. Use the same audio settings and container format as for H.264 video codec. Limitations:
    1. Browsers do not support this codec this means that you can not use it for direct web-browser streaming.
    2. Windows 7 doesn’t support hardware accelerated video decoding for this codec.
    3. Takes much longer to encode with software video encoder compared to H.264
  3. VP9 is an alternative video encoder from Google, it is very close to H.265 in terms of video quality for the same bitrate and it is open source software. Use Vorbis audio codec and WebM container format. Limitations:
    1. All Apple products don’t officially support it.
    2. @TODO: Need to test hardware decoding/Codecs on Win 7 and Win 10

If you want to ensure that all your users will be able to watch your videos, you should provide several streaming options to them. The absolute minimum would be:

  1. Original H.265 video in the highest resolution and framerate for high-end users.
  2. H.264 4k video option for the maximum compatibility.

3. Intro to the integration

There are two ways to integrate DeoVR into the site.

A. Single video deeplink
A file .json with the description of one video is uploaded on the server. Then a button is added to the site. Once deeplink button is clicked, DeoVR launches and starts the video from this file. The video is played immediately.

B. Multiple videos selection

Called from DeoVR browser
Single .json file containing the list of videos is uploaded on the server. Then, in DeoVR “Internet browser” user calls the site (e.g. www.deovr.com) and gets access to the Selection Scene where they can choose a video from a list.

Called from a deeplink on the site
Single .json file containing the list of videos is uploaded on the server. Then a button with a deeplink to this file (e.g. “deovr://https://www.deovr.com/something.json”) is added to the site. By opening the site in a browser and clicking on the deeplink button, DeoVR launches and shows the user Selection Scene.

You can either implement just one integration, or both in the same time.

IMPORTANT!
DeoVR saves settings for videos internally, title will be the title of the video, but if you want to send multiple videos with same title—use ID to distinct between them.

"title": "ExampleVideo"
"id": 555

1. Resolutions
Create a list of "videoSources". Set the URL and resolution for each video source.

"videoSources":
[
    {
        "resolution": 1080,
        "url": "C:/JsonExampleVideos/ExampleVideo_1080p.mp4"
    },
    {
        "resolution": 1440,
        "url": "C:/JsonExampleVideos/ExampleVideo_1440p.mp4"
    }
]

2. Video preview
You can add a video file which will be used to show the rewind of the file in the player.

"videoThumbnail":"https://yoursite.com/ExampleVideo_SeekLookUp.mp4"

The video should be in a low resolution as well as with low fps in order to save resources. In DeoVR for mobile devices this function may be unavailable.

3. Neccessary thumbnail and preview (optional) in case of playing from Selection Scene
In case of playing this video from the list, it is necessary to add the following lines:

"videoPreview":"https://yoursite.com/ExampleVideo_Preview.mp4"
"thumbnailUrl":"https://yoursite.com/ExampleVideo_Thumbnail.jpg"

The field ‘videoPreview’ contains the link to the video file, which is shown when moving the cursor to this video in the list. This field is not required.

The field ‘thumbnailUrl’ should contain the link to the file with the image shown in the list. This field is required in case of using the list.

4. Timestamps
Create a new list of "timeStamps". Each time stamp has time in seconds and a name value.

"timeStamps":
[
    {
        "ts": 15,
        "name": “Rabbit jumps”
    },
    {
        "ts": 30,
        "name": “Rabbit Sleeps”
    }
]

IMPORTANT!
For correct work of timeStamps you should also specify length of the video in seconds, otherwise unnecessary.

"videoLength": 60

5. Corrections
Declare "corrections" and set horizontal ("x") or vertical ("y") offset [min -7.5, max 7.5], brightness ("br"), contrast ("cont") or saturation ("sat") values [min -70, max 70].

"corrections":
    {
        "x": 5,
        "y": -5,
        "br": -10,
        "cont": 10,
        "sat": 20
    }

6. Other
"stereoMode" can be set to "sbs" for side by side stereoscopic layout, "tb" for top-bottom layout or "off" for monoscopic videos.

"stereoMode": "tb"

"screenType" can be set to "flat" for a 2D quad display inside of VR, "dome" for 180° videos, "fisheye" for fisheye lens videos and "sphere" for 360° videos, “220” for 220° mesh and “250” for 250° mesh.

"screenType": "sphere"

7. Full JSON example

{
  "encodings":[
    {
      "name":"h264",
      "videoSources":[
        {
          "resolution":1080,
          "url":"https://yoursite.com/ExampleVideo_1080p.mp4"
        },
        {
          "resolution":1440,
          "url":"https://yoursite.com/ExampleVideo_1440p.mp4"
        },
        {
          "resolution":1920,
          "url":"https://yoursite.com/ExampleVideo_1920p.mp4"
        },
        {
          "resolution":2160,
          "url":"https://yoursite.com/ExampleVideo_2160p.mp4"
        },
        {
          "resolution":2880,
          "url":"https://yoursite.com/ExampleVideo_2880p.mp4"
        },
        {
          "resolution":3360,
          "url":"https://yoursite.com/ExampleVideo_3360p.mp4"
        },
        {
          "resolution":3840,
          "url":"https://yoursite.com/ExampleVideo_3840p.mp4"
        }
      ]
    }
  ],
  "title":"ExampleVideo",
  "id":123,
  "videoLength":60,
  "is3d":true,
  "screenType":"sphere",
  "stereoMode":"tb",
  "skipIntro":0,
  "videoThumbnail":"https://yoursite.com/ExampleVideo_SeekLookUp.mp4",
  "videoPreview":"https://yoursite.com/ExampleVideo_Preview.mp4",
  "thumbnailUrl":"https://yoursite.com/ExampleVideo_image.jpg",
  "timeStamps":[
    {
      "ts":15,
      "name":"Wall"
    },
    {
      "ts":30,
      "name":"Window"
    },
    {
      "ts":45,
      "name":"Door"
    }
  ],
  "corrections":{
    "x":5,
    "y":-5,
    "br":-10,
    "cont":10,
    "sat":20
  }
}

IMPORTANT!
“is3d” should always be true, in case of false video will be forced monoscopic.

Json file contains the a collection of single videos (see Single Video Deeplink). You can create multiple scenes which will be displayed as the tabs below the interface in Selection Scene.

Json file should contain at least one scene. Each scene corresponds with its own list of videos. An example of list of scenes format:

{
   "scenes":[
      {
         "name":"Trailers",
         "list":[
            … description of video as in single video deeplink ...
         ]
      },
      {
         "name":"Full Videos",
         "list":[
            … description of video as in single video deeplink ...
         ]
      }
   ],
   "authorized":"0"
}

In that case, two scenes named «Trailers» and «Full Videos» will be created.

An example of a full file with one scene and two videos:

{
  "scenes":[
    {
      "name":"Library",
      "list":[
        {
          "encodings":[
            {
              "name":"h264",
              "videoSources":[
                {
                  "resolution":1080,
                  "url":"https://yoursite.com/ExampleVideo_1080p.mp4"
                },
                {
                  "resolution":1440,
                  "url":"https://yoursite.com/ExampleVideo_1440p.mp4"
                }
              ]
            }
          ],
          "title":"ExampleVideo1",
          "screenType":"sphere",
          "stereoMode":"tb",
          "skipIntro":0,
          "videoThumbnail":"https://yoursite.com/ExampleVideo1_SeekLookUp.mp4",
          "videoPreview":"https://yoursite.com/ExampleVideo1_Preview.mp4",
          "thumbnailUrl":"https://yoursite.com/ExampleVideo1_image.jpg",
          "timeStamps":[
            {
              "ts":15,
              "name":"Wall"
            }
          ],
          "corrections":{
            "x":5,
            "y":-5,
            "br":-10,
            "cont":10,
            "sat":20
          },
          "is3d":true,
          "videoLength":60,
          "id":123
        },
        {
          "encodings":[
            {
              "name":"h264",
              "videoSources":[
                {
                  "resolution":1080,
                  "url":"https://yoursite.com/ExampleVideo2_1080p.mp4"
                },
                {
                  "resolution":1440,
                  "url":"https://yoursite.com/ExampleVideo2_1440p.mp4"
                }
              ]
            }
          ],
          "title":"ExampleVideo2",
          "screenType":"sphere",
          "stereoMode":"tb",
          "skipIntro":0,
          "videoThumbnail":"https://yoursite.com/ExampleVideo2_SeekLookUp.mp4",
          "videoPreview":"https://yoursite.com/ExampleVideo2_Preview.mp4",
          "thumbnailUrl":"https://yoursite.com/ExampleVideo2_image.jpg",
          "timeStamps":[
            {
              "ts":15,
              "name":"Wall"
            }
          ],
          "corrections":{
            "x":3,
            "y":-3,
            "br":-5,
            "cont":5,
            "sat":10
          },
          "is3d":true,
          "videoLength":65,
          "id":234
        }
      ]
    }
  ],
  "authorized":"0"
}

If the list of videos is too big, you can use shortened format.
In this case, instead of all fields required for the video to be played, only four fields are used with an addition of the field ‘video_url’, which contains Json to the full description of each video (see Single Video Deeplink).

The required fields of shortened format:

  • thumbnailUrl - the address of the picture of the thumbnail;
  • title – the title of the video;
  • videoLength – the length of the video (in seconds);
  • video_url – the address of Json with the description of the video

Example:

{
  "scenes":[
    {
      "name":"Library",
      "list":[
        {
          "title":"Play with a pretty dog",
          "videoLength":79,
          "thumbnailUrl":"https://deovr.com/s/images/feed/thumb1.png",
          "video_url":"https://deovr.com/deovr/video/id/1"
        },
        {
          "title":"Bikini car wash",
          "videoLength":242,
          "thumbnailUrl":"https://deovr.com/s/images/feed/thumb2.png",
          "video_url":"https://deovr.com/deovr/video/id/2"
        },
        {
          "title":"Date with a girl",
          "videoLength":401,
          "thumbnailUrl":"https:\/\/deovr.com\/s\/images\/feed\/thumb3.png",
          "video_url":"https://deovr.com/deovr/video/id/2"
        }
      ]
    }
  ]
}

Choosing of .json file name for DeoVR
If you want your site to be accessible from DeoVR Browser with the link as «http://www.yoursite.com», put file ‘deovr’ (without extension and quotes) into the root directory of the server.
When following a link containing only domain name, DeoVR will request the data at the address «http://www.yoursite.com/deovr».
In case calling any other URL, e.g. «http://www.yoursite.com/video/test» the request will be made at the same link without changes. The result of the server response should be a .json with a list of video description.

Authorization in DeoVR
In case of transferring login, the result of server response should also contain field “authorized” with the following values:
1 — user is successfully authorized;
0 — user without an account;
-1 — authorization error.
In case of authorization error, DeoVR shows the following message: «Invalid login or password!».
If authorization is not used, the field is not required or its value should equal to 0.

In case of authorization attempt, the current link with .json file (both the list and the video) will be opened through POST request method. Fields ‘login’ and ‘password’ will respective data entered by user. The result of the authorization should be reflected in the field ‘authorized’ of .json described earlier.
In case of using authorization, ‘login’ and ‘password’ will be transferred to the requests of obtaining scenes, as well as to the requests of video description.

6. Images Support

To provide user a feed of images (instead of videos), you must pass an image path (link to the image) in to the "path" variable instead of encoding classes. For StereoMode, ScreenType you can set screenType and stereoMode parameters or specify flags directly in the image file names.

Example:

{
  "scenes":[
    {
      "name":"Library",
      "list":[
        {
          "path":"https://yoursite.com/picture1_tb_360.jpg",
          "title":"ExamplePicture1",
          "screenType":"sphere",
          "stereoMode":"tb",
          "thumbnailUrl":"https://yoursite.com/thumbnail_picture1.jpg",
          "corrections":{
            "x":5,
            "y":-5,
            "br":-10,
            "cont":10,
            "sat":20
          },
          "is3d":true,
          "id":123
        },
        {
          "path":"https://yoursite.com/picture2_sbs_360.jpg",
          "title":"ExamplePicture2",
          "screenType":"sphere",
          "stereoMode":"sbs",
          "thumbnailUrl":"https://yoursite.com/thumbnail_picture2.jpg",
          "is3d":true,
          "videoLength":65,
          "id":234
        }
      ]
    }
  ],
  "authorized":"0"
}

7. Remote control

This guide will demonstrate you how remote control works. You can find a simple remote control client app with a source code here:

https://deovr.com/s/DeoRemoteControlTest.zip

The client is written in C# Windows Forms.

Preparing DeoVR:

  1. Launch DeoVR app
  2. Open settings menu in selection scene or file browser ("gear in right top corner")
  3. Toggle switch "Enable remote control"
  4. On Windows you will get notification from the firewall. Please confirm incoming connections
  5. Start any video from selection scene or file browser.

Preparing Remote Control Client

  1. Unpack and launch DeoRemoteControlTest app
  2. If you started it on the same PC with DeoVR, click “Connect”, otherwise you should specify DeoVR IP address first
  3. After connection established you will get message “Client connected”, after this you will start to receive data from DeoVR app
  4. While you’re in the video, you will receive:
    • Path to the video file
    • Player state (could be play or pause)
    • Current video time
    • And current playback speed
  5. You will be able to send to DeoVR:
    • New video path (new video are going to be opened immediately in DeoVR)
    • SeekTo value (will set new video time)
    • Playback speed (will change current playback speed)
    You can also send multiple parameters in the same packet.

Source code of this app is written in C# and is free to use. RemoteControlClient. cs could be used for comunication in separate program. It implements Connect, Disconnect and Send methods and OnConnected, OnDisconnected and OnDataReceived events. OnDataReceived would be executed each 1 second, when you are in the video player. OnDataReceived and Send works with RemoteApiData class which could be filled and sent or received back from DeoVR. You can check button handlers in TestForm. cs to see how to fills RemoteApiData.

Raw DeoVR remote control protocol description
Remote control client should connect to the device with running DeoVR on TCP port 23554.After the connection has been established, DeoVR will start sending a packet each one second. Remote client also must send a packet (empty or with json) to DeoVR each one secondfor pinging purposes.

Packet structure:

  • Each packet starts with 4-bytes integer value with length of json data represented in UTF8 format.
  • If length is zero, no video data is passed. Empty packets are used for ping purposes.
  • If DeoVR won't receive any type of packet for more then 3 seconds it will close the connection.

Here's a structure of encoded json:

{
    "path":"D:/test.mp3",
    "duration":123.45,
    "currentTime":10.5,
    "playbackSpeed":1.0,
    "playerState":0
}

PlayerState is enumerator: Play = 0, Pause = 1. You can send packet with path, currentTime and playbackSpeed fields. DeoVR will automatically open new file/path, seek to special time or change playback speed. The only one condition, that you have to be inside video player in DeoVR.

8. Spatial Audio

DeoVR adds support for a spatial audio. Final VR video should be provided in .mkv container, with audio encoded in Opus codec. Only tbe8_2 format is supported at the moment. To learn more about creating spatial audio proceed to Spatial Workstation website. Select FB360 Matroshka (experimental) when exporting from Spatial Workstation.

To enable support of spatial audio on your website via DeoVR deeplink scheme add a separate “encodings_spatial” field in your JSON with encodings. Similar to “encodings” specify all other information but instead point to your .mkv files only with the spatial track, important: regular stereo track should be excluded from the file.

If you are playing a file with Path and not Encodings, the file should have the flag “-FB360” or “_FB360”. For local playback Opus track should be encoded as a first track and regular stereo as a second track.

As of April 2021 switching between audio track manually is not yet supported.

{
  "scenes":[
    {
      "name":"Library",
      "list":[
        {
          "encodings":[
            {
              "name":"h264",
              "videoSources":[
                {
                  "resolution":1080,
                  "url":"https://yoursite.com/ExampleVideo_1080p.mp4"
                },
                {
                  "resolution":1440,
                  "url":"https://yoursite.com/ExampleVideo_1440p.mp4"
                }
              ]
            }
          "encodings_spatial":[
            {
              "name":"h264",
              "videoSources":[
                {
                  "resolution":1080,
                  "url":"https://yoursite.com/SpatialExampleVideo_1080p.mkv"
                },
                {
                  "resolution":1440,
                  "url":"https://yoursite.com/SpatialExampleVideo_1440p.mkv"
                }
              ]
            }
          ],
          "title":"ExampleVideo1_Spatial",
          "screenType":"sphere",
          "stereoMode":"tb",
          "skipIntro":0,
          "videoThumbnail":"https://yoursite.com/ExampleVideo1_SeekLookUp.mp4",
          "videoPreview":"https://yoursite.com/ExampleVideo1_Preview.mp4",
          "thumbnailUrl":"https://yoursite.com/ExampleVideo1_image.jpg",
          "timeStamps":[
            {
              "ts":15,
              "name":"Wall"
            }
          ],
          "is3d":true,
          "videoLength":60,
          "id":123
        }
  ],
  "authorized":"0"
}

9. Naming convention

To play local files in the correct mode or add the appropriate flag to the file name. Currently the following flags are supported:

to switch StereoMode:

  • _180
  • _360
  • _fisheye
  • _mkx200
  • _vrca220

to switch Screen Type:

  • "LR" or "3DH" or "SBS"
  • "TB" or "3DV" or "OverUnder"

For example:

Title_SBS_180.mp4 sets Side by Side & 180°
My_video_SBS_mkx200.mp4 sets Side by Side & 200°
Title_FB360_SBS_180.mkv sets Side by Side & 180 & Spatial sound

10. HLS, RTMP, RTSP

From now on you can easily stream from the Zcam K2 Pro camera to Oculus Quest and Quest 2 headsets with the help of Raspberry Pi. This brings the director's monitor view right into VR and you can adjust the scene before you press the record button. You might try recording and streaming at the same time.

Make sure to mount the Raspberry Pi SD image from the description to this video before you start. Use the original charger or appropriate power bank for the Raspberry Pi. Make sure to keep your Raspberry Pi away from routers or other WiFi devices.

Streaming works from a master camera, which shows footage in mono at 5Mbps with approximately 10 seconds latency. The master camera or camera A is the one with a screen and the record button.

Make sure you have DeoVR installed on your Quest headset. 

  1. Turn the camera on.
  2. In the camera menu go to Connect>Network>ETH. Set “Direct”
  3. Turn on your Raspberry Pi. Connect the Raspberry Pi to the K2 Pro camera using the ethernet cable. 
  4. The streaming will start automatically.
  5. Connect your headset to the “RPI4_Camera_GW” WiFi network
  6. In the DeoVR player app go to the browser and switch to http:// (click on https:// to switch it to http://). 
  7. Type 10.0.0.1 (http://10.0.0.1) in the browser, click enter, then click on “Start Stream”.  You will experience a 10 sec delay.
  8. If the camera is upside down, use the player settings to revert it. Go to Settings > Image tab and drag the “Rotation” slider to +/- 180 degrees.
  9. Look at the image in VR and make any final adjustments to the scene.
This is RTSP streaming, the only option of K2 camera.

RTMP support is only available on the Windows platform.
RTSP not supported.
HLS supported.

{
    "scenes": [
        {
            "name": "VR180 Samples",
            "list": [
                {
                    "path": "https://yoursite.com/Example.m3u8",
                    "title": " ExampleStream",
                    "screenType": " sphere",
                    "stereoMode": "sbs",
                    "thumbnailUrl": "https://yoursite.com/ExampleTumb.jpg",
                    "is3d": true,
                    "id": "01"
                }
            ]
        }
    ],
    "authorized": "0"
}

11. Supported videos