Guide: Passthrough AR videos at DeoVR in 2024

Passthrough AR videos are a game-changing development for VR and DeoVR is at the forefront. Here's everything you need to know about Passthrough, Augmented Reality (AR) and mixed reality VR content in 2024. 

Passthrough AR compatibility: 

• Works best: Quest 3, Quest Pro (stereoscopic color passthrough and the Pico 4 (monoscopic color passthrough)
• Works but not recommended: Quest 2 and Valve Index (monoscopic black and white)
• Not compatible: Oculus Link PCVR or the VIVE XR Elite headset
• Coming soon: Apple Vision Pro

What is Passthrough AR video for virtual reality?

Passthrough is a way to blend the real and virtual worlds using DeoVR. It can also be described as mixed reality or Augmented Reality (AR). It uses the passthrough/AR function on virtual reality headsets to bring VR content into the user’s real-world surroundings. A VR headset with a front-facing camera is required, such as the Quest 3 or Valve Index.

The objective of the passthrough AR experience is to achieve a believable overlap of VR videos with the real world where a user can actively include their very own real live environment, objects, and even themselves into their VR experience. 

Check out the full list of passthrough AR videos on DeoVR.

There are three types of passthrough at DeoVR: regular, alpha packing and AI passthrough. We've explored the impact of each type in various situations on the Quest 3. Find out more in the testing the three different types of passthrough at DeoVR blog. 

Instructions for viewing chroma key Passthrough content with DeoVR: 

Creators can submit the specific passthrough parameters for their passthrough content when uploading (or at a later date). This makes it even easier to view perfect passthrough AR content at the press of a button. If content was uploaded before this date, and not yet updated, follow these instructions: 

1. Find a suitable VR video on DeoVR with with the passthrough tag.
2. Open the video and enter the Player Settings (right-hand side panel). Go to the ‘Passthrough’ section at the bottom. Toggle ‘Enable passthrough’ on.
3. Now adjust the settings to make the background fade to reveal the real-world surroundings while the main action is still visible. (Many uploaders list the best passthrough values in the description of their videos.) 
   • Use Hue, Saturation and Brightness sliders to match the color you want to have transparent
   • Then increase the color range to start removing the color and colors similar to it.
   • Use Falloff to soften the transition from visible to transparent pixels.

DeoVR currently incorporates an easy-to-tweak color wheel picker that allows you to quickly select the target color, and adjust its hue, brightness, saturation, color range, as well as its falloff level. This gives you the best control on the chroma keying adjustments where needed.

What is Alpha Channel Passthrough?

The most recent and advanced version of passthrough is called alpha passthrough and involves the creator embedding an alpha channel inside the video itself.

It doesn't require any preview settings. If the uploaded file name contains 'Alpha,' the passthrough toggle automatically appears on the main panel. Simply click the button to switch between passthrough and non-passthrough. Currently, it only works with fisheye VR video as it packs the alpha channel next to fisheye circles.

The following headsets have been tested to work well with the video passthrough feature of DeoVR:

• Quest 3, 3S, Quest Pro, and Quest 2
• PICO4 and PICO3
• Valve Index, VIVE Pro, and VIVE Pro 2
• Apple Vision Pro support is coming soon

Tips on improving your passthrough experience

• If the transparent edges of the keyed video are slightly colored, try to drop the Brightness value by a couple of notches.
• If your target video is not built for passthrough (not a single-colored background) You can still force a holographic effect on it to create a blend of your video background and the camera passthrough. All you need to do is to reduce the Falloff level until you have an agreeable mixture of a holographic blend. It won't be perfect, but it should give you a believable holographic video experience. You can increase or decrease the Brightness value accordingly to make the video image more prominent than your camera background feed.
• Try to align the video content with your real-world furniture, body parts, or even objects for extra immersion
• Activate the 6DOF and set it to the highest value for X, Y, and Z to 0.40 to make the subject in the video more rooted and not moving too much when you move your head. These are prominent effects that you might encounter when there are close-ups of subjects in a VR video scene.
• If your headset supports hand tracking, activate the DeoVR hand tracking system to virtually touch your virtual video content (Quest, Quest 2, and Quest Pro only). This would give you extra immersion in the sensory touch department. What the eyes see, the mind feels.
• If you have some physical toys that can bring extra sensory stimulation, try it out with your VR video passthrough experience. Just be careful with your real surrounding.
• If you are using color passthrough via the Valve Index, you can help improve the quality of the camera passthrough by making your physical surrounding more brightly lit. The brighter, the better.
• DeoVR will remember each video on its Passthrough settings that you had previously tweaked, so you don't have to memorize any values.

What is Alpha Channel Passthrough? 

DeoVR's expert devs have taken passthrough to the next level with the new alpha channel feature. Put simply, it places green screen/background removal into the post-production level, meaning the final results will look much better. 

How does passthrough look on various VR headsets? 

Watch this useful video to see how passthrough  AR videos look on the most popular VR headsets (go here to try the salsa passthrough video), then see below for his comprehensive look at the world of passthrough, AR and VR. Check the DeoVR documentation for more technical info, including how to create your own Passthrough content. 

See how passthrough the Meta Quest 3 compares to the Meta Quest Pro. 

Important Reminder for SteamVR headset users:

1) One time only: Please ensure that your camera is enabled: SteamVR-> settings-> camera = on and Room view Camera Mode is set to Translucent Passthrough

2) Extra step before enabling passthrough in DeoVR: Please activate the room view when you want to activate the passthrough mode in DeoVR.

You may double-tap the system button on your VR controller or head unit button to quickly toggle room mode off or on. Then follow the step-by-step instruction above.

What next for the passthrough AR video experience?

We at DeoVR are constantly improving the way things work. There will always be a better way to achieve a better immersion and experience. And we are currently working on an improved solution to the current passthrough approach. As mentioned above, the alpha channel has greatly improved this feature but the DeoVR team are always working on exciting new ideas and features. Keep checking back for the latest updates about passthrough and all things VR. 

We’d love to know your thoughts and ideas about this incredible feature. Join the discussion over on our friendly forum!