Quest 3 Passthrough  + Recording

Guides Quest 3 Passthrough + Recording

Ok, since I shared some Passthrough videos of VAM and Virtual Desktop with the new Meta Quest 3, many people are asking me to write a little guide how to do this. I am bad at explaining and English, but it’s not that difficult so I try 😉

Here’s an example on how it could look like:


<insert video here>

This guide is centered on Quest 3, but it’ll also apply to any headset supported by Virtual Desktop with minor adjustments.

Virtual Desktop:
Have the latest version of Virtual Desktop installed on your Quest (v1.29.0 as of today).

On your Quest, launch VD and go to the Streaming Tab and enable VR Passthrough with settings like this (other can be better, but I had good results with them):

Similarity: 5%
Smoothness: 0%
Color: 98, 120, 100

1699090431276.jpeg
1699090476937.jpeg


Other settings like VR Frame Rate / Bit Rate depend on your PC’s capabilities. I use a bitrate of 200 Mbps and 90fps with the AV1 Codec on my system and get solid 90fps with a simple 1 person atom scene.

Choose the Preferred Codec which fits your computer HW in the Virtual Desktop Streamer App under “Options”. I have a quite beefy PC, so I go for “AV1 10-bit” (only for Quest 3 with Nvidia 4000 and AMD 7000 series GPUs). You might want to test which Codec works best on your PC, it very much depends on the capabilities of your GPU and Wifi quality.

VAM:
- download https://hub.virtamate.com/resources/camera-background-color.24185/

Now you should be ready to go.

1. Launch Virtual Desktop
2. Start VAM via Virtual Desktop
3. Load your scene (prefer empty scene with just your person atoms and lights applied for pure passthrough experience of your models in your real environment or remove CUA’s and such for existing scenes)
4. Add the camera-background as a session or scene plugin. Apply the same background colors as specified in your Virtual Desktop Passthrough settings in the Camera Background plugin (open it's Custom UI for it and type in the RGB values)

1699090644832.png
1699090666145.png


5. Enjoy

Some additional info:
- start VAM from Virtual Desktop with a batch file like this to bypass SteamVR (FPS gain):
"C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "F:\VR\Clean\VaM.exe"
Replace your path to VAM accordingly.

- to record video on Quest headsets (don’t know about others): hold the Oculus button and the right trigger, stop it the same way.
Note: passthrough environment will not work if you have disabled Guradian

- Quest 3 capture settings: in the Experimental Settings, activate Advanced Camera Settings to configure aspect ratio, bitrate and some other tweaks, else you’ll have a low quality square recording
Author
Saint66
Downloads
1,297
Views
42,262
Version
1.1
First release
Last update
Rating
5.00 star(s) 14 ratings

More resources from Saint66

Latest updates

  1. Don't disable Guardian if you want to record Passthrough

    Passthrough will not work if you have disabled Guardian.
  2. Updated some instructions and added screenshots

    Updated some instructions and added screenshots for better understanding

Latest reviews

I wish there was a way the model could be behind your hands, like If I put my hand up in passthrough it is behind them. Like I cant put my hand up in-between myself and the model.. Awesome experience tho.
Upvote 0
Anyone know how to do this 'wired' via usb?
Upvote 0
Fantastic Guide! Thanks!!
Upvote 0
Does this work for oculus rift S?
Saint66
Saint66
Obviously not, since VD (and also ALXR) don’t support the Rift
Upvote 0
OMG. WTF. She's sitting next to me on the couch, chilling... and add voxta and ALM to it... and we are having a conversation.... omg. Quest 3 and Passthru, and these instructions... wow
Upvote 1
Incredible!!
Upvote 0
using your setting since near to one month .... meriting 10 stars.
Upvote 0
Great work and great english (better than mind ;) ! thank you :D
Upvote 0
Nice quick guide, appreciate the assistance! I do find that the suggested color is far to muted and can can match too closely to shaded skin tones unless you use overly bright and warm lighting. I've had the best results with straight 255 green, and an aggressive 40% similarity and 0 smoothness. Tlgair can get a little less clean but essentially makes for a clean outline of the model otherwise.

One thing mentioned in the guide may not be available for everyone right now, and that's the experimental video settings to change aspect ratio. I don't have that setting available. Quest ubdates historically don't roll out to everyone the same day, and sometimes takes like a week. I honestly hate how they do this and it's wild that it's the only devices I've ever owned that really does this.

Maybe op joined the public test channel for updates? I have my quest all default except it's in developer mode.
Saint66
Saint66
Thanks for the review.
About the chroma color: seems realyy varying a lot depending on the scene lighting, the used skin, hair, clothes etc...some people have best results with blue or green or a mixed one. As I said, you haver to experiment.
I'll surely try out some with more aggresive similarity, thx for the heads up!

As for the experimental settings: I am on no PTC version, using v57. They were just there and I could activate them. Strangely, after doing this, they were not present anymore in the experimental features, but I still have the advanced camera options.
Upvote 0
Ok, I got myself a Quest 3 (I also own a Quest 2) and I can only say this. This is nothing short of amazing. Holy shit. If you are on the fence on getting the Quest 3. This is a game changer.
Upvote 0
Back
Top Bottom