VD for passthrough

M

Moos

New member
Messages
1
Reactions
0
Points
1
Now that VD has passthrough option has anyone been able to come up with utilizing that for passthrough in Virtamate? I find ALVR very unstable.
 
The VD update only works for games that have passthrough support coded into them, and because Virtual Desktop is closed source, no one except for the dev can make a green-screen passthrough effect like korejan did with ALXR (alvr fork).
 
dinalpepe said:
The VD update only works for games that have passthrough support coded into them, and because Virtual Desktop is closed source, no one except for the dev can make a green-screen passthrough effect like korejan did with ALXR (alvr fork).
Click to expand...
I am wondering if it would be possible to make VAM plugin that utilize the VD passthrough feature then??
 
It happens the same to me. VD performs perfectly for me but with ALXR there is no way, it looks horrible and the performance is very unstable ;(
 
dinalpepe said:
The VD update only works for games that have passthrough support coded into them, and because Virtual Desktop is closed source, no one except for the dev can make a green-screen passthrough effect like korejan did with ALXR (alvr fork).
Click to expand...

lmfao guys I wish we could do actual passthrough with VD but the passthrough present is just for your home environment. There is no chroma key passthrough, they say they're investigating it but its highly unlikely to be a priority. if you're suggesting that its already possible to use the official meta passthrough api over link with the fake ovr runtime I'm very skeptical but curious. AFAIK that's an older runtime that wouldnt even have those features and in addition there's no actual way to build for passthrough in a release app on PC AFAIK. you can use it in editors but otherwise I don't believe it works in a packaged build. I could be wrong and its only a problem if you distribute the app but I'm unsure.


I'd be interested in hacking together actual openxr for vam if it turned out there was a way to use passthrough over link outside of the unity editor. I know some people were using it with opencomposite and it does work with VD oculus runtime, but unity 2018 wouldnt have support for the updated plugins and therefore probably doesnt have any chance of native passthrough even if we solved the other issues. The only way I could see it working would be to use bepinex and inject vrgn-openxr which is used in koikatsu to inject openxr in unity 5.62 but again idk if that could actually implement passthrough
 
greyfoxut said:
Yeah I just tried the VD Beta 1.29 on my quest 2 and it works even better than ALVR, it has a customisable chroma color and two sliders one for similarity and one for smoothing, I was able to get a really good cut-off with very limited edge color.

It was a little tricky to choose the beta version, first I had to log in to the meta website linked above, choose the beta version in the details, then go onto my headset and uninstall and install virtual desktop, but then it was just the same as usual :)
Click to expand...

The only reason why I never improved the mask mode shader or made it configurable in alxr is because of the lack UI would have made it pointless or very awkward. I also had to prioritize working on other areas with my limited free time.

I'm currently working on adding UI to the client side so the limitations will be gone and the shader rewritten with various options.

After this I'm going to be adding more features. Chroma-keying is not the best or the most efficient way to mask out. Chroma-keying was always a stop gap solution for me, not a be all end all.
 
Last edited:
I have been trying out the new beta version of VD ,with the passthrough and have got to say its a game changer ( for me anyway). No setting up as such, apart from a bit of edging it's perfect, I am sure it will be developed more in the future. Just preordered the Quest 3, so hopefully happy days ;)
 
Just got latest version of VD now and trying out the pass-through with VAM on PICO 4... the default black background in VAM (no light) is not the best key colour since most objects in scene also contain a lot of it and so disappear. How do you change the global background colour in VAM to be a solid blue colour instead of black? I tried skyboxes but they are gradients.

EDIT: Found solution - this... https://hub.virtamate.com/resources/camera-background-color.24185/

I changed the key colour in VD and VAM plugin to be 0 0 255 green, since it seems to work best without causig parts of character middle to also become transparent. Also set Similarity to 30, and left Smoothness to 10. There is still a slight green edge to character but if it s cyberpunk anyway, it just looks part of original character. I didn't turn anti-aliasing off because that also destroys smoothness in middle of char, so all looks much jaggier... prefer no jaggies anywhere and a bit more green on edges. Colour passthrough on PICO 4 is just "good enough" right now - would be better in stereo obviously.

Only thing this cant really do is the hand tracking or interaction with character, as your real hands are on show and end up huge in the passthrough display and behind the character.

WOW this is great - having your favourite virtual character grounded in your room is a game changer :D
 
Last edited:
Don't use pure green, use a more unobtrusive color, just my 2c.

I use VD with these settings to have almost no outline anymore:
Similarity 5, Smoothness 0, colors go to 98, 120, 100
 
atani said:
Just tried it with a Pico 4 and it works great.
Now there's the VD and ALVR options for passthrough 🎉 🥳
Click to expand...
I also use pico 4, I have selected passthrough in the background selection, how to open VD passthrough in VAM, is it also by pressing the menu key +B key?
 
DragonPhantom said:
It doesn't work.o_O
Because Iit needs to open the game with stream？I open the game directly on VD, or what options need to be turned on or off in the VD Settings
Click to expand...
I start it from VD with bypassing Steam with a batch-file like this:
"C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "F:\VR\Clean\VaM.exe"
Just replace the path to your VAM folder.

I changed no other settings in VD, only enabled the passthrough option with the settings above (had to turn down streaming quality from Godlike to High, else you cant enable the passthrough function).
It runs butter smooth on my Quest Pro, btw. Looking forward to see it on the Quest3 next Tuesday ;)
 
Last edited:
On a Pico 4, guess it's the same for others, you just need to activate Passthrough in VD, set a colour, also set the environment as Passthrough.
Running VaM without Steam using a bat file like Saint66 mentions, you need to have in the Session Plugins the plugin from SallyFX, camera something, and try matching the colour code you set in VD. Sometimes this plugin doesn't show anything as passthrough but clicking the spectrum will show it again.
 
Here's some recorded with my new Quest 3 and VD today.
I have to research how to change the recording settings to some higher quality.

But coming from a Quest Pro, I can say passthrough is not perfect but way better on Q3, I can roam my entire house including stairs, read phone and monitor, its a huge improvement.
 

Attachments

  • com.oculus.shellenv-20231010-124606.mp4
    17.1 MB
That AR is so much better than my Pico 4's, I'm jealous. Just having depth perception is a huge difference in usability 😭
 
Saint66 said:
Here's some recorded with my new Quest 3 and VD today.
I have to research how to change the recording settings to some higher quality.

But coming from a Quest Pro, I can say passthrough is not perfect but way better on Q3, I can roam my entire house including stairs, read phone and monitor, its a huge improvement.
Click to expand...
Thanks for sharing this. Gotta say that it's pretty impressive especially compared to quest 2's passthrough ability.
 
Saint66 said:
Here's some recorded with my new Quest 3 and VD today.
I have to research how to change the recording settings to some higher quality.

But coming from a Quest Pro, I can say passthrough is not perfect but way better on Q3, I can roam my entire house including stairs, read phone and monitor, its a huge improvement.
Click to expand...
did you use your bare hand on with the controller when you touched her face in that video ?
 
Saint66 said:
Yes, I had the controllers in my hand
Click to expand...
If you pick up an object inside VaM and then put the controller down and wait for it to go to sleep - once it does, you end up holding the object in your virtual hand anyway even without the controller as I'm guessing the hand tracking gets activated and still works as it goes through
 
You must log in or register to reply here.

Similar threads

Z
Passthrough while loading a scene?
Replies
0
Views
220
zadish2000
Z
P
Running VAM with Passthrough using ALXR
Replies
3
Views
5K
janet67
J
A
How I fixed fatal crashes on every scene load with VD and Quest 2
Replies
1
Views
646
capta134
C
krch
How to Customize ALVR-Client for Quest 2 Passthrough on Windows with Any Mask Color
2
Replies
45
Views
7K
VRCube
VRCube
U
question for creators for vam
Replies
3
Views
400
AWWalker
AWWalker
Back
Top Bottom