PassthroughKey

Plugins + Scripts PassthroughKey

Download [<1 MB]
The User Guide (top of page 1) now covers v2.0
 
Last edited:
Thanks, that log helps. Two things are still changing the image after PassthroughKey:
  • VamDlssNr (NR) is still active on the VR camera and runs after PassthroughKey’s last pass. Turning it off in its menu apparently doesn’t remove it. Disable the plugin itself and restart VaM.
  • ClearUpscale renders the character and background with its own cameras and builds the final image, which replaces the key background. Turn that off too.
With both off, the background should key.
Advanced > Test pattern: the bottom row should be see-through.

Making PassthroughKey work together with DLSS / NR upscalers is on my list.
Thanks, removed the plugin . It' still doesn't work with the correct settings (As far as I can tell). Test pattern bottom row shows a grayscale and no see through. Here's the new log, key pass seems to be correctly at the end of the chain now though:


FPSLimiter initialized with Unity target FPS and VR frame pacing enabled.
PassthroughKey: hair & cloth capsule floor added at the real floor.
PassthroughKey: floor shadows on (no PT main light - straight down, floor y 0.00)
PassthroughKey: body floor collider on layer 12 'Tongue' - collides with 6/8 body collider layers.
PassthroughKey: cameras (name / depth / clear / UI layer / stereo):
'ScreenCameraClear' depth -1 clear Color UI False eye None
'CenterEyeAnchor' depth 0 clear Color UI True eye Both <- protect pass
'ScreenUICamera' depth 10 clear Nothing UI True eye None
'PassthroughKey_UnderFloorCam' depth -101 clear Color UI False eye None (renders to texture)
'PassthroughKey_ShadowCam' depth -100 clear Color UI False eye None (renders to texture)
(disabled/manual) 'ClearUpscale_CharCamera' depth 2 clear Color UI False eye Both
(disabled/manual) 'ClearUpscale_BgCamera' depth 1 clear Color UI True eye Both
(disabled/manual) 'RightHandAnchor' depth 0 clear Nothing UI False eye None
(disabled/manual) 'LeftHandAnchor' depth 0 clear Nothing UI False eye None
(disabled/manual) 'RightEyeAnchor' depth 0 clear Color UI True eye Right
(disabled/manual) 'LeftEyeAnchor' depth 0 clear Color UI True eye Left
(disabled/manual) 'HiResScreenshotCamera' depth 0 clear Color UI True eye None
(disabled/manual) 'LoResScreenshotCamera' depth 0 clear Color UI True eye None
VR: True key pass: active
PassthroughKey: components on 'CenterEyeAnchor' (render order, hdr False, colour space Linear):
0. UnityEngine.Transform
1. UnityEngine.AudioListener
2. UnityEngine.Camera
3. MeshVR.PerfMonCamera
4. MK.Glow.MKGlow (disabled)
5. Leap.Unity.LeapXRServiceProvider (disabled)
6. LeapMotionProviderSwitcher
7. PassthroughKeyOpaquePass
8. PassthroughKeyPass
PassthroughKey: RIG CHAIN (head camera up to the root; local pos / world y):
0. 'CenterEyeAnchor' local (-0.075, 1.795, -0.094) world y 1.619 scale 1
1. 'TrackingSpace' local (0.000, 0.000, 0.000) world y -0.176 scale 1
2. 'OVRRig' local (0.000, 0.000, 0.000) world y -0.176 scale 1
3. 'HeightOffset' local (0.000, -0.176, 0.000) world y -0.176 scale 1
4. '[CameraRig]' local (0.000, 0.000, 0.000) world y 0.000 scale 1
5. 'NavRig' local (0.000, 0.000, 0.000) world y 0.000 scale 1
6. 'WorldScaleAdjust' local (0.000, 0.000, 0.000) world y 0.000 scale 1
7. 'CoreControl' local (0.000, 0.000, 1.500) world y 0.000 scale 1
8. 'SceneAtoms' local (0.000, 0.000, 0.000) world y 0.000 scale 1
navigationRig '[CameraRig]' world y 0.000
PassthroughKey: TRACKED under 'TrackingSpace' (world y / height above play-space origin):
'LeftEyeAnchor' world y 1.618 above origin 1.794
'CenterEyeAnchor' world y 1.619 above origin 1.795
'CenterEye' world y 1.619 above origin 1.795
'HUDReferenceDown' world y 1.209 above origin 1.385
'Capsule' world y 1.615 above origin 1.791
'Sphere1' world y 1.621 above origin 1.797
'Sphere2' world y 1.617 above origin 1.793
'HeadPossessPoint' world y 1.619 above origin 1.795
'AltHUDAttachPoint' world y 1.634 above origin 1.811
'ThoughtBubble' world y 1.760 above origin 1.936
'ClearUpscale_BgCamera' world y 1.619 above origin 1.795
'ClearUpscale_CharCamera' world y 1.619 above origin 1.795
'RightEyeAnchor' world y 1.621 above origin 1.797
'TrackerAnchor' world y -0.176 above origin 0.000
'RightHandAnchor' world y 1.232 above origin 1.408
'RightHandPossessPoint' world y 1.193 above origin 1.369
'RightHand' world y 1.232 above origin 1.408
Scene height offset 0.000, floor match False
 
Last edited:
Thanks, removed the plugin . It' still doesn't work with the correct settings (As far as I can tell). Test pattern bottom row shows a grayscale and no see through. Here's the new log, key pass seems to be correctly at the end of the chain now though:


FPSLimiter initialized with Unity target FPS and VR frame pacing enabled.
PassthroughKey: hair & cloth capsule floor added at the real floor.
PassthroughKey: floor shadows on (no PT main light - straight down, floor y 0.00)
PassthroughKey: body floor collider on layer 12 'Tongue' - collides with 6/8 body collider layers.
PassthroughKey: cameras (name / depth / clear / UI layer / stereo):
'ScreenCameraClear' depth -1 clear Color UI False eye None
'CenterEyeAnchor' depth 0 clear Color UI True eye Both <- protect pass
'ScreenUICamera' depth 10 clear Nothing UI True eye None
'PassthroughKey_UnderFloorCam' depth -101 clear Color UI False eye None (renders to texture)
'PassthroughKey_ShadowCam' depth -100 clear Color UI False eye None (renders to texture)
(disabled/manual) 'ClearUpscale_CharCamera' depth 2 clear Color UI False eye Both
(disabled/manual) 'ClearUpscale_BgCamera' depth 1 clear Color UI True eye Both
(disabled/manual) 'RightHandAnchor' depth 0 clear Nothing UI False eye None
(disabled/manual) 'LeftHandAnchor' depth 0 clear Nothing UI False eye None
(disabled/manual) 'RightEyeAnchor' depth 0 clear Color UI True eye Right
(disabled/manual) 'LeftEyeAnchor' depth 0 clear Color UI True eye Left
(disabled/manual) 'HiResScreenshotCamera' depth 0 clear Color UI True eye None
(disabled/manual) 'LoResScreenshotCamera' depth 0 clear Color UI True eye None
VR: True key pass: active
PassthroughKey: components on 'CenterEyeAnchor' (render order, hdr False, colour space Linear):
0. UnityEngine.Transform
1. UnityEngine.AudioListener
2. UnityEngine.Camera
3. MeshVR.PerfMonCamera
4. MK.Glow.MKGlow (disabled)
5. Leap.Unity.LeapXRServiceProvider (disabled)
6. LeapMotionProviderSwitcher
7. PassthroughKeyOpaquePass
8. PassthroughKeyPass
PassthroughKey: RIG CHAIN (head camera up to the root; local pos / world y):
0. 'CenterEyeAnchor' local (-0.075, 1.795, -0.094) world y 1.619 scale 1
1. 'TrackingSpace' local (0.000, 0.000, 0.000) world y -0.176 scale 1
2. 'OVRRig' local (0.000, 0.000, 0.000) world y -0.176 scale 1
3. 'HeightOffset' local (0.000, -0.176, 0.000) world y -0.176 scale 1
4. '[CameraRig]' local (0.000, 0.000, 0.000) world y 0.000 scale 1
5. 'NavRig' local (0.000, 0.000, 0.000) world y 0.000 scale 1
6. 'WorldScaleAdjust' local (0.000, 0.000, 0.000) world y 0.000 scale 1
7. 'CoreControl' local (0.000, 0.000, 1.500) world y 0.000 scale 1
8. 'SceneAtoms' local (0.000, 0.000, 0.000) world y 0.000 scale 1
navigationRig '[CameraRig]' world y 0.000
PassthroughKey: TRACKED under 'TrackingSpace' (world y / height above play-space origin):
'LeftEyeAnchor' world y 1.618 above origin 1.794
'CenterEyeAnchor' world y 1.619 above origin 1.795
'CenterEye' world y 1.619 above origin 1.795
'HUDReferenceDown' world y 1.209 above origin 1.385
'Capsule' world y 1.615 above origin 1.791
'Sphere1' world y 1.621 above origin 1.797
'Sphere2' world y 1.617 above origin 1.793
'HeadPossessPoint' world y 1.619 above origin 1.795
'AltHUDAttachPoint' world y 1.634 above origin 1.811
'ThoughtBubble' world y 1.760 above origin 1.936
'ClearUpscale_BgCamera' world y 1.619 above origin 1.795
'ClearUpscale_CharCamera' world y 1.619 above origin 1.795
'RightEyeAnchor' world y 1.621 above origin 1.797
'TrackerAnchor' world y -0.176 above origin 0.000
'RightHandAnchor' world y 1.232 above origin 1.408
'RightHandPossessPoint' world y 1.193 above origin 1.369
'RightHand' world y 1.232 above origin 1.408
Scene height offset 0.000, floor match False

Thanks, the key pass is last now, good.
The solid bottom row in the test pattern is actually the good result: it means VD's image settings are right. See-through there would mean a problem.

Two checks:
ClearUpscale is still loaded. Its cameras are still in your scene. It's part of the same DLSS package or a separate plugin. Remove that as well and restart VaM.

Quick isolation test: Advanced → untick Use depth (off = key everything). The whole picture becomes the key colour.

If you now see your room everywhere, VD is fine and something in the scene is still in the way. Send me Log visible scene objects.

If it's still black everywhere, it's on the VD side. Check that passthrough is switched on in VD (Settings → Streaming → VR Passthrough), that VD has the Quest's passthrough / spatial data permission, and that the R G B in VD match the plugin's box exactly.

Which key colour are you using?
Tick Use depth back on afterwards.
 
Thanks, don't know what plugin is responsible for ClearUpscale though.

Thanks, the key pass is last now, good.
The solid bottom row in the test pattern is actually the good result: it means VD's image settings are right. See-through there would mean a problem.

Two checks:
ClearUpscale is still loaded. Its cameras are still in your scene. It's part of the same DLSS package or a separate plugin. Remove that as well and restart VaM.

Quick isolation test: Advanced → untick Use depth (off = key everything). The whole picture becomes the key colour.

If you now see your room everywhere, VD is fine and something in the scene is still in the way. Send me Log visible scene objects.

If it's still black everywhere, it's on the VD side. Check that passthrough is switched on in VD (Settings → Streaming → VR Passthrough), that VD has the Quest's passthrough / spatial data permission, and that the R G B in VD match the plugin's box exactly.

Which key colour are you using?
Tick Use depth back on afterwards.
Still black. I won't take more of your time on this. I will wait for a plugin update. Thank you
 
I noticed I cant use chroma keyifier plugin with this? I add objects to the screen to like make it look like the model is behind a table or something but I cant use the chroma key on stuff now?

 
Last edited:
I noticed I cant use chroma keyifier plugin with this? I add objects to the screen to like make it look like the model is behind a table or something but I cant use the chroma key on stuff now?

Thanks for the feedback.
I didn't try Keyifier my self but PassthroughKey needs to be at the end of the rendering chain/pipeline, and that might not be compatible with Keyifinder.
What exactly do you yse it for?
PassthroughKey will offer similar functionality like surfaces, walls and doorways, in the next update though.
 
Thanks for the feedback.
I didn't try Keyifier my self but PassthroughKey needs to be at the end of the rendering chain/pipeline, and that might not be compatible with Keyifinder.
What exactly do you yse it for?
PassthroughKey will offer similar functionality like surfaces, walls and doorways, in the next update though.
I use it on dildos, i use it when i create boxes or cylinders to make certain things in my room appear Infront of the model, like if she is standing behind a real table her legs dont show. If you dont do that you can never get her to stand behind something without showing their body infront of it.
Also I make alot of scenes and need to be able to add objects like cubes and turn them on and off in transparency to place them while making scenes. Its a pain to go to the plugin every time and disable it and re enable it to see if the object is in the right spot.

**edit
Another thing I should mention is alot of scenes I use cubes and stuff to make sure the model doesnt go through furniture. I use the chrome key plugin to make it transparent once I have it placed where I need it/them. Maybe make a switch than can turn on and off transparency of objects.
 
Last edited:
I use it on dildos, i use it when i create boxes or cylinders to make certain things in my room appear Infront of the model, like if she is standing behind a real table her legs dont show. If you dont do that you can never get her to stand behind something without showing their body infront of it.
Also I make alot of scenes and need to be able to add objects like cubes and turn them on and off in transparency to place them while making scenes. Its a pain to go to the plugin every time and disable it and re enable it to see if the object is in the right spot.

**edit
Another thing I should mention is alot of scenes I use cubes and stuff to make sure the model doesnt go through furniture. I use the chrome key plugin to make it transparent once I have it placed where I need it/them. Maybe make a switch than can turn on and off transparency of objects.
Such switch exists already. (under Scene Handling)

Why doesn't Keyifinder work today:
Virtual Desktop can only cut one colour. Keyifier uses its own colour, while PassthroughKey keeps the background on its key colour and hides or protects everything else, so Keyifier objects never get cut.

In the next update PassthroughKey will recognise Keyifier objects and turn them into its own key colour automatically, shadows included. Your existing scenes with tables, boxes and occluders will just work, with only PassthroughKey's values set in Virtual Desktop.
Alongside that, you'll be able to mark any atom as an occluder or shadow catcher directly in PassthroughKey.
 
PassthroughKey 3.0 – coming soon
Your VaM characters already live in your real room.
In 3.0 they belong there!

What's coming:
• Room setup in the headset – trace your floor, ceiling, walls, windows, doors and furniture with the controller. It takes a few minutes, and it's saved to your profile.
• Real walls, real furniture – her body, hair and clothing collide with your walls and furniture. She can sit on your real sofa without sinking in.
• Occlusion – your real table hides her legs behind it. Walk around and she stays behind it.
• Shadows on walls and furniture – not just on the floor anymore.
• Chroma Keyifier support – as promised. Keyifier occluders and shadow catchers in your scenes work with PassthroughKey.
• Auto correction – a pose that lands her inside a wall gets fixed automatically.
• Quick menu – scene height by hand, show the original scene, re-align the room and save, all from VaM's toolbar.

3.0 is a feedback release.
Room setup is a big new step, and I'd like to hear what works in your room and what doesn't before polishing it further.

Still free, still CC BY-NC. Virtual Desktop (VDXR / Oculus mode) on Quest.
Stay tuned. – CraSh.it
 
Last edited:
PassthroughKey 3.0 – coming soon
Your VaM characters already live in your real room. In 3.0 they can see it.

What does that mean?
Also could you make the VR hands passthrough like that other mod like this one? Thats an awesome feature.
You know what would be cool but probably impossible, is being able to see yourself in mirrors.
 
What you mean by "what does that mean"?
Its a list of upcoming new features.
Also, I'm working on a separate plugin for better vr hands.
 
"Your VaM characters already live in your real room. In 3.0 they can see it."

Just didnt understand what that meant.
 
CraSh.it updated PassthroughKey with a new update entry:

PassthroughKey 3.0

PassthroughKey 3.0 is out – a feedback release.


New:


  • Room setup in the headset: floor, ceiling, walls, windows, doors and furniture (boxes and cylinders).
  • Room collision: body, hair and clothing collide with your walls and furniture.
  • Room occlusion: your walls and real furniture hide what’s behind them.
  • Shadows on walls and furniture.
  • Auto Scene Correction: a pose that ends up in a wall or outside of your room, is...

Read the rest of this update entry...
 
After using it, the character seems to have really appeared beside me. It's much better than the previous effect without a shadow!
It would be even more perfect if it could be compatible with DLSS5!
Now that DLSS5 is enabled, the darker areas become transparent, but the sense of reality is still much stronger than before it was enabled, as if this person were really right beside you.
 
  • "Room occlusion: your walls and real furniture hide what’s behind them."

    Is there a way to do this with our body? like when you put your hands infront of a model they are behind them all of the sudden. I hate the floating VR hands so that would be magic lol. would be crazy if sitting next to them our arms and stuff appear infront of them. I tried possessing a males arms and hands and using the invisibility tool on the rest of the male body to just keep the arms and hands but it didnt work very well.
 
Hi! First of all, thank you for this awsome plugin (y) (y) (y) . I really like its passthrough and room-light matching features, especially the ability to capture real-world lighting and align it with the virtual environment.

I have noticed a few issues with the lighting system, and I would like to suggest some improvements.

1. Ambient lighting settings are overwritten and not properly saved

The plugin seems to control VaM's global Scene Lighting settings, including Ambient Light Color and ambient intensity/effect values.

The problem is that when I manually adjust these settings in VaM, the plugin may overwrite them again. Also, my final manual adjustments do not seem to be properly saved in the plugin's Profiles.

It would be better to either leave VaM's native ambient lighting settings untouched by default, or provide a proper option to control and save all the ambient lighting values.

2. The Light Matching controls are limited

Currently, the plugin provides its own simplified lighting controls, mainly brightness and color temperature.

However, these controls do not expose all the settings available in VaM's native Light Atom UI. For example, I would like to adjust light range, shadows, shadow strength, softness and other native light properties.

Also, when I manually adjust a light through VaM's native Light Atom interface, those changes do not seem to be saved by the plugin's Profile system. The Profile appears to restore the plugin's own settings instead.

My suggestion is to let users edit the generated lights directly through VaM's native Light Atom controls, and make sure the Profiles save the complete light configuration, including all manually adjusted properties.

3. A bug: VR controller movement resets the lighting

This is probably the most frustrating issue.

After manually adjusting the lights and ambient settings, simply moving the Quest controller or using the joystick can cause the plugin to reapply its stored lighting settings.

As a result, my manual adjustments are lost and the lighting returns to the previously saved plugin values.

Could the spatial tracking system be separated from the light-property settings? Moving or repositioning the VR space should only update the lights' positions and rotations, and should never overwrite their brightness, color, range, shadows or other properties.

4. 🙏Suggestion: Support up to three registered lights and three ground fake shadows

I think the lighting system could be much simpler and more flexible.

Instead of limiting users to only two lights, it would be great to have a simple "+ Light" button in the Light Matching menu.

Each time the user presses it, the plugin would create another light, just like the existing generated lights, and automatically register it with the plugin's spatial tracking system.

Users could then freely move and adjust each light through VaM's native controls, while the plugin handles spatial alignment and saves everything in the Profile.

I personally think supporting three lights simultaneously would be ideal. More than three is probably unnecessary for this feature, although even allowing two would already be an improvement.

It would also be great if the ground fake-shadow system could support up to three simultaneous shadows, corresponding to the registered lights. Two would be acceptable, but three would be preferable.

I really like the existing real-world lighting capture and spatial alignment features. I don't think they need to be redesigned. I just hope the lighting management and Profile system can be made more flexible and reliable.

Thank you again for your work!
 
Back
Top Bottom