Guide last updated: August 2024
How to use Hand Tracking in VAM in VR with Virtual Desktop!
*Difficulty level: 3/5*
Update 2.0 is here baby! NEW: No more accidental grabbing, movement (finally), and more!
What is this?
Imagine using your hands in VAM without any LeapMotion or external sensors, purely just the Quest. Now you can!
You can either use Steam Link (Free) or Virtual Desktop (Paid)
DISCLAIMER: Hand tracking is NOT perfect. This is still a beta technology, and will not be as accurate as controllers. You will need GOOD LIGHTING and to clean your lenses.
What will I need?
- A Quest VR headset (Quest 2, Quest 3, Quest Pro)
- Steam Link OR Virtual Desktop (Quest version)
- Hands (2 preferably)
NEW METHOD (STEAM LINK)
Steps to get handy:
1) Install and set up the latest Beta of Steam Link and SteamVR on your PC and Headset.
This has 2 steps, 1 is Opting into SteamVR Beta, 2 is Opting into Quest Beta
Link to installing new Beta: https://store.steampowered.com/news/app/250820?updates=true&emclan=103582791435040972&emgid=6892282513997962009DISCLAIMER:
(Don't forget you need to opt into the Beta for Quest too!)
2) Turn on Hand Tracking on your Quest
Settings > Movement Tracking > Hand and body tracking
(I personally like to use Double tap controllers to switch to hand tracking, very handy)
4) Launch Steam Link. Make sure you're on the latest beta.
5) Launch VAM through SteamVR.
6) VAM User preferences settings:
i. VR 1 tab
- Disable Show Controllers
- Enable Hide Trackers if you are using "full body tracking"
- Enable or Disable Use controller Hand Pose, this is up to you for the "default" hand resting pose
- Fingers (Thumb Tracking Factor & Finger Tracking Factor) set to maximum 1.00
- Optional: Finger Bend Offset to 0 if you want a "flat hand"
ii. VR 2 tab
- Enable Use Collision
- Enable Left & Right
- For hand choice, you can choose which type of Hand you want. I am using Male 2
- The rest are optional, I would leave them on default, up to you if you want to tweak
7) Switch to Hand Tracking on your quest. You can do this by putting down your controllers and waiting, or tapping them with each other.
If hand tracking is enabled properly, you will see this in the VD input tab:
SteamVR Bindings
This will allow you to select and click things, and also move & open the menu. I made these bindings in SteamVR, so if you're too lazy you can search for them on Steam the bindings are called VaM Hand Tracking 2.0.
(Make sure you set the Current Controller as "Hand Tracking" in Steam Bindings to see mine)
Controls:
1) Select: Pinch your Index Finger + Thumb together. Works with both hands.
NOTE: Do not pinch index with your hands facing upwards/palm up, this will activate the Menu instead.
2) Move: Hold your Ring Finger (4th finger) + Thumb together (Steam icon will show up). Then move your hand around to grab the world to move. Palm up is easier.
3) Open Menu: Hold your Pinky (5th finger) + Thumb together (Steam icon will show up). Works with both hands, aim hand vertical to get menu straight like in the gif.
3) Grabbing: No more accidental grabbing, I've removed the function to grab with your whole hand. Now to grab, double pinch your index finger + thumb on the body part you want to grab.
4) VamX: To open the VamX menu, just the game menu normally and select VamX UI. Then press Hide/Show for the UI.
*Please note, these controls are not 100% accurate or perfect, they are just the skeletal inputs we have to work with right now
Tips:
- You can also use this in passthrough mode, it's extra immersion
- Using Embody plugin with it also adds a lot of immersion
- I would switch back to controllers whenever needing to adjust things
- The Main Menu is a bit hard to navigate with hands, controllers is easier
- If you want to hide the Quest menu icons on your finger, you can do so by going to Quest Settings > Experimental Features > Turn on Wrist Gestures
- Make sure you have GOOD lighting and clean your lenses!
OLD METHOD (VIRTUAL DESKTOP)
Voila enjoy
View attachment 340241
Credits:
Scene: Haley - Armageddon by ascorad
Scene: Cyberpunk Cowgirl by CuddleMocap
Model: Sakurako by Callimohu
Model: Iseul by SuperSamoth
How to use Hand Tracking in VAM in VR with Virtual Desktop!
*Difficulty level: 3/5*
Update 2.0 is here baby! NEW: No more accidental grabbing, movement (finally), and more!
What is this?
Imagine using your hands in VAM without any LeapMotion or external sensors, purely just the Quest. Now you can!
You can either use Steam Link (Free) or Virtual Desktop (Paid)
DISCLAIMER: Hand tracking is NOT perfect. This is still a beta technology, and will not be as accurate as controllers. You will need GOOD LIGHTING and to clean your lenses.
What will I need?
- A Quest VR headset (Quest 2, Quest 3, Quest Pro)
- Steam Link OR Virtual Desktop (Quest version)
- Hands (2 preferably)
NEW METHOD (STEAM LINK)
Steps to get handy:
1) Install and set up the latest Beta of Steam Link and SteamVR on your PC and Headset.
This has 2 steps, 1 is Opting into SteamVR Beta, 2 is Opting into Quest Beta
Link to installing new Beta: https://store.steampowered.com/news/app/250820?updates=true&emclan=103582791435040972&emgid=6892282513997962009DISCLAIMER:
(Don't forget you need to opt into the Beta for Quest too!)
2) Turn on Hand Tracking on your Quest
Settings > Movement Tracking > Hand and body tracking
(I personally like to use Double tap controllers to switch to hand tracking, very handy)
4) Launch Steam Link. Make sure you're on the latest beta.
5) Launch VAM through SteamVR.
6) VAM User preferences settings:
i. VR 1 tab
- Disable Show Controllers
- Enable Hide Trackers if you are using "full body tracking"
- Enable or Disable Use controller Hand Pose, this is up to you for the "default" hand resting pose
- Fingers (Thumb Tracking Factor & Finger Tracking Factor) set to maximum 1.00
- Optional: Finger Bend Offset to 0 if you want a "flat hand"
ii. VR 2 tab
- Enable Use Collision
- Enable Left & Right
- For hand choice, you can choose which type of Hand you want. I am using Male 2
- The rest are optional, I would leave them on default, up to you if you want to tweak
7) Switch to Hand Tracking on your quest. You can do this by putting down your controllers and waiting, or tapping them with each other.
If hand tracking is enabled properly, you will see this in the VD input tab:
SteamVR Bindings
This will allow you to select and click things, and also move & open the menu. I made these bindings in SteamVR, so if you're too lazy you can search for them on Steam the bindings are called VaM Hand Tracking 2.0.
(Make sure you set the Current Controller as "Hand Tracking" in Steam Bindings to see mine)
Controls:
1) Select: Pinch your Index Finger + Thumb together. Works with both hands.
NOTE: Do not pinch index with your hands facing upwards/palm up, this will activate the Menu instead.
2) Move: Hold your Ring Finger (4th finger) + Thumb together (Steam icon will show up). Then move your hand around to grab the world to move. Palm up is easier.
3) Open Menu: Hold your Pinky (5th finger) + Thumb together (Steam icon will show up). Works with both hands, aim hand vertical to get menu straight like in the gif.
3) Grabbing: No more accidental grabbing, I've removed the function to grab with your whole hand. Now to grab, double pinch your index finger + thumb on the body part you want to grab.
4) VamX: To open the VamX menu, just the game menu normally and select VamX UI. Then press Hide/Show for the UI.
*Please note, these controls are not 100% accurate or perfect, they are just the skeletal inputs we have to work with right now
Tips:
- You can also use this in passthrough mode, it's extra immersion
- Using Embody plugin with it also adds a lot of immersion
- I would switch back to controllers whenever needing to adjust things
- The Main Menu is a bit hard to navigate with hands, controllers is easier
- If you want to hide the Quest menu icons on your finger, you can do so by going to Quest Settings > Experimental Features > Turn on Wrist Gestures
- Make sure you have GOOD lighting and clean your lenses!
OLD METHOD (VIRTUAL DESKTOP)
1) Install and set up the latest version of Virtual Desktop streamer on your PC (NON-Steam version, install the PC streamer app for your correct Headset)
Version 1.30.4 onwards supports Tracking
Quest VD Download: https://www.meta.com/experiences/2017050365004772/
PC Streamer App download: https://www.vrdesktop.net/
FAQ: If you're unsure how to do this, YouTube search "How to install Virtual Desktop + (your headset)"
2) Install Virtual Desktop on your headset
3) Turn on Hand Tracking on your Quest
Settings > Movement Tracking > Hand and body tracking
(I personally like to use Double tap controllers to switch to hand tracking, very handy)
4) Launch Virtual Desktop. Make sure you're on the latest version.
i. Enable Hand tracking under Input tab
ii. Enable Forward tracking data to PC under Streaming tab
iii. Enable Index controllers. Optional, you can also enable Vive trackers for "full body tracking"
5) Launch VAM in SteamVR mode.
(If you did everything right, your controllers will be INDEX controllers in SteamVR, like below. If you are seeing Quest controllers, something went wrong, scroll down for troubleshooting)
6) VAM User preferences settings:
i. VR 1 tab
- Disable Show Controllers
- Enable Hide Trackers if you are using "full body tracking"
- Enable or Disable Use controller Hand Pose, this is up to you for the "default" hand resting pose
- Fingers (Thumb Tracking Factor & Finger Tracking Factor) set to maximum 1.00
ii. VR 2 tab
- Enable Use Collision
- Enable Left & Right
- For hand choice, you can choose which type of Hand you want. I am using Male 2
- The rest are optional, I would leave them on default, up to you if you want to tweak
7) Switch to Hand Tracking on your quest. You can do this by putting down your controllers and waiting, or tapping them with each other.
If hand tracking is enabled properly, you will see this in the VD input tab:
You're done, enjoy!
Bonus: SteamVR Bindings
For normal VAM (Sensitive grabbing)
This will allow you to select and click things, and also open the menu. I made these bindings in SteamVR, so if you're too lazy you can search for them on Steam the bindings are called Hand Tracking VaM (Normal)
Controls:
1) Select: Pull your index finger like you're pulling a real life trigger on a VR controller
2) Open Menu: Pull your index finger twice FAST (Must be fast, sometimes using two hands at the same time works better)
3) Grip/Grab: Pull your last 3 fingers like you're pulling the grip button on a VR controller
These are the binding settings:
- Trigger Single: Interact with UI
- Trigger Double: Open, Close or Move Menu UI
For VAMX (Less sensitive grabbing)
I also made VamX bindings. They technically can be used on both normal VaM and VamX, but the controls are different. Up to you what you prefer. On steam it's called Hand Tracking VamX
Controls:
1) Select: Pull your last 3 fingers like you're pulling the grip button on a VR controller
2) Open Menu: Pull your index finger twice FAST (Must be fast, sometimes using two hands at the same time works better)
Use Open vamX UI & Hide/Show buttons to toggle vamX:
*Please note, these controls are not 100% accurate or perfect, they are just the skeletal inputs we have to work with right now
Troubleshooting:
- If it's not working or only showing Quest controllers instead of Index controllers in SteamVR, make sure your SteamVR addons are ONLY having ON for Virtual Desktop Streamer (Quest)
Launch SteamVR > Right click SteamVR > Settings > Startup/Shutdown > Manage Add-Ons:
Version 1.30.4 onwards supports Tracking
Quest VD Download: https://www.meta.com/experiences/2017050365004772/
PC Streamer App download: https://www.vrdesktop.net/
FAQ: If you're unsure how to do this, YouTube search "How to install Virtual Desktop + (your headset)"
2) Install Virtual Desktop on your headset
3) Turn on Hand Tracking on your Quest
Settings > Movement Tracking > Hand and body tracking
(I personally like to use Double tap controllers to switch to hand tracking, very handy)
4) Launch Virtual Desktop. Make sure you're on the latest version.
i. Enable Hand tracking under Input tab
ii. Enable Forward tracking data to PC under Streaming tab
iii. Enable Index controllers. Optional, you can also enable Vive trackers for "full body tracking"
5) Launch VAM in SteamVR mode.
(If you did everything right, your controllers will be INDEX controllers in SteamVR, like below. If you are seeing Quest controllers, something went wrong, scroll down for troubleshooting)
6) VAM User preferences settings:
i. VR 1 tab
- Disable Show Controllers
- Enable Hide Trackers if you are using "full body tracking"
- Enable or Disable Use controller Hand Pose, this is up to you for the "default" hand resting pose
- Fingers (Thumb Tracking Factor & Finger Tracking Factor) set to maximum 1.00
ii. VR 2 tab
- Enable Use Collision
- Enable Left & Right
- For hand choice, you can choose which type of Hand you want. I am using Male 2
- The rest are optional, I would leave them on default, up to you if you want to tweak
7) Switch to Hand Tracking on your quest. You can do this by putting down your controllers and waiting, or tapping them with each other.
If hand tracking is enabled properly, you will see this in the VD input tab:
You're done, enjoy!
Bonus: SteamVR Bindings
For normal VAM (Sensitive grabbing)
This will allow you to select and click things, and also open the menu. I made these bindings in SteamVR, so if you're too lazy you can search for them on Steam the bindings are called Hand Tracking VaM (Normal)
Controls:
1) Select: Pull your index finger like you're pulling a real life trigger on a VR controller
2) Open Menu: Pull your index finger twice FAST (Must be fast, sometimes using two hands at the same time works better)
3) Grip/Grab: Pull your last 3 fingers like you're pulling the grip button on a VR controller
These are the binding settings:
- Trigger Single: Interact with UI
- Trigger Double: Open, Close or Move Menu UI
For VAMX (Less sensitive grabbing)
I also made VamX bindings. They technically can be used on both normal VaM and VamX, but the controls are different. Up to you what you prefer. On steam it's called Hand Tracking VamX
Controls:
1) Select: Pull your last 3 fingers like you're pulling the grip button on a VR controller
2) Open Menu: Pull your index finger twice FAST (Must be fast, sometimes using two hands at the same time works better)
Use Open vamX UI & Hide/Show buttons to toggle vamX:
*Please note, these controls are not 100% accurate or perfect, they are just the skeletal inputs we have to work with right now
Troubleshooting:
- If it's not working or only showing Quest controllers instead of Index controllers in SteamVR, make sure your SteamVR addons are ONLY having ON for Virtual Desktop Streamer (Quest)
Launch SteamVR > Right click SteamVR > Settings > Startup/Shutdown > Manage Add-Ons:
Voila enjoy
View attachment 340241
Credits:
Scene: Haley - Armageddon by ascorad
Scene: Cyberpunk Cowgirl by CuddleMocap
Model: Sakurako by Callimohu
Model: Iseul by SuperSamoth
