Virtual Desktop: Update enables Quest hand tracking on PC via VDXR

I conducted a short test by launching VaM via VDXR (using OpenVR and OpenComposite) and tested some scenes with Embody. I was able to see the movement of each finger separately, but only in a small range (like 10-20 degrees). Thus, finger-tracking information is transferred to VaM, but it probably needs some additional configuration.
 
I detected (by the way) another VD "improvement" with this crap last "update"... (worst vam latency) 💩 🤣
 

After setting Finger tracking factor to 1.0 - finger tracking works great. But some problems with hand angle.
 
nicman20061 said:
I just run VaM via opencomposite (using this guide and discussion https://hub.virtamate.com/resources/how-to-launch-without-steamvr-performance-increase.30496/). Also configured VD to use VDXR runtime on PC, and switch on "forward tracking data" on quest side.
Click to expand...

Hold up are you using a quest pro? The articles seem to indicate that it's only for that, because I'm pretty sure I'm using all of those settings you reference. Except for "forward tracking data" which Im not sure is an option for Quest 2.

edit: Nevermind I found "forward tracking data" in the streaming options of VD. I turned it on and tried it, but I still wasn't able to track more than just pinching with the thumb and forefinger. Anybody else able to replicate what Nicman did on a Quest 2?
 
Last edited:
vertigoeighty said:
Hold up are you using a quest pro? The articles seem to indicate that it's only for that, because I'm pretty sure I'm using all of those settings you reference. Except for "forward tracking data" which Im not sure is an option for Quest 2.

edit: Nevermind I found "forward tracking data" in the streaming options of VD. I turned it on and tried it, but I still wasn't able to track more than just pinching with the thumb and forefinger. Anybody else able to replicate what Nicman did on a Quest 2?
Click to expand...
I'm using quest 3
 
vertigoeighty said:
Hold up are you using a quest pro? The articles seem to indicate that it's only for that, because I'm pretty sure I'm using all of those settings you reference. Except for "forward tracking data" which Im not sure is an option for Quest 2.

edit: Nevermind I found "forward tracking data" in the streaming options of VD. I turned it on and tried it, but I still wasn't able to track more than just pinching with the thumb and forefinger. Anybody else able to replicate what Nicman did on a Quest 2?
Click to expand...
I have the same thing, only a full turn of the hand works, and clenching the fist with the thumb and forefinger. Quest 3
nicman20061 said:
I'm using quest 3
Click to expand...

Can you show settings at VR1 and VR2? And what Runtime do you use, VDXR or Oculus? I'll try start with VDXR, but its always start with Oculus
 
Tyrelio said:
I have the same thing, only a full turn of the hand works, and clenching the fist with the thumb and forefinger. Quest 3


Can you show settings at VR1 and VR2? And what Runtime do you use, VDXR or Oculus? I'll try start with VDXR, but its always start with Oculus
Click to expand...
I used VDXR runtime. Here are some screenshots and video with my configs.
photo_2024-02-10_18-18-59.jpgphoto_2024-02-10_18-18-55.jpgscrn2024-02-10 182349.png
scrn 2024-02-10 181956.png
 
Last edited:
Thx for info, looks like its some buggy with opencomposite in my side. Cannot create DX texture swap chain: err -2 - looking for help on google)
 
Tyrelio said:
Thx for info, looks like its some buggy with opencomposite in my side. Cannot create DX texture swap chain: err -2 - looking for help on google)
Click to expand...
Check your bat file, it should be "START "VaM" VaM.exe -vrmode", without "OpenVR" in the end.
 
Does finger tracking work with the vam.bat method, i.e. launching in VD Streamer or does it only work with OpenComposite? I can't seem to get OpenComposite to work. Same -2 error above.
 
I manage to get it work like this:

  1. Do exactly what nicman20061 said
    1. Use VDRX runtime in Virtual Desktop Streamer setting
    2. In my Quest 3's Virtual Desktop, enable Forward tracking data to PC
    3. Create a new .bat START "VaM" VaM.exe -vrmode
      1. You may call it "VaM (OpenXR).bat"
    4. Download OpenComposite
      1. Open OpenComposite.exe
      2. Switch runtime to OpenComposite (OpenXR)
  2. Use a openvr_api.dll from a Skyrim Mod (Source: https://hub.virtamate.com/threads/openxr-now-works-with-vam.32890/page-2#post-133847)
    1. First make a backup of the original openvr_api.dll
    2. Rename the original openvr_api.dll to "./VaM_Data/Plugins/openvr_api.pre-open-xr.dll"
    3. Download and extract the openvr_api.dll from https://www.nexusmods.com/skyrimspecialedition/mods/85389?tab=files&file_id=464347
    4. Place the extracted file into "./VaM_Data/Plugins/openvr_api.dll"
  3. Wear your Oculus and start Virtual Desktop client
  4. Start VaM.exe(This works when you do swtich runtime OpenComposite (OpenXR))
    1. VaM (OpenXR).bat is not required if your default runtime is set to OpenComposite already
    2. if Runtime = VDXR, then you are running via OpenComposite

Settings:
- Once you are in VaM, go to user preference and set Finger tracking factor to 1.0.

My observation:
  1. Finger tracking works
  2. In-game hand model is misaligned when using controller. However, the laser pointer works well.
    I had to apply offset to the position and rotation to align the hand model

    1708805955338.png
  3. Controller's grip button and trigger button no longer bend the in-game fingers
  4. Without controller, hand tracking's orientation only works correctly when palm is facing up, its orientation become misaligned otherwise
    1. Changing the offset / rotation does not change the hand tracked model.
 
screamingoctopus said:
I manage to get it work like this:

  1. Do exactly what nicman20061 said
    1. Use VDRX runtime in Virtual Desktop Streamer setting
    2. In my Quest 3's Virtual Desktop, enable Forward tracking data to PC
    3. Create a new .bat START "VaM" VaM.exe -vrmode
      1. You may call it "VaM (OpenXR).bat"
    4. Download OpenComposite
      1. Open OpenComposite.exe
      2. Switch runtime to OpenComposite (OpenXR)
  2. Use a openvr_api.dll from a Skyrim Mod (Source: https://hub.virtamate.com/threads/openxr-now-works-with-vam.32890/page-2#post-133847)
    1. First make a backup of the original openvr_api.dll
    2. Rename the original openvr_api.dll to "./VaM_Data/Plugins/openvr_api.pre-open-xr.dll"
    3. Download and extract the openvr_api.dll from https://www.nexusmods.com/skyrimspecialedition/mods/85389?tab=files&file_id=464347
    4. Place the extracted file into "./VaM_Data/Plugins/openvr_api.dll"
  3. Wear your Oculus and start Virtual Desktop client
  4. Start VaM.exe(This works when you do swtich runtime OpenComposite (OpenXR))
    1. VaM (OpenXR).bat is not required if your default runtime is set to OpenComposite already
    2. if Runtime = VDXR, then you are running via OpenComposite

Settings:
- Once you are in VaM, go to user preference and set Finger tracking factor to 1.0.

My observation:
  1. Finger tracking works
  2. In-game hand model is misaligned when using controller. However, the laser pointer works well.
    I had to apply offset to the position and rotation to align the hand model

    View attachment 338837
  3. Controller's grip button and trigger button no longer bend the in-game fingers
  4. Without controller, hand tracking's orientation only works correctly when palm is facing up, its orientation become misaligned otherwise
    1. Changing the offset / rotation does not change the hand tracked model.
Click to expand...
So it seems mostly useless then?

You can't interact with any buttons or UI at all either?
 
So, I followed al the steps above, but don't seem to have individual finger tracking. Any idea where I should look?

ETA: Seems to be working better, but has weird issue with hands "jumping" around as I try to gently move or rotate them. Still, it's pretty cool seeing the individual fingers of the VR hands emulating the motions of my real fingers.
 
Last edited:
So, I've had this work exactly once. Settings are correct. I can make my VAM hands disappear by turning them palm-up and bringing thumb and forefinger together, but I haven't had individual finger response except that one time.
 
Virtual Desktop had another update a few weeks ago that allows you to emulate Valve Index Finger Tracking, so now finger tracking works really well on the Quest 2/3. Make sure you have hand tracking enable in your quest settings, and In Virtual Desktop make sure to select "Forward tracking data to PC" and "Emulate Index Controllers" in the streaming settings on your quest. At that point you can launch VAM normally thru SteamVR and the individual finger tracking should be enabled.

Once in VAM you can tweak the sensitivity of the tracking in the VR Hands settings as referenced above.

Not sure if there is a way to get it working in oculus mode though. I didn't have any luck with that, and had to resort to using SteamVr.
 
Last edited:
You must log in or register to reply here.

Similar threads

DanVR
Quest 2 - Hand Tracking 2.0 is... WOW!!!
2
Replies
37
Views
26K
alex888
A
Zweistein
Oculus Hand Tracking Support?
Replies
1
Views
3K
T_bob
T_bob
A
Quest 3 + life size doll + hand tracking- WOW!
Replies
5
Views
3K
EasyVam
EasyVam
S
VaM + Thin Twin + Virtual Desktop + Quest 3
Replies
3
Views
2K
stylesi
stylesi
Mel
Virtual Desktop
Replies
8
Views
815
Mel
Mel
Back
Top Bottom