Guides How to use Hand Tracking in VAM | 2.0 (Quest)

[thinoreos]

thinoreos submitted a new resource:

How to use Hand Tracking in VAM (Quest) - A guide on how to use Hand Tracking in VAM with Virtual Desktop

Guide last updated: March 2024
How to use Hand Tracking in VAM in VR with Virtual Desktop!
*Difficulty level: 3/5*

What is this?
Imagine using your hands in VAM without any LeapMotion or external sensors, purely just the Quest. Now you can!
View attachment 340228View attachment 340229View attachment 340243[ATTACH type="full"...

Read more about this resource...

 

[Madry]

Is it possible to use this for the oculus rift S?

 

[thinoreos]

Is it possible to use this for the oculus rift S?

Sorry not possible, only with Quest 2, 3 & Pro for now

 

[thinoreos]

thinoreos updated How to use Hand Tracking in VAM (Quest) with a new update entry:

1.1

Quick info fix, description changes

Read the rest of this update entry...

 

[Saint66]

@VRCube I also have the Leap Motion 2 controller and tested the VD solution just out of curiousity.
It's a great solution for users w/o additional HW like Ultraleap or expensive gloves and such.

But: I won't throw away my Leap2, because it's way more accurate and faster, the VD solution is nice but also somehow laggy and limited compared to Leap.
Anyway, that's a good addition from the VD guys, I won't complain.

They also emulate body tracking, but thats not really useable in it's current first beta stage and prolly would never reach anything near to real trackers.

 

[Bob Nothing]

I have a Quest 3... Does this mean I would be able to do finger motion capture in recording animations?

 

[Trogdor69]

while hand tracking could you use the controllers to track other parts of the body simultaneously? Using embody in the tracker settings perhaps?

 

[thinoreos]

I have a Quest 3... Does this mean I would be able to do finger motion capture in recording animations?

Yes you can, there are some people online who use Quest to record mocap for hand tracking since a while ago. I personally haven't tested it but you can search around online!

 

[thinoreos]

while hand tracking could you use the controllers to track other parts of the body simultaneously? Using embody in the tracker settings perhaps?

Yes, the other trackers show up (eg legs) but anything below lower body kinda sucks at tracking

 

[Saint66]

I have a Quest 3... Does this mean I would be able to do finger motion capture in recording animations?

Yes, just tried with Embody and Timeline.
I recorded finger motion into an existing animation which had no animated fingers.
Worked out great.

I used a QPro, but that doesn’t make a difference

 

[Wxy_332]

Does anyone know how can we used Quest 3 controllers for the foot tracking and our hands for the hand tracking using this beta? Is it possible?

 

[thinoreos]

Does anyone know how can we used Quest 3 controllers for the foot tracking and our hands for the hand tracking using this beta? Is it possible?

Quest controllers for foot is a bit complicated, but if you use external trackers it will work together with this beta. (eg vive trackers, slime, etc)

 

[One-Headlight]

Dont have a good connection to streamvr, is there a way to work with quest link?

 

[Dan2st0n3d]

Hello, is it compatible with pico 3?

 

[thinoreos]

Dont have a good connection to streamvr, is there a way to work with quest link?

No, it requires SteamVR & Virtual Desktop to work

 

[thinoreos]

Hello, is it compatible with pico 3?

Sorry no only Quest

 

[thinoreos]

thinoreos updated How to use Hand Tracking in VAM (Quest) with a new update entry:

1.2

Added selecting items and opening the menu!

Read the rest of this update entry...

 

[flyleaf]

Hi i have a question. what is cowgirl scene in your guide?
I can't find on hub
If you have any information , please tell me.
Thank you!

 

[thinoreos]

Hi i have a question. what is cowgirl scene in your guide?
I can't find on hub
If you have any information , please tell me.
Thank you!

It's at the bottom of the credits, https://hub.virtamate.com/resources/haley-armageddon.24118/

 

[animetiddyenthusiast]

Dont have a good connection to streamvr, is there a way to work with quest link?

that's an issue on VAM's side. rn the only hand tracking supported is steamvr or leapmotion. it would be possible to recreate the same feature as VD but in a windows app but it will still suffer from added latency and is generally a lot of work. If we weren't on a 6 year old version of unity it would be feasible to add openxr support but with current version no. vam2 supports openxr/oculus xr natively so this should just be a feature, but realistically it may have to be added by plugin authors

as for leg tracking its just not at the level meta promised yet so no surprise VD hasnt done anything with it. I know meta also promised some estimation for quest 2 that I think just doesnt exist yet lol. But both quest 2 and quest 3 can use decent waist tracking which VD 100% should've implemented as that makes enough difference that you dont even really need leg tracking unless you're trying to kick people or something hadnt seen the update lol. vive trackers do actually work on quest 2 which is surprising

 

[Trogdor69]

Quest controllers for foot is a bit complicated, but if you use external trackers it will work together with this beta. (eg vive trackers, slime, etc)

I haven't been able to make this(Simultaneous Handtracking/ Body tracking w/ controller as tracker) work yet. Seems the controllers can not be used to track something other than your hands, while you're using handtracking for your hands. In embody, the controllers are listed as Left Hand/Right Hand then a few other vive tracker options for the hips/legs.. If there was an option or a toggle to force the controllers to look like vive trackers instead of controllers i think we'd be good to go. Otherwise as soon as you touch the controller the hand tracking just goes back to controller and doesn't stay on hand tracking.

 

[thinoreos]

I haven't been able to make this(Simultaneous Handtracking/ Body tracking w/ controller as tracker) work yet. Seems the controllers can not be used to track something other than your hands, while you're using handtracking for your hands. In embody, the controllers are listed as Left Hand/Right Hand then a few other vive tracker options for the hips/legs.. If there was an option or a toggle to force the controllers to look like vive trackers instead of controllers i think we'd be good to go. Otherwise as soon as you touch the controller the hand tracking just goes back to controller and doesn't stay on hand tracking.

Sorry not entirely sure what you mean, but you can only either do:
- Hand tracking with IOBT
- Controllers with IOBT
but not both 1 controller and 1 hand tracking

If I get what you're trying to do, you want to enable body tracking with hand tracking in Embody, correct? In that case don't touch the controllers at all, just use hand tracking to select everything. You can use my steamvr bindings for selecting stuff

 

[thinoreos]

that's an issue on VAM's side. rn the only hand tracking supported is steamvr or leapmotion. it would be possible to recreate the same feature as VD but in a windows app but it will still suffer from added latency and is generally a lot of work. If we weren't on a 6 year old version of unity it would be feasible to add openxr support but with current version no. vam2 supports openxr/oculus xr natively so this should just be a feature, but realistically it may have to be added by plugin authors

as for leg tracking its just not at the level meta promised yet so no surprise VD hasnt done anything with it. I know meta also promised some estimation for quest 2 that I think just doesnt exist yet lol. But both quest 2 and quest 3 can use decent waist tracking which VD 100% should've implemented as that makes enough difference that you dont even really need leg tracking unless you're trying to kick people or something hadnt seen the update lol. vive trackers do actually work on quest 2 which is surprising

To be fair, VAM VR is already quite ahead compared to a lot of VR games (despite being on a super old engine) with leap motion, tracking, etc. But yes you're right hopefully VAM2 does everything it needs to

 

[Trogdor69]

Sorry not entirely sure what you mean, but you can only either do:
- Hand tracking with IOBT
- Controllers with IOBT
but not both 1 controller and 1 hand tracking

If I get what you're trying to do, you want to enable body tracking with hand tracking in Embody, correct? In that case don't touch the controllers at all, just use hand tracking to select everything. You can use my steamvr bindings for selecting stuff

I want to use hand tracking while simultaneously using the controllers to track hip or leg or something else. Doesn’t seem to be possible at the moment unless you can make the controllers show up as vive trackers instead of controllers.

 

[thinoreos]

I want to use hand tracking while simultaneously using the controllers to track hip or leg or something else. Doesn’t seem to be possible at the moment unless you can make the controllers show up as vive trackers instead of controllers.

As far as I know, using Quest controllers to track other body parts is very hard to do already, and this is an additional layer of things you want to do... I think your best bet is to buy real trackers