PlugIn Developing Oculus Quest HandTracking

[Heeman]

Hello Guys,

im new to PlugIns for VAM and im very interested to take a try to develop a plugIn for Handtracking on oculus quest. BUT, im not into plugIn development in VAM yet and my first Question is if it is possible at all to implement Hand Tracking for Oculus Quest? Possible therefore because of existing limitations in modifying camera things and so on. Is there someone who is experienced in such things and is there a source code/VAMUnity3d/git project? Thank you!

Cheers, Heeman.

 

[Acid Bubbles]

You might want to look at this project:

https://www.reddit.com/r/VAMscenes/comments/b93bsd

Since it's now built-in it's not maintained, but you might find code that you can use in it.

 

[SukebeDude]

I actually had hand tracking in game in one session without any plugins or anything, it was purely by accident as i noticed it tracking both my controllers and my hands, i haven't been able to replicate it and every time i bring it up in other forums it's pretty much ignored, presumably because they don't use a Quest or maybe they don't believe me, surely i can't have been the only one who experienced this.

 

[Heeman]

I actually had hand tracking in game in one session without any plugins or anything, it was purely by accident as i noticed it tracking both my controllers and my hands, i haven't been able to replicate it and every time i bring it up in other forums it's pretty much ignored, presumably because they don't use a Quest or maybe they don't believe me, surely i can't have been the only one who experienced this.

So you had been able to move your fingers and you saw your fingermovements in virt-a-mate?

 

[SukebeDude]

So you had been able to move your fingers and you saw your fingermovements in virt-a-mate?

Yes, though the finger movement were a bit limited, as in it wouldn't let me completely close my hands, it was probably not 1:1 in terms of individual fingers, but i did for sure control them with the Quest tracking, now that i think about it i might have been controlling the "controller" hands rather than hooking into the Leap function.

I had other issues with the Quest at the time due to an app (Yur, found on sidequest) being incompatible with the recent firmware update on the Quest, might have something to do with that, maybe it somehow kept the tracking awake through the Oculus Link which usually disable it, i had to get rid of Yur though as it messed with the controller tracking making them drift away all the time, it probably has been fixed by now but maybe you can find an older version to experiment with.

This all happened like a week or two ago.

 

[Heeman]

Yes, though the finger movement were a bit limited, as in it wouldn't let me completely close my hands, it was probably not 1:1 in terms of individual fingers, but i did for sure control them with the Quest tracking, now that i think about it i might have been controlling the "controller" hands rather than hooking into the Leap function.

I had other issues with the Quest at the time due to an app (Yur, found on sidequest) being incompatible with the recent firmware update on the Quest, might have something to do with that, maybe it somehow kept the tracking awake through the Oculus Link which usually disable it, i had to get rid of Yur though as it messed with the controller tracking making them drift away all the time, it probably has been fixed by now but maybe you can find an older version to experiment with.

This all happened like a week or two ago.

Sounds strange. I doubt that it is possible that the controller tracking which is programmed can be used for the hand tracking. Also it is nearly impossible that the quest sdk can adapt it by itself to move fingers etc.

 

[SukebeDude]

Sounds strange. I doubt that it is possible that the controller tracking which is programmed can be used for the hand tracking. Also it is nearly impossible that the quest sdk can adapt it by itself to move fingers etc.

Well, i don't know why but i did 100% control it with the Oculus hand tracking, there is zero doubt about that, i went out of my way to make sure, i'd love for someone smarter than me to figure out how since i'm unable to replicate it, just know that it is without a doubt possible, i just don't know how.

 

[TheLAD]

Im running VAM via Virtual Desktop on the Quest, when you switch to hands tracking (in the occulus quest menu) then go back to VAM, the vam controllers follow your hands, but it is very limited, I havent even been able to move or do anything, except to see the vam hands moving the same as my hands?

 

[SukebeDude]

Im running VAM via Virtual Desktop on the Quest, when you switch to hands tracking (in the occulus quest menu) then go back to VAM, the vam controllers follow your hands, but it is very limited, I havent even been able to move or do anything, except to see the vam hands moving the same as my hands?

Hey, that works great, this is pretty much what i saw, you can grab nodes by closing your pinky, ring and middle finger at the same time, can just close the hand too, not sure if the index finger is doing anything but you can kinda wiggle it, but you can't really move yourself around, would be nice if a plugin could remap stuff to be more hand tracking friendly, maybe grab the world to move around by "pressing" with your the thumbs, that said i could have sworn i used link (i almost never use VD anymore since it lowers performance due to using Steam VR and has more latency) but i could have been mistaken since this clearly works, thanks for sharing.

 

[Heeman]

Im running VAM via Virtual Desktop on the Quest, when you switch to hands tracking (in the occulus quest menu) then go back to VAM, the vam controllers follow your hands, but it is very limited, I havent even been able to move or do anything, except to see the vam hands moving the same as my hands?

I have a problem with my Virtual Desktop. I tried to do it also with vd, but my controllers wont show up after the game has started. Someone know whats the problem??

 

[SukebeDude]

I have a problem with my Virtual Desktop. I tried to do it also with vd, but my controllers wont show up after the game has started. Someone know whats the problem??

That's not much to go on. Did you get the sideloaded Quest version? Do you have Steam VR installed?

 

[Heeman]

That's not much to go on. Did you get the sideloaded Quest version? Do you have Steam VR installed?

I solved it with updating VAM. Don`t know what the problem was. I think i messed up the start parameters, when i started VAM with the OpenVR bat file. Thank you anyway.

 

[Heeman]

PS: Im rly satisfied how good VD with VAM works. Its a big improvement in some cases.