• Hello Guest!

    We have recently updated our Site Policies regarding the use of Non Commercial content within Paid Content posts. Please read the new policy here.

    An offical announcement about this new policy can be read on our Discord.

    ~The VaMHub Moderation Team
  • Hello Guest!

    We posted an announcment regarding upcoming changes to Paid Content submissions.

    Please see this thread for more information.

Kinect2VAM - Full body tracking on the cheap

Plugins Kinect2VAM - Full body tracking on the cheap 1.3

Kinect_v2.0.jpg


DESCRIPTION
This is a UDP server/client combination consisting of of a PC based server app which collects and streams real time body joint tracking data from a Microsoft Kinect V2.0 3D sensor, and a VAM client plugin which receives this data and applies it to the Person atom it is attached to.
Pre-requisites are a Kinect V2.0 device fitted with with a PC adapter and its supporting runtime drivers installed on your Windows 10 PC.


Kinect2VAM_UI.jpg


CREDITS
Almadiel, for creating the original Kinect 2.0 plugin, VaMotionBridge, on which this one is based.

INSTRUCTIONS
Unpack the server app "Kinect2VAM.exe" and place it in your VAM root folder.
Run the application and allow Windows firewall to create a rule allowing local traffic only.
This is necessary to enable server/client communication.
Confirm the app is tracking your body movements by opening the skeleton display screen.
The app status bar will confirm this.
Inside VAM, load the plugin onto a Person atom and open up the plugin's UI.
Click on the "Connect" button and the server status should change to "Connected".
Align your body to match the current pose that the Person atom is in, then click on the "Calibrate" button.
This will start a 5 second countdown to allow you to re-align yourself before a "snapshot" is taken.
This stores the instantaneous differences between Kinect and VAM joint data matching your pose.
This data is then used to create realtime tracking in VAM matching that of your own body.

Please remember that there are limits to what you can expect from this plugin.
The Kinect was designed to collect body data from players directly facing the device, so don't expect it to track joints that are out of direct line of sight of the camera. i.e Hands behind your back.
That said, it can still be a useful addition and a lot of fun. Especially if you are using a VR device that does not support multiple trackers.
If you have a Kinect camera gathering dust somewhere in a cupboard, give it a new lease of life with VAM.

P.S. I have included the PC app source code for those who are paranoid about running something from an unknown source in case it contains a trojan or virus. You can check out the code and recompile it if you wish.
Author
VeeRifter
Downloads
4,004
Views
13,173
First release
Last update
Rating
4.89 star(s) 9 ratings

More resources from VeeRifter

Latest updates

  1. Kinect2VAM v1.3

    Fixes a bug that was disabling any active Possession mode when the 'Select None' button was...
  2. Kinect2VAM v1.2

    Registered the "Kinect Tracking On" toggle to make it available to triggers.
  3. Kinect2VAM v1.1

    In order to preserve the integrity of any scene, the Person atom will no longer be teleported to...

Latest reviews

Snappy and easy to set up! Any plans for face/finger tracking?
V
VeeRifter
Thanks for the review. Sorry, but face/finger tracking not possible.
Upvote 0
Okay, so I'm giving 5 stars since I think it's super cool that you made this possible. But I also want to warn other people trying this, as I did. The tracking is pretty crappy and I bought the Kinect second hand to try this, and and sending it back after toying with this for 10 minutes. If you want to use this to record some dancing motions (as was my plan) forget it, this is not the solution. But still, I really appreciate making this plugin for people who have one laying around collecting dust.
V
VeeRifter
Sorry to hear that your experience wasn't what you expected. However, I would never encourage anyone to buy a Kinect just for this purpose.
I did say this in the overview:
"Please remember that there are limits to what you can expect from this plugin.
The Kinect was designed to collect body data from players directly facing the device, so don't expect it to track joints that are out of direct line of sight of the camera. i.e Hands behind your back.
That said, it can still be a useful addition and a lot of fun. Especially if you are using a VR device that does not support multiple trackers.
If you have one of these devices gathering dust somewhere in a cupboard, give it a new lease of life with VAM"
Upvote 0
I've tried this and it works!!! No delays between real movement and vr person movement.
My kinect is very badly positioned and so I have issues with tracking the legs and feet. I will experiment further but this is nothing short of amazing.
I'm the process of researching this kinect to VaM thing I found out a project that translates kinect joints to vive pucks. The joints show up as vive pucks in steamVR. Also interesting. the project is named kinect2vr. It has a new releases called amethyst: https://github.com/KinectToVR/Amethyst-Releases. I've tried it and again, because my kinect is very badly positioned it doesn't track that well.
Anyway, I thought about something for your project that I think would be amazing to translate mocap into VaM. Why not, instread of your plugin receiving joint positions from kinect, do it from the output of OpenCV tracking the joint movements of either real life or from a file? Like I'm thinking of running opencv on a file, like that Sharon Stone iconic cross leg movie scene, and translating that to VaM!!
Upvote 0
Really nice results. Thanks!
Upvote 0
Works really well. A great app for the V2 Kinect which has a better FoV for my setup and I feel has better tracking (others claim the V1 is better but I disagree and that's a hardware issue anyway not this plugin).
It's simple to setup and use and allows almost anybody to do full body capture.
Thanks for all the work done on this great plugin.
Upvote 0
It was easy to set-up and connected right away. Surprised in desktop mode how fluent it was however in VR I found the latency to be quite high and unless paired with Snug its kind of weird to Possess a character. Also, need to adjust Max Velocity, Spring and Damper settings to make it move better but its still laggy. Overall its a really clean and cool way to bring Kinect tracking into VAM. I see it used mostly for Mo-Cap standing right in front of the Kinect camera. The limitations of the Kinect don't really allow for "natural interactions" with another character in VR.
Upvote 0
Definitely 5 stars for the effort and genious idea to bring a cheap MoCap alternative to everyone! Can´t thank you enough for this. We need way more MoCap performers and varity in VaM. Bringing the "art of MoCap" closer to beginners opens up an other door for creative minds around.

I can only imagine the furstration some people might have thinking "how can I bring my character to life? How can I make her move fast and easy"

Your pluging in a row with other fantastic plugins like Timeline giving a simple solution to everyone. Sure it has limitations. It´s obviously not the popular and most precise "vive tracking" for VaM but it is a great beginning!

We made some experiemtns with the Kinect V2 I guess 1,5 years ago. It was not successful because of the rough translation. Now with your plugin I bet its way more easy. Will experiment with it just for fun! Great job!
V
VeeRifter
Many thanks for your kind words. Please let me know your thoughts once you've had time to play with the plugin.
Upvote 0
I'll be trying this today! Excited! I'll let you know how it goes! 5 stars just for the attempt no matter what, much appreciated!
V
VeeRifter
Thank you ! I sincerely hope it lives up to your expectations. Looking forward to hearing more from you when you've tried it out.
Upvote 0
Excellent
V
VeeRifter
Wow, thanks, that was quick. Hope you still say the same after you've actually had time to try it.
Upvote 0
Back
Top Bottom