Kinect2VAM - Full body tracking on the cheap

Plugins Kinect2VAM - Full body tracking on the cheap

Download [0.01 MB]
Overview Updates (3) Reviews (9) History Discussion
V

VeeRifter

Well-known member
Featured Contributor
Messages
181
Reactions
655
Points
93
VeeRifter submitted a new resource:

Kinect2VAM - Full body tracking on the cheap - Uses a Microsoft Kinect V2.0 3D scanner to provide full bodytracking data to VAM

View attachment 67916

DESCRIPTION: This is a UDP server/client combination consisting of of a PC based server app which collects and streams real time body joint tracking data from a Microsoft Kinect V2.0 3D sensor, and a VAM client plugin which receives this data and applies it to the Person atom it is attached to.
Pre-requisites are a Kinect V2.0 device fitted with with a PC adapter and its supporting runtime drivers installed on your Windows 10 PC.
PC ADAPTER
[URL...
Click to expand...

Read more about this resource...
 
Here's what I have installed and running: K2EX, SteamVR, VaM current version, scene with atom and plugin on atom. I have gone into the Windows firewall settings and allowed KinectToVAM to have access. I have confirmed in KinectToVR 0.8.0 EX that Kinect Status, Driver Status, and SteamVR Status all show "Success" and I can see the tracking skeleton accurately in KinectToVR. I can't figure out why on the plugin it will not allow for me to connect. It just persistantly shows as Disconnected.
 
btacoses said:
Here's what I have installed and running: K2EX, SteamVR, VaM current version, scene with atom and plugin on atom. I have gone into the Windows firewall settings and allowed KinectToVAM to have access. I have confirmed in KinectToVR 0.8.0 EX that Kinect Status, Driver Status, and SteamVR Status all show "Success" and I can see the tracking skeleton accurately in KinectToVR. I can't figure out why on the plugin it will not allow for me to connect. It just persistantly shows as Disconnected.
Click to expand...
I assume that you did click on the "Connect" button in the plugins UI ?
 
Hey! nice plugin, looks very promising

Is it normal that as soon as i activate the plugin on a character the character suddenly respawn on a different place ?
 
danou said:
Hey! nice plugin, looks very promising

Is it normal that as soon as i activate the plugin on a character the character suddenly respawn on a different place ?
Click to expand...
Yes, the Person atom is placed at Unity world position 0,0,0 for calibration purposes. Once calibrated you are free to move the Person atom wherever you like, and the tracking will remain in sync. If you then later hit the reset button, the Person atom will be returned to that same position ready for a new calibration.
 
May I pose some noobie questions?
  1. I'm a little confused as to which version of the Kinect is needed for this?
    (There's eg "Microsoft Xbox One Kinect V2 Sensor" "Microsoft Xbox One Kinect Sensor", "Xbox One X S 360 Kinect V2", "Xbox one Kinect", Xbox 360 Kinect" ...)
  2. Searching on Amazon (GER), the one entry that has a thumbnail that matches the pic in the post for this plugin sez "Out of Stock"? Is this thingy out of production?
  3. Listings on EBay are around 120-140€ (140-160$): Is this a fair price? Is this gadget worth it (given that it seems out of production & plenty more modern alternatives available)
  4. Any chance for a plugin for different DIY Mocap solutions?
 
Case said:
May I pose some noobie questions?
  1. I'm a little confused as to which version of the Kinect is needed for this?
    (There's eg "Microsoft Xbox One Kinect V2 Sensor" "Microsoft Xbox One Kinect Sensor", "Xbox One X S 360 Kinect V2", "Xbox one Kinect", Xbox 360 Kinect" ...)
  2. Searching on Amazon (GER), the one entry that has a thumbnail that matches the pic in the post for this plugin sez "Out of Stock"? Is this thingy out of production?
  3. Listings on EBay are around 120-140€ (140-160$): Is this a fair price? Is this gadget worth it (given that it seems out of production & plenty more modern alternatives available)
  4. Any chance for a plugin for different DIY Mocap solutions?
Click to expand...
1. It's a Kinect V2.0 There's no mistaking thhe difference. There's a picture of it in the overview. It's definitely not the Xbox 360 which was the first one produced.
2. Probably, since the Kinect is no longer made either. I bought the power adapter recently on Amazon.uk for around £25, but your luck may vary.
3. No, I bought my pre-owned Kinect on eBay for £25, but they are no longer in production so I guess I was lucky.
This is more for those who already have one gathering dust somewhere.
4. None, I made this plugin to help someone out and I have no plans whatsoever to do any more mocap stuff (except maybe for the Xbox 360 Kinect)
But even then, that's just a remote chance, as I need a break from all this to get on with some real life stuff.
 
Great plugin, thank you!
I have issues about calibration：
1. What is the indication of a successful calibration?
2. Where is the "snapshot" stored to adjust my alignment? I do not know how to align further.
3. After successful calibration, how can I possess the atom? Any further steps needed?
 
Last edited:
gt123 said:
could you update it for kinect V1(X360)?
Click to expand...
It's not as simple as an update. These are totally different devices, both hardware and software wise. It would need a complete rewrite of both server and client code. It's unlikely I will be making a 360 version at present. Maybe sometime in future.
 
living1069 said:
Great plugin, thank you!
I have issues about calibration：
1. What is the indication of a successful calibration?
2. Where is the "snapshot" stored to adjust my alignment? I do not know how to align further.
3. After successful calibration, how can I possess the atom? Any further steps needed?
Click to expand...
1. Your VAM Person precisely tracks your body movements.
2. The 'snapshot' is stored dynamically as an array of offsets. You cannot access this directly.
if you want to fine tune by recalibrating a single or multiple joints, select those only before hitting calibrate button again (do not use the reset button as this will clear all joint offset data)
3. You can possess the Person in the normal manner either before or after calibration.
 
gt123 said:
could you update it for kinect V1(X360)?
Click to expand...
VeeRifter said:
It's not as simple as an update. These are totally different devices, both hardware and software wise. It would need a complete rewrite of both server and client code. It's unlikely I will be making a 360 version at present. Maybe sometime in future.
Click to expand...

Hi @VeeRifter, I tried to replace the Kinect SDK v2.0 in your UDP server project with v1.8 so that it can basically work for Kinect V1, except for some known problems may need you to resolve
- I don't know how to convert the BoneOrientation object in SDK v1.8 into JointOrientation in v2.0, so I simply applied a method which is obviously wrong
- the client side may also need to be tunned for Kinect V1, as the number of joints tracked by Kinect V1 and V2 is different
- the movement in the VaM is much slower than the skeleton window, not sure if there is anything can be optimized

Attached the modified project, hopefully it can help.

Besides, I noticed that the plugin doesn't work well with Embody plugin, I wonder if you can let Embody plugin control the hands and heads when Kinect2Vam handles the rest of the joints?
 

Attachments

  • KinectV1ToVam.zip
    376 KB · Views: 0
Thank you for working on this plugin.

I have bought a Kinect V2 to test out the plugin. I have got it working but with some flaws.

I have managed to record an animation with VAM animation but I have to arm every single atom for record each time. Is there no way to arm them all ?

When I tried to record mocap animation with timeline V242 (latest is V244) then I select all needed joints under targets and start record animation.
But this is not working. As soon as record starts the body will nearly freeze or flip around until record is stopped. No movements are recorded.

Seems like timeline is canceling the kinect input when record starts. Can you confirm this ?

What is the best way to record your body movements then please ?

I have tried all this in VAM Desktop mode but would like to use it in VR later.
 
Frett said:
I have managed to record an animation with VAM animation but I have to arm every single atom for record each time. Is there no way to arm them all ?
Click to expand...
No, that's the way VAM's built-in animation is set up and you can't change it.
Frett said:
When I tried to record mocap animation with timeline V242 (latest is V244) then I select all needed joints under targets and start record animation.
But this is not working. As soon as record starts the body will nearly freeze or flip around until record is stopped. No movements are recorded.
Seems like timeline is canceling the kinect input when record starts. Can you confirm this ?
Click to expand...
Sorry, I cannot confirm your findings. Latest Timeline records mocap OK for me. You have to add all joints as targets and ensure each target is selected (white highlighted) before hitting the Record button. Press Stop to finish, then immediately go to Kinect2VAM UI and uncheck 'Kinect Tracking On' before attempting playback otherwise Timeline will be fighting Kinect for control of joints.
Frett said:
What is the best way to record your body movements then please ?
Click to expand...
Personal choice. Whichever method you find easiest.
 
VeeRifter updated Kinect2VAM - Full body tracking on the cheap with a new update entry:

Kinect2VAM v1.1

In order to preserve the integrity of any scene, the Person atom will no longer be teleported to World position 0,0,0 when the plugin is first loaded or calibration is reset. Instead the Person atom will just remain in the same pose and position, wherever they are located in the scene. It is now up to the user to determine how they then pose the Person atom prior to calibration. Obviously, The requirement to physically match that pose prior to calibration still remains.
Click to expand...

Read the rest of this update entry...
 
haloworld said:
Hi @VeeRifter, I tried to replace the Kinect SDK v2.0 in your UDP server project with v1.8 so that it can basically work for Kinect V1, except for some known problems may need you to resolve
- I don't know how to convert the BoneOrientation object in SDK v1.8 into JointOrientation in v2.0, so I simply applied a method which is obviously wrong
- the client side may also need to be tunned for Kinect V1, as the number of joints tracked by Kinect V1 and V2 is different
- the movement in the VaM is much slower than the skeleton window, not sure if there is anything can be optimized

Attached the modified project, hopefully it can help.

Besides, I noticed that the plugin doesn't work well with Embody plugin, I wonder if you can let Embody plugin control the hands and heads when Kinect2Vam handles the rest of the joints?
Click to expand...
Sorry it's taken so long to respond to your post. Many thanks for your input on this. Please don't worry about Bone Orientations, the plugin only uses Position and tracking state data for VAM.
I've adjusted the size of server and client side arrays accordingly to the number of tracked joints, and also the position of each joint in that array.
I've just released a new plugin for the XBox 360 Kinect which performs quite well for me, with mimimal lag, although tracking is not the best. Others may not share the same experience. Depends a lot on the system spec. I suspect.
As regards possession, the plugin (both versions) already checks whether a joint is possesssed, and disables Kinect tracking for that joint if it is.
Mabe you could try using the built-in possession mode, or use AcidBubbles 'Improved POV' plugin instead of 'Embody'.
 
Would someone be willing to post a video of the motion capture ability of this? I'd love to try this to make some short animations, but dont know if i should go ahead and buy a kinect
 
Jackaroo said:
Would someone be willing to post a video of the motion capture ability of this? I'd love to try this to make some short animations, but dont know if i should go ahead and buy a kinect
Click to expand...
There's already a demo of the kinect v2.0 motion capture posted on the hub here made by user @Jiraiya.
 
Hi @VeeRifter, thanks a lot for your great work, I bought a K2 just for your plugin and its unbelievable. You're talented !!!
I tried a mocap and it feels very good. I tried to, to possess a second Atom to play with the first Atom and it worked pretty well.
I wonder if we can use first view creator with hand collision on VR without addin it to an atom ? just wondering... :)
 
eros75008 said:
Hi @VeeRifter, thanks a lot for your great work, I bought a K2 just for your plugin and its unbelievable. You're talented !!!
I tried a mocap and it feels very good. I tried to, to possess a second Atom to play with the first Atom and it worked pretty well.
I wonder if we can use first view creator with hand collision on VR without addin it to an atom ? just wondering... :)
Click to expand...
Thanks very much for your positive comments, much appreciated !
Not sure exactly what you mean in your question though ?
 
You must log in or register to reply here.

Similar threads

V
Plugins Kinect360ToVAM - Another cheap full body tracking option
Replies
19
Views
2K
Korielingo
Korielingo
Riddler
Textures Full Body Freckles
Replies
0
Views
925
Riddler
Riddler
LDR
Scenes Curves of the Body
Replies
1
Views
443
LDR
LDR
jjbuilds3d
Assets Larry The Body Pillow
Replies
2
Views
570
jjbuilds3d
jjbuilds3d
Fetixxer
Scenes I`m On Masturbation Duty Full Version
Replies
2
Views
289
atani
atani
Back
Top Bottom