Question Using a full-body real-time mocap system (not vive) with VAM?

[clevenger]

Over the holidays I may get access to a professional mocap system, the type you see where there are a couple of dozen sensors on your body tracking your skeleton including fingers. This is the type used by professional animators, video game developers...etc.

This has real-time mocap output and they say they have plugins for Unity though I'm not sure of all the details yet. I'm wondering it would be possible to use this live mocap plugin of theirs with VAM, possibly with some custom VAM plugin code, or would support have to be written into VAM to do it?

I've seen examples of this setup being used to live animate characters in Unity's editor, but I'm not sure how big a deal it would be to get it to drive VAM without the source code.

Just to be clear, I know I can record mocap with this and import it into VAM. What I want to do is real-time animate the character like you can do with VIVE trackers, so I could live stream the output, talk to people through a VAM character...etc.

I'm a coder but I've mostly worked with Unreal Engine, no experience with Unity (yet).

So is this possible? How would I set it up?

 

[clevenger]

I did get the mocap suit, I don't have a live-into-vam setup (would be REALLY nice) but after editing the BVH files to get the bone names right I was able to get it to work.

Along the way I found out these mocap systems are getting cheap, there is a commercial one (with fingers!) for $1500 now. Sounds like a lot but if you think about it this is about the same price ballpark as buying a VIVE setup. I've seen some guys doing an open source DIY one they claim will cost $800 to make.

 

[clevenger]

Also just found http://wrnch.ai which does full body motion capture with an iphone or PC with one or more webcams. Looks pretty cool.

 

[yasparukko]

Is this free? You could use kinect as well

 

[XRWizard]

I'm really interested in this, I'd like to get my girlfriend to help out with my animations What was $1500 suit?

Wondering if BVH can be converted to Timeline keyframes?

Ideally, it would be nice to have mocap directly in vam.

 

[clevenger]

@XRWizard The $1500 suit is from Perception Neuron. All their setups you strap the sensors (up to 32) onto your body with something like a climbing harness, by the time you're done it looks like some weird bondage rig. The cheap version has all the sensors connected by wires and is a bit more fragile, more expensive ones are totally wireless. Unlike some of the other suits, these guys include the software and DO NOT charge a subscription fee. A lot of Vroid/Vtuber software works with it.

They have some .cs plugins to bring mocap into unity live but I'm not sure how to use them with VAM, I'm more experienced with Unreal than Unity.

I did manage to hack the VAM BVH plugin to import their BVH files, it's not 100% yet, but it works. Once you've got it imported you can bake it into VAM's mocap and then import that into timeline. Not an easy path, but it works.

@yasparukko The http://wrnch.ai iphone/ipad app is free, if you sign up as a developer (also free) they have Unity and Unreal plugins too. If you want to use multiple cameras on a PC you need to buy a license, not sure what it costs, but you can get a PC demo license free for a month to try it.

I tested it out the other day alongside a Kinect One and the wrnch.ai stuff worked way better. Still occasionally gets confused if you turn profile to the camera, but most of the time it works.

 

[XRWizard]

Thanks heaps for this @clevenger this is super helpful. The bvh file sound like a bit of pain, it would be so much nicer to run it straight in vam... I'll try wrnch.ai anyway. If the workflow is ok it might still be a good way to go. Cheers!

 

[Hugo]

Is this free? You could use kinect as well

That is good news only is the problem that virt a mate does not recognize that I have the program 4vr driver or something like that I will despair soon there is also no tutorial about full body tracking with kinect in virt a mate