Question Rokoko Motion capture in VAM

[TheDJin]

Hi, everyone.
The purpose of this post is to know if there was a way to have the complete motion capture of a system like Rokoko's in Vam (body and fingers included in the capture).

Just to explain. Rokoko is one of the cheapest motion capture systems on the market.
And I sometimes find suits on ebay for very interesting prices.
Hence my question.
What if there was a way to have a complete motion capture system that could be streamed into VAM? (Real time or not)

Thanks for your feedback.

 

[Jocks3D]

Last year I asked a question here to the VaM artists that make mocap scenes about what they use and sadly I didn't get any response. It must be a closely guarded secret, or maybe the mocap artists don't want any competition.

It would be great if people like @ReignMocap, @KittyMocap and others could share their mocap equipment and process because many people like @TheDJin and myself might make our own mocap scenes to share with the community.

 

[ReignMocap]

Last year I asked a question here to the VaM artists that make mocap scenes about what they use and sadly I didn't get any response. It must be a closely guarded secret, or maybe the mocap artists don't want any competition.

It would be great if people like @ReignMocap, @KittyMocap and others could share their mocap equipment and process because many people like @TheDJin and myself might make our own mocap scenes to share with the community.

Sorry bro. You need to do your own research on this one. There is enough information for you to figure it out. And yeah...people do not want competition. Can you blame them ?

 

[Matt_]

Last year I asked a question here to the VaM artists that make mocap scenes about what they use and sadly I didn't get any response. It must be a closely guarded secret, or maybe the mocap artists don't want any competition.

It would be great if people like @ReignMocap, @KittyMocap and others could share their mocap equipment and process because many people like @TheDJin and myself might make our own mocap scenes to share with the community.

What kind of info do you need, are you starting for m scratch?

 

[Jocks3D]

What kind of info do you need, are you starting for m scratch?

Thanks @Matt Richard, that's the kind of response I usually see here on the hub, an experienced user offering to help out someone else and share knowledge that might benefit the entire community. And yes, I am starting from scratch and would be interested in the tools, software and techniques experienced VaM mocap artists are using. The VaM hub has detailed user guides on almost every aspect of the program but no one has ever gone into detail on mocap scenes and how they are being made. This always surprised me since some of the best scenes here are mocaps. If more people knew how to make them it could help grow the community and interest in VaM.

 

[Matt_]

Thanks @Matt Richard, that's the kind of response I usually see here on the hub, an experienced user offering to help out someone else and share knowledge that might benefit the entire community. And yes, I am starting from scratch and would be interested in the tools, software and techniques experienced VaM mocap artists are using. The VaM hub has detailed user guides on almost every aspect of the program but no one has ever gone into detail on mocap scenes and how they are being made. This always surprised me since some of the best scenes here are mocaps. If more people knew how to make them it could help grow the community and interest in VaM.

First I would say its hard to get complex Mocaps and it takes time to clean up and fix even once you make them. I went the vive tracker route with buying 8 trackers with a Vive headset and two controllers. I use Embody which will assign tracker and do the setup and Timeline to add the target nodes and record the mocap. It work well but I usually do the same mocap 10 times and then goto desktop and fix bad keyframes. Its alot of work. Also each time you put on the trackers you need to redo setup. KittyMocap started with the same setup and you can see in her early stuff alot of bad keyframes that needed to be fixed. They switched to a suit with I believe is $30K and out of the range from normal people.
I started doing more stationary animations where you start with a pose and a 60 sec loop, I then record each target seperately. So ill do the head first , hit record and move the head around for 60 sec, then do maybe the hip and then a hand. Then copy that setup into another anim2 in timeline. I then make another animation. Remember to save alot. Also Mocap creation takes alot of data sometime ill have a 300mb file with just a few 60 second animations. You can use the reduce keyframe function in timeline to make this smaller. I also add a audio track and Lipsync gives me good results. Get alot of ripped audio from Pornhub.
Timbo I believe uses vive trackers and has some special gloves to do finger stuff...somehow he doesnt use Embody to assign trackers and he records directly into VAM. I like timeline since I can fix keyframes easliy.

 

[Matt_]

This is with Person 1 Head , Hip, and Knee then Person 2 Head targets in timeline. Takes a few minutes to make.

 

[Matt_]

This is in VR just target Person 1 head and hip then Person 2 head im timeline with an audio track.

 

[Matt_]

THis is with trackers but I didnt have alot of room and its hard for me to walk in VR,

 

[Matt_]

Sometimes it better to just cycle thru different facial expressions

 

[TheDJin]

First I would say its hard to get complex Mocaps and it takes time to clean up and fix even once you make them. I went the vive tracker route with buying 8 trackers with a Vive headset and two controllers. I use Embody which will assign tracker and do the setup and Timeline to add the target nodes and record the mocap. It work well but I usually do the same mocap 10 times and then goto desktop and fix bad keyframes. Its alot of work. Also each time you put on the trackers you need to redo setup. KittyMocap started with the same setup and you can see in her early stuff alot of bad keyframes that needed to be fixed. They switched to a suit with I believe is $30K and out of the range from normal people.
I started doing more stationary animations where you start with a pose and a 60 sec loop, I then record each target seperately. So ill do the head first , hit record and move the head around for 60 sec, then do maybe the hip and then a hand. Then copy that setup into another anim2 in timeline. I then make another animation. Remember to save alot. Also Mocap creation takes alot of data sometime ill have a 300mb file with just a few 60 second animations. You can use the reduce keyframe function in timeline to make this smaller. I also add a audio track and Lipsync gives me good results. Get alot of ripped audio from Pornhub.
Timbo I believe uses vive trackers and has some special gloves to do finger stuff...somehow he doesnt use Embody to assign trackers and he records directly into VAM. I like timeline since I can fix keyframes easliy.

Thanks Matt for all your experience sharing.

the 30k suit you're talking about, is the Xsens suit.
which I find very expensive. added to an annual subscription ranging from 3k to 10k to be able to use it.
adding that the company has a bad reputation for customer service.

Hence my original question about Rokoko.
which offers virtually the same level of quality.
Even better soon, when they'll release a new device called "Coil" which will allow the actor to be positioned in space exactly. all at a really affordable prices.

the suit, actually. is arround 3.5k (with fingers). wish is almost 10 time less than xsens.

for me, it's a hobby.
if there is any chance that i can use it in vam. i could afford it.

 

[Matt_]

I saw the Rokoko and it seems to have its own software which makes me think its not steam compatible and the not VAM compatible. Looks like you Mocap in their subscription based software then would have to import it if possible to VAM . I find just doing stationary animations is really easy and shying away from full Mocap

 

[Jocks3D]

Thank you @Matt Richard for all the info. I was wondering if some of the people who are doing mocap are in deed using that expensive suit. And if so, I don't understand why they wouldn't divulge this. It's not like everyone here on the hub would run out and buy one of those suits to compete with them.

For me the nut I've been unable to crack is walking or moving legs with a controller and Timeline. I can't seem to find a way to do that so it mimics my own legs. Trying to create it manually with poses and Timeline has not been successful for me.

@TheDJin I've played around with Rokoko and have exported some BVH files to use in VaM but they've been rough. I'm seriously thinking about getting one of the new Sony Mocopi mocap devices for under $500 but I'm waiting for someone to make a tutorial on how to use its BVH exports into a program like DAZ or VaM. If the Mocopi BVH files work with VaM that could be the game changer we've been waiting for.

 

[Matt_]

Start out with getting a few vive trackers (if your using base stations) . I got mine from ebay a few at a time, paid alot less used and got luck a few times. One lady had a box of VR stuff from a storage locker and wanted just to get rid of it. Make sure they come with the dongle. Get two for your feet to walk around.

 

[TheDJin]

Start out with getting a few vive trackers (if your using base stations) . I got mine from ebay a few at a time, paid alot less used and got luck a few times. One lady had a box of VR stuff from a storage locker and wanted just to get rid of it. Make sure they come with the dongle. Get two for your feet to walk around.

Thanks for this advice. I think you're right. I see that the vive trackers gives some of really good results. My interest in the Rokoko suit, is the add of the fingers tracking. it looks great.

 

[TheDJin]

Thank you @Matt Richard for all the info. I was wondering if some of the people who are doing mocap are in deed using that expensive suit. And if so, I don't understand why they wouldn't divulge this. It's not like everyone here on the hub would run out and buy one of those suits to compete with them.

For me the nut I've been unable to crack is walking or moving legs with a controller and Timeline. I can't seem to find a way to do that so it mimics my own legs. Trying to create it manually with poses and Timeline has not been successful for me.

@TheDJin I've played around with Rokoko and have exported some BVH files to use in VaM but they've been rough. I'm seriously thinking about getting one of the new Sony Mocopi mocap devices for under $500 but I'm waiting for someone to make a tutorial on how to use its BVH exports into a program like DAZ or VaM. If the Mocopi BVH files work with VaM that could be the game changer we've been waiting for.

Big thanks for your return.
Just to ask.
what do you mean by have been rough? what kind of problems?
And how did you import BVH to vam? and did rokoko exported BVH files compatible with VAM? (sorry about my questions, I'm not used to BVH files)
do the bvh import finger tracking?

and last thing.
I think Rokoko have some streaming things directly to Unity. I suppose that, someone talented in scripting can use these data to make it stream in Vam. But I'm just supposing. My knowledge in scripting is near ZERO.

 

[Jocks3D]

what do you mean by have been rough? what kind of problems?
And how did you import BVH to vam?

I could never seem to get the rigging correct. I found some methods on the internet and YouTube that described rigging a Daz Gen 2 character to a Rokoko animation but when I tried, it was never very smooth. I suppose if I spent more time on it maybe I could have made it work. I actually had better luck with Deep Motion.

As for BVH files, there is a plugin here on the hub for importing a BVH animation. And upon searching on the net you can find hundreds of BVH files that can be used with this plugin in VaM.

 

[Jocks3D]

Start out with getting a few vive trackers

I've been holding back on going the Vive route. I somehow have the feeling that as soon as I invest in Vive gear someone will come out with a cheaper and better mocap solution. There seems to be a lot of activity in mocap development lately.

 

[groinc]

Many years ago, I was looking for a mocap solution for a series of shorts I wanted to do. Literally nothing worked out of the box. All solutions had lots of tweaking and cleaning after capture. Even movie grade mocap needs cleaning and tweaking. in the end, I went for the 'free option' - download a ton of free mocap files off the internet and put what you want together like a jig saw. Save out as bvh or fbx and go from there. Granted, i couldnt do anything bespoke, but in truth, most shots were folk sitting and standing anyway. Walk and run captures are pretty much free everywhere. i just altered my vision to suit my library.

I'd do the same here in vam too if I was that way inclined, but I prefer puppeteering with vams own animation capture. I get great results. I may use my iphone for face capture later but for now, its way better cheaper and no less ballache than a mocap suit

Also, I dont want to crap on reign, kitty, timbo etc, but all there stuff looks the same after a while. novelty wears off real soon. I think vam is a very limited way of demonstarting what mocap can do.

 

[Jocks3D]

I prefer puppeteering with vams own animation capture. I get great results.

I'd love to hear more about your process @groinc since my results have not been that great.

 

[VamTimbo]

Last year I asked a question here to the VaM artists that make mocap scenes about what they use and sadly I didn't get any response. It must be a closely guarded secret, or maybe the mocap artists don't want any competition.

It would be great if people like @ReignMocap, @KittyMocap and others could share their mocap equipment and process because many people like @TheDJin and myself might make our own mocap scenes to share with the community.

Short answer: real-time: NO. Post processing: Yes. You can take the output from these suits (BVH Files) and import them into VAM - but you’ll need to clean up the animation quite a bit in Timeline. YMMV.

 

[Altagate33]

THis is with trackers but I didnt have alot of room and its hard for me to walk in VR,

View attachment 276850

Can you share this scene please and thank you