Plugins BodyNet (Free)

[TBD]

8n2ybclyk982 submitted a new resource:

BodyNet - use your favourite look in any scene - neural network to match the body structure of any morph preset

Did you ever try to use your favourite appearance in a scene but it just wouldn't line up?
Now you can! This plugin matches the body structure of any appearance to any morph preset.

View attachment 295700
(15x speed)

Sounds cool but how do I use it?
It's fairly simple. The plugin comes with a scene that hooks everything up for you.
The scene contains two person atoms and the plugin is set up to match "target" (right) to "source" (left).
Just...

Read more about this resource...

 

[TBD]

The hub didn't pick up the dependency on V2.___UniversalMorphCollection___.latest for whatever reason. This is my first upload, so bear with me.

 

[TBD]

8n2ybclyk982 updated BodyNet - use your favourite look in any scene with a new update entry:

bundled morph

I ended up bundling the morph from V2.___UniversalMorphCollection___.latest because I just couldn't get it to work as a dependency. Looking at that package I don't even know who the morph's original author is so credit to "!!!", ScoobyAS, Mr. CadillacV8 and vecterror.
If any of you don't like me bundling this morph maybe we can fix the dependency.

Read the rest of this update entry...

 

[PiddlyD]

If this works out, it'll be revolutionary for VAM.

 

[TBD]

8n2ybclyk982 updated BodyNet - use your favourite look in any scene with a new update entry:

fixed training data

fixed a major bug with training data persistence and re-trained the network

Read the rest of this update entry...

 

[ouroborolapse]

Im confused about what this plugin does, so it preserves the appearance of characters even if you load another scene?

 

[TBD]

Im confused about what this plugin does, so it preserves the appearance of characters even if you load another scene?

It morphs one appearance to match the other. Once that's done you can save it as a morph or appearance preset and use it in the scene you tried to match the model from.

Example:
You found this really neat cow girl scene but if you load your own appearance the hip doesn't line up.
Just save that scene's morph preset, run it through my plugin, save the resulting morph preset and load that in the cow girl scene.

 

[TBD]

8n2ybclyk982 updated BodyNet - use your favourite look in any scene with a new update entry:

Better UI & training data

• Improved UI
• Fresh training data which will yield far better results when you hit "Apply"
• Reduced training rate to prevent overfitting
• Reduced the network depth cutting the time each generation takes roughly in half
• Training rate can be adjusted in the plugin's options (scene plugin)
  • Decrease training rate when you want to teach the network a bunch of different morphs, thereby improving its versatility and accuracy.
  • Increase training rate if you want to...

Read the rest of this update entry...

 

[qwerty134930]

How many "runs" does training go through? The progress bar is nice but doesn't really help to show how far along the training is when it just shows the progress of the steps in the current run.

 

[TBD]

How many "runs" does training go through? The progress bar is nice but doesn't really help to show how far along the training is when it just shows the progress of the steps in the current run.

Training will continue until it's stopped manually. I agree it's a bit confusing and I'll think about how to improve the UI.
You can stop training at any point. It just means the plugin hasn't updated the whole network. If you just want to let it create a morph preset for you it's fine to stop as soon as you're happy with the alignment. Waiting for the full run or even multiple is only really necessary if you want create optimized training data.

Each run teaches the network a single point of data. The question we're training the network to answer is: "What change Y should we apply to these morph sliders if the difference between the bodies is X?" A training run tries to find the best change Y for the given difference X. The change is applied and the next run trains for the resulting difference.

 

[TBD]

8n2ybclyk982 updated BodyNet - use your favourite look in any scene with a new update entry:

fixed arm length morph not loading

The bundled morph for arm length sometimes didn't load.
I updated the plugin page, too. Hopefully, it's a bit easier to understand what this actually does and what it's for.

Read the rest of this update entry...

 

[Dkahbe7]

So does this actually copy the body morph? Say I have a certain body morph (a skinny one) and I apply this to a new character (a bigger one). Does the new character become skinny? e.g. gets the same body? Or does it do something else and preserve the body of the original character?

 

[TBD]

So does this actually copy the body morph? Say I have a certain body morph (a skinny one) and I apply this to a new character (a bigger one). Does the new character become skinny? e.g. gets the same body? Or does it do something else and preserve the body of the original character?

It takes whatever morphs you give it and adjusts the following morphs until the body proportions match without touching anything else:

• Body Scale
• Upper Body Length
• Lower Body Length
• Upper Torso Length
• Legs Length
• Neck Length
• Shoulder Width
• Arms Short

 

[darkesthours443]

Hi, I have tried your system and I am left confused by the training button.
I managed to "succesfully" convert the target into the "size" of the source, but the main thing it visually only seems to change is the characters height and length of certain body parts (this is fine, this is what I was looking for, it's the next part that confuses me).

The moment I press "train" it will adjust the target on the first "step" 1/38 adjusts everything on the target to match the source.

Any additional steps it does after that, I don't see ANY visual changes to it, step 2/38 until 38/38 are completely pointless it appears to me, to which I can only conclude I am doing something wrong!

^ Ontop of that, currently the "train" button, and then specificaly step 1/38 (as the rest of the steps doesn't seem to do anything additional in my case) is visually identical to the "apply" button which is supposed to be less accurate from what I understood in your description?

 

[TBD]

Hi, I have tried your system and I am left confused by the training button.
I managed to "succesfully" convert the target into the "size" of the source, but the main thing it visually only seems to change is the characters height and length of certain body parts (this is fine, this is what I was looking for, it's the next part that confuses me).

The moment I press "train" it will adjust the target on the first "step" 1/38 adjusts everything on the target to match the source.

Any additional steps it does after that, I don't see ANY visual changes to it, step 2/38 until 38/38 are completely pointless it appears to me, to which I can only conclude I am doing something wrong!

^ Ontop of that, currently the "train" button, and then specificaly step 1/38 (as the rest of the steps doesn't seem to do anything additional in my case) is visually identical to the "apply" button which is supposed to be less accurate from what I understood in your description?

Unless you start over with a fresh network ("start from scratch"), training continues to refine the existing network. That's why you see a big change at the start of the training run. It's exactly the same as what happens when you press apply.

If you want to see more rapid changes while training you can increase the training rate in the plugin's options (it's a scene plugin). Just be aware that a high training rate causes the network to be biased towards the most recent training runs so you will get worse results when you apply it to a different morph preset.

the main thing it visually only seems to change is the characters height and length of certain body parts

That's exactly what the plugin is designed to do. It matches body proportions while preserving everything else. What else would you like to match? Maybe I can add more features.

 

[darkesthours443]

Unless you start over with a fresh network ("start from scratch"), training continues to refine the existing network. That's why you see a big change at the start of the training run. It's exactly the same as what happens if you press apply.

If you want to see more rapid changes while training you can increase the training rate in the plugin's options (it's a scene plugin). Just be aware that a high training rate causes the network to be biased towards the most recent training runs so you will get worse results when you apply it to a different morph preset.

To try to further understand your system,
Am I correct to assume that the "goal" is to get all of the lines and circles to be green or near green at the "end" of a training session?

Because other then looking visually at the actual model I am uncertain how to "judge" when a training is complete, so to speak.
Right now im kinda eyeballing it, but as I said before I'm having a hard time looking at the actual model and decide wheter the process is finished or not.

I assume I'm supposed to look at the circles and lines in a certain matter?

Edit*
Also curious what "Loss:" indicates/means, in the UI messages, at every training step.

 

[darkesthours443]

That's exactly what the plugin is designed to do. It matches body proportions while preserving everything else. What else would you like to match? Maybe I can add more features.

Perhaps a toggle to include general breasts size would be great if this is possible at all, altho this is purely a QOL thing.
It's perhaps one of the only thing that requires adjustment after running it trough the training session, to get it to match with certain scenes their animations (For example when a the model moves her hands over her chest, you'll get into trouble if your custom model has twice the breast size).
I could see the same for glutes, but only in extreme cases.

 

[TBD]

To try to further understand your system,
Am I correct to assume that the "goal" is to get all of the lines and circles to be green or near green at the "end" of a training session?

Because other then looking visually at the actual model I am uncertain how to "judge" when a training is complete, so to speak.
Right now im kinda eyeballing it, but as I said before I'm having a hard time looking at the actual model and decide wheter the process is finished or not.

I assume I'm supposed to look at the circles and lines in a certain matter?

Edit*
Also curious what "Loss:" indicates/means, in the UI messages, at every training step.

Yes, the colour indicates how well that body part lines up. The circles represent the difference in a joint's position, the lines' colour represents the difference in length.
The "Loss" is a numerical value representing how big the difference between the skeletons is. It's basically the sum of the joints' differences with some weights applied to them. This is what training runs aim to minimize.
Once the logged values stop to decrease over time, the current result is pretty much as good as it gets with the network you're using. If you're not happy with the alignment you might need to create a new network because your models are too dissimilar to the ones the network was trained on.

 

[TBD]

Perhaps a toggle to include general breasts size would be great if this is possible at all, altho this is purely a QOL thing.
It's perhaps one of the only thing that requires adjustment after running it trough the training session, to get it to match with certain scenes their animations (For example when a the model moves her hands over her chest, you'll get into trouble if your custom model has twice the breast size).
I could see the same for glutes, but only in extreme cases.

I can see the use case but this would require a separate plugin. Currently, it's optimizing for Rigidbody positions and that would have to look at a lot of colliders. There are also as many morphs you could adjust for breasts and glutes as there are models for vam. I could give it a shot but the results will probably be very hit and miss.

 

[darkesthours443]

Yes, the colour indicates how well that body part lines up. The circles represent the difference in a joint's position, the lines' colour represents the difference in length.
The "Loss" is a numerical value representing how big the difference between the skeletons is. It's basically the sum of the joints' differences with some weights applied to them. This is what training runs aim to minimize.
Once the logged values stop to decrease over time, the current result is pretty much as good as it gets with the network you're using. If you're not happy with the alignment you might need to create a new network because your models are too dissimilar to the ones the network was trained on.

THIS, answers a TON of questions, cheers my friend.

Currently I've been running the training mode for ~20 runs.
At the end of run 1 the loss was 3.4
At the end of run 12 the loss said 1.18
But now, at run 13, the loss is going up again, to 1.20
Run 14 now, loss 1.27.
Run 15 start; Loss 1.31
Run 15 End; Loss 1.11

Does this mean that after X amount of runs, since the "loss" is going up again, the end result is getting worse again?
(But at the end of run 15 it seemed to rectivify it's earlier issue)

What i'm aiming for is to get a "Loss" number that is a close as possible to "1.0" ?

 

[TBD]

THIS, answers a TON of questions, cheers my friend.

Currently I've been running the training mode for 12 runs.
At the end of run 1 the loss was 3.4
At the end of run 12 the loss said 1.18
But now, at run 13, the loss is going up again, to 1.20
Run 14 now, loss 1.27.
Run 15; Loss 1.31

Does this mean that after X amount of runs, since the "loss" is going up again, the end result is getting worse again?

What i'm aiming for is to get a "Loss" number that's as small as possible?

Usually you won't see much improvement on the second or third run. I should point out that each run calculates and applies changes to your morph. Hence, additional runs start with the result of the one before. This might be why the loss starts to increase later down the line.
You may or may not get better results if you stop after the first run, reset your morphs and start again.

However, doing a lot of training runs on the same looks overfits the network for this specific case. If you don't see good results after one or two runs my pretrained network might not be suitable. Loss values below 1.5 should be fine.

If you see significantly higher loss values maybe you could send me your morphs so I can train a more versatile network.

 

[TBD]

If you try to train for heavy outliers it might be best to start with a fresh network and crank the training rate up. That way you can train the specific case instead of re-educating the existing network.

 

[darkesthours443]

Usually you won't see much improvement on the second or third run. I should point out that each run calculates and applies changes to your morph. Hence, additional runs start with the result of the one before. This might be why the loss starts to increase later down the line.
You may or may not get better results if you stop after the first run, reset your morphs and start again.

However, doing a lot of training runs on the same looks overfits the network for this specific case. If you don't see good results after one or two runs my pretrained network might not be suitable. Loss values below 1.5 should be fine.

If you see significantly higher loss values maybe you could send me your morphs so I can train a more versatile network.

Thanks for the replies mate, I now understand what to look for and how to interpret what is considered a "good" training!

 

[TBD]

Thanks for the replies mate, I now understand what to look for and how to interpret what is considered a "good" training!

Well, "good" training would be to do a single run on every morph preset combination you can come up with while using the lowest training rate possible.

 

[darkesthours443]

Well, "good" training would be to do a single run on every morph preset combination you can come up with while using the lowest training rate possible.

Fair enough hah!
My main useage tho is, I have a couple of custom scenes/animations I created.
And now with your tool I can just take the Model I used in my custom scenes, put her as the source.
And then just load in any random other model on the target, run the training a couple times and it spits out an appearance with a loss of ~1.2 on average after 3-4 runs.

So far I've managed to "convert" about 5 random models (they range from being petite, to being huge).
And upon loading each of these converted models into my custom scenes, they all align perfectly! (Even tho, at the end of a training session, the model for example still had allot of red dots and red lines)
Hands, neck, mouth, all of it is in the correct position for the animations to not act weird.

Those 3-4 training runs I do take about ~5 minutes at a physics rate of 120, I say this because I don't mind waiting 5 minutes per model heh, as long as the result is reliable I don't see the issue in having to wait a bit longer.


There was another random mocap scene I have from I believe kitty or Zen, to succesfully get a model loaded into that, with proper allignments, DID require to have almost all of the dots/lines to be green during training.

Everytime I tried to load a model into that scene which didn't have at least 90% of the dots/lines greenish it would missalign completely during animation transitions.
The only conclusion I could draw from this is that mocap animations are allot more finicky/require more precision when it comes to the training.