Plugins + Scripts VAM AI

[Dlesser]

Dlesser submitted a new resource:

VAM AI - VAM AI

Preface. I recommend reading this.​

Hi everyone.


I’ve done a huge amount of work, but I’m still only at the beginning of my path.


This is the first neural network working directly inside Virt-A-Mate itself. And I am rightfully one of the first who did this in VAM.


This is a prototype, so treat it not as a final product, but as the beginning of something bigger. I started this project a long time ago. But I abandoned it because I didn’t have enough...

Read more about this resource...

 

[Dlesser]

After 10 hours of training, I noticed a problem: the neural network was using small forces, insufficient to properly control the body. The training will need some improvements and additional credit for using the forces correctly. I'll try to fix this in the next version 2.
For now, I think it's worth trying training with more Exploration Noise and increasing the force limit slider to 1200-1600.

 

[Clive Bragin]

Soo much instructions info and all useless....did you even read it before post it ? 1-st what kind of plug-in is this ? Session or atom ? 2-nd witch plug-in must be load 1-st ? 3-rd Is it need i-net connection....when you doing something and wanna share it atleast explain in some order how to be used....

 

[LeChuck99]

Soo much instructions info and all useless....did you even read it before post it ? 1-st what kind of plug-in is this ? Session or atom ? 2-nd witch plug-in must be load 1-st ? 3-rd Is it need i-net connection....when you doing something and wanna share it atleast explain in some order how to be used....

Take the big description with copy and paste in to chatgpt and ask him what this plugin does.

 

[Dlesser]

Friends, I'll take any criticism you might give me; I have thick skin. Just don't get upset. This is challenging work for me, and it really takes me a lot of time to continue and improve. There's still so much I haven't explored. I'm just sharing what I have. And if you can do better, I'd be happy to see your version or learn from you.

 

[LeChuck99]

Friends, I'll take any criticism you might give me; I have thick skin. Just don't get upset. This is challenging work for me, and it really takes me a lot of time to continue and improve. There's still so much I haven't explored. I'm just sharing what I have. And if you can do better, I'd be happy to see your version or learn from you.

the idea is not bad. i am now thinking about it, to try something similar but lesser for fuckery stuff but more like some standard animations, like random idle poses to different reactions, or like falling/slipping animations, maybe some gunshot hit reactions for action animations like those zombie shooter plugins. Instead of creating 3-4 animations and chose via state machine. Actually, i am still landing in the State Machine hell, if i am doing something half interactive stuff with some triggers, colliders etc.
Instead of train one model and mixing always already learned stuff inside against the forgetting, train single dedicated and merge them at the end. If that not working, then maybe make a Mixture-of-experts model. But this way, i would still have the already learned stuff as strong as i can get and can play around with the merges, without to have to train the entire mass of animations from zero if something vent wrong at the end.

 

[Dlesser]

the idea is not bad. i am now thinking about it, to try something similar but lesser for fuckery stuff but more like some standard animations, like random idle poses to different reactions, or like falling/slipping animations, maybe some gunshot hit reactions for action animations like those zombie shooter plugins. Instead of creating 3-4 animations and chose via state machine. Actually, i am still landing in the State Machine hell, if i am doing something half interactive stuff with some triggers, colliders etc.
Instead of train one model and mixing always already learned stuff inside against the forgetting, train single dedicated and merge them at the end. If that not working, then maybe make a Mixture-of-experts model. But this way, i would still have the already learned stuff as strong as i can get and can play around with the merges, without to have to train the entire mass of animations from zero if something vent wrong at the end.

Yeah, that kind of problem with state machines really does exist, especially if you want to build something of your own instead of just sticking to ready-made setups. But honestly, neural networks can turn into an even bigger hell for you. This is not a simple topic at all, and on top of that, it’s better to have good hardware.
I’m not trying to talk you out of it. Quite the opposite — I’d really like to see other people in the VAM scene pick this up too. The more people who start working on it, the better.
If you’re training separate actions and there aren’t that many of them, then in some cases you can just freeze layers. But that approach stops working when you need to learn a large number of different actions and try to combine all of that into one system. That’s where the real complexity starts.
The other problem is that pretty much nobody in VAM has seriously worked on this yet, unfortunately. I’m hoping that either I manage to make it all work in the end, even though it’s going to take a huge amount of time, because I’m doing this alone and I’m still learning this field myself, or that by then the technology will have advanced a lot more and it won’t take so much fucking around to make something like this on your own.

 

[LeChuck99]

Yeah, that kind of problem with state machines really does exist, especially if you want to build something of your own instead of just sticking to ready-made setups. But honestly, neural networks can turn into an even bigger hell for you. This is not a simple topic at all, and on top of that, it’s better to have good hardware.
I’m not trying to talk you out of it. Quite the opposite — I’d really like to see other people in the VAM scene pick this up too. The more people who start working on it, the better.
If you’re training separate actions and there aren’t that many of them, then in some cases you can just freeze layers. But that approach stops working when you need to learn a large number of different actions and try to combine all of that into one system. That’s where the real complexity starts.
The other problem is that pretty much nobody in VAM has seriously worked on this yet, unfortunately. I’m hoping that either I manage to make it all work in the end, even though it’s going to take a huge amount of time, because I’m doing this alone and I’m still learning this field myself, or that by then the technology will have advanced a lot more and it won’t take so much fucking around to make something like this on your own.

it depends on it, how much time have to invest into such things. I am mostly training LLMs and WAN, LTX stuff etc.
SAC / RL for actions is still new world for me. I know the principle, but i think, its a huge amount of adjusting around.
Actually i am done with my plugins for clothing painting with alpha etc, Skin Painting and an "action machine" which combines some things with sounds. Some Quality of Life thing, to predefine Triggered things, for easy reusing them. Maybe iam dropping an eye into this SAC stuff. btw. i have the hardware for it, but not the time. My approach would be to do it modular, for later merging.

 

[IIEleven11]

That description was less than ideal. I have absolutely no idea what this thing does. You had chatgpt write a novella and it did a horrible job.

If you take the all this time to make something then I think out of respect for yourself and your own time that you would also write clear and consice descriptions/instructions. This way, others can use it and help develop it if they wish.

Ultimately, its incoherent and i'm sorry if that sounds harsh. So what, i fuck shit up all the time, im the king of catastrophe. who cares, ill just fix it.

 

[Dlesser]

That description was less than ideal. I have absolutely no idea what this thing does. You had chatgpt write a novella and it did a horrible job.

If you take the all this time to make something then I think out of respect for yourself and your own time that you would also write clear and consice descriptions/instructions. This way, others can use it and help develop it if they wish.

Ultimately, its incoherent and i'm sorry if that sounds harsh. So what, i fuck shit up all the time, im the king of catastrophe. who cares, ill just fix it.

Thank you, I accept the criticism. I will improve.

 

[LeChuck99]

Thank you, I accept the criticism. I will improve.

 

[LeChuck99]

write something like this, and THEN your big description as user manual:

This plugin adds an experimental AI-like control system to Virt-A-Mate.
Instead of only playing fixed animations, it tries to learn body movement by doing.
It can:

• control parts of the body
• detect position, contact, and penetration
• react to the current scene
• train itself over time based on what works and what does not
• let the user guide or “coach” the movement while it learns
  

It is a prototype that teaches a VaM character how to move more intelligently instead of only following pre-recorded motion/animations.


It is not a finished product.
It is a prototype, needs setup and training time, patiente, testing, and can be unstable in VaM1.
So it is more like an advanced experiment than a plug-and-play tool.

 

[rutheniumm]

I really love this idea, i was trying to do this before but using an external python connection directly to VAM via a plugin. Didnt work as i expected but was fun to experiment with. You should really check out BoneReceiver it is actually pretty cool and its actually so simple.

 

[yuropeeyan]

I've just made an account just to say that your demo video is literally the stuff of nightmares, reminds me one particular I had.

Actually I'd love if you cobbled together a demo scene demonstrating how already trained models work, or at least made some sample presets for us to train them on.

Also the description is still a mess, could make it concise and use some pictures actually, place instructions first and trivia last I think (in-plugin tooltips or several example plugin presets for different stages of training for example would also do the trick, I guess).

Anyway, the plugin looks interesting, but I'm still lost on how to use it as intended.

 

[Dlesser]

I really love this idea, i was trying to do this before but using an external python connection directly to VAM via a plugin. Didnt work as i expected but was fun to experiment with. You should really check out BoneReceiver it is actually pretty cool and its actually so simple.

AI-Driven Skeleton Engine (BoneReceiver) - Plugins + Scripts -

This is an experimental project to drive Virt-A-Mate (VaM) skeleton animations using a Python-based CVAE Machine Learning engine. [ Quick Start Guide ] 1. Download the ML Skeleton Engine (Python) [...

hub.virtamate.com

This isn't "LLM for scenes" or some universal model trained "entirely on VaM." It's a PyTorch model for skeletal motion generation, specifically GRU-based CVAE—the class is called PoseGRU_CVAE in the code. It takes a history of poses and condition flags, generates the next pose, then the Python engine sends the result to VaM via UDP, and a C# plugin applies it to the controllers.

What is this model, exactly?

Architecture — CVAE + GRU;
Input — a sequence of length 20 (SEQ_LEN = 20);
Latent space — 24 (LATENT_DIM = 24);
Hidden size — 1024;
Controls only 9 bones/controllers: hip, chest, head, both arms, both legs, both knees.

How is its pose format structured:

Each target receives 9 numbers;
The first 3 are the position;
the remaining 6 are the rotation in 6D representation;
then this is converted to a quaternion and sent to VaM.

How it works at runtime:

Python reads flag.csv as a list of condition names;
maintains a GUI with sliders for these conditions;
loads the latest .pth checkpoint from the current folder;
generates the pose;
sends a packet via UDP to the IP/port from target_ip.csv;
The VaM plugin BoneRemoteReceiver listens on UDP port 9998 by default and applies coordinates/quaternions to the same 9 controllers.

So yes, it's implemented in Python, but with an important caveat:
the repo contains mostly inference/runtime, not the full training pipeline. The published files show CVAE_GEN.py, BoneReceiver.cs, START_ENGINE.bat, flag.csv, target_ip.csv, README, and a var package; there's no separate train script, dataset, or DataLoader/optimizer/loss loop in the repo. CVAE_GEN.py itself loads the completed .pth file, not trains the model.

Is it trained "on scenes"?

It's highly likely that it was trained using extracted movements/poses from VaM scenes and community assets, not "on scenes" as in some full-fledged simulator. The README explicitly states that the model was trained using community assets like Voxta Demo, Beta Scene, dance/mocap, and other free/CC scenes/assets. However, the author didn't publish the exact dataset extraction process in the repo, so I won't pretend to know his pipeline down to the last detail.

flag.csv also reveals this: it doesn't contain abstract tokens, but behavioral labels like idle, listen, think, speak, wave, dance, love, sit, missionary, sleep, and cowgirl. These look more like conventionally labeled movement modes than a free understanding of the scene.

What does this mean in simple terms?

This isn't a "smart agent," but a motion priors generator:

You specify a mixture of states/flags;
the model wanders in latent space;
decodes smooth motion;
VaM receives commands for nine controllers.

That's why it looks "simple but alive" in the video: it doesn't think; it generates believable movements based on a trained motion distribution.