Plugins + Scripts VAMCompanion

Actually, I made it work a while ago, just read u post, I will study the instructions,a lot of new things to learn, it actually listens very very well, takes a lil bit of time for the replies which is fine, for now, hehe, now VAM and the rocket scientist replies don’t match very much….idso there anyway to change that ? The way it replies ? But so far it is amazing what you have done…wish you continue developing it….Thank you very much..!!

delid42 said:

Actually, I made it work a while ago, just read u post, I will study the instructions,a lot of new things to learn, it actually listens very very well, takes a lil bit of time for the replies which is fine, for now, hehe, now VAM and the rocket scientist replies don’t match very much….idso there anyway to change that ? The way it replies ? But so far it is amazing what you have done…wish you continue developing it….Thank you very much..!!

Click to expand...

I think so, yes. In the System Prompt, describe how you want the model to behave in the scene.
Unfortunately, it is difficult to evaluate what may be causing this without knowing what your System Prompt and Role Prompt are. It may also be caused either by the specific LLM model you are using, or by the way the role or System Prompt is written.
I would recommend writing clear behavior rules in the System Prompt and checking how the model’s behavior changes. If possible, please let me know whether this helped and which model you used.
Also, to better understand your problem, you can send me your Role Prompt, System Prompt, and the model you are using in a private message, along with a more detailed description of the issue, so I can try to reproduce it.

Hey, thanks so much for your advice and tips! I'm currently learning about harnesses and have realized that, without even knowing it, I've already put together a very simple one. The AI can already trigger certain actions, but I first have to define them in the plugin and set a trigger word. When the AI types that word in *...*, the trigger is automatically activated the moment the last sentence before the trigger word is spoken. But your approach goes much further—if I understand correctly, the AI can decide for itself what to trigger without any actions being defined in advance. For example, it could automatically. trigger a timeline animation if it feels like the timing is right!? Am I understanding this correctly? Can you tell me what in-game data you send to the AI so it knows what’s happening in the game right now? Or do you do this using images?

Bplus said:

Hey, thanks so much for your advice and tips! I'm currently learning about harnesses and have realized that, without even knowing it, I've already put together a very simple one. The AI can already trigger certain actions, but I first have to define them in the plugin and set a trigger word. When the AI types that word in *...*, the trigger is automatically activated the moment the last sentence before the trigger word is spoken. But your approach goes much further—if I understand correctly, the AI can decide for itself what to trigger without any actions being defined in advance. For example, it could automatically. trigger a timeline animation if it feels like the timing is right!? Am I understanding this correctly? Can you tell me what in-game data you send to the AI so it knows what’s happening in the game right now? Or do you do this using images?

Click to expand...

I think so, yes. In the System Prompt, describe how you want the model to behave in the scene.
Unfortunately, it is difficult to evaluate what may be causing this without knowing what your System Prompt and Role Prompt are. It may also be caused either by the specific LLM model you are using, or by how the role or System Prompt is written.
I would recommend writing clear behavior rules in the System Prompt and checking how the model’s behavior changes. If possible, please let me know whether this helped and which model you used.
Also, to better understand your problem, you can send me your Role Prompt, System Prompt, and the model you are using in a private message, along with a more detailed description of the issue, so I can try to reproduce it.
And to answer your question: yes, my server provides the AI with information about what it can do in the scene on every request. Each time, new screenshots, the full dialogue, and the full available toolset are sent to LM Studio. This includes triggers and my scripts with controls that the AI can use.
The AI itself does not have memory. On every request, you need to show it what it can do and return information about what it has already done. Otherwise, it will forget. An LLM does not have its own memory. It is just a text generator, good or bad. That is why the full context, screenshots, tools, and in-game scripts it can use have to be sent again every time.

Dlesser said:

I think so, yes. In the System Prompt, describe how you want the model to behave in the scene.
Unfortunately, it is difficult to evaluate what may be causing this without knowing what your System Prompt and Role Prompt are. It may also be caused either by the specific LLM model you are using, or by the way the role or System Prompt is written.
I would recommend writing clear behavior rules in the System Prompt and checking how the model’s behavior changes. If possible, please let me know whether this helped and which model you used.
Also, to better understand your problem, you can send me your Role Prompt, System Prompt, and the model you are using in a private message, along with a more detailed description of the issue, so I can try to reproduce it.

Click to expand...

Message Sent / Thank you..!!

check this out a online model with a test interface will that work to the project ?



Voice test interface

MrVR said:

check this out a online model with a test interface will that work to the project ?

View attachment 605430

Voice test interface

HF Realtime Voice - a Hugging Face Space by smolagents

Voice chat over WebSocket against a HF speech-to-speech

huggingface.co

Click to expand...

I’m pretty sure this is not what you need at all. If you can choose a voice from a list, then it looks like the voices are predefined. My setup works differently: the voice is built from a reference WAV file that you provide yourself, and then the model speaks using that sample as the voice reference. Please read the instructions on my page, especially the part explaining where to download the XTTS model.

Error when launching VAMCompanion.2

donkilluminatti said:

View attachment 605708

Error when launching VAMCompanion.2

Click to expand...

Hi! This error happens because the program was not extracted correctly. VAMCompanion1.zip and VAMCompanion2.zip are two parts of the same program.

Please:

1. Download both ZIP files.
2. Extract both ZIP files to the same location so that they merge into one VAMCompanion folder.
3. Do not run the program directly from the ZIP file, and do not extract the two ZIP files into separate folders.
4. Run VAMCompanion.exe from the merged folder.

The folder should look approximately like this:

VAMCompanion\
----VAMCompanion.exe
----_internal\
--------select.pyd
If the error still appears, please check your antivirus or Windows Security quarantine. It may have removed the select.pyd file. You do not need to install Python or the select module separately.

Hi,
it works generally well for me on a lot of differents scenes. I only have an issue where VAM companion always use the same role prompt even with a different one in place and save restart server...something to do with the cache ? where does the role prompt are stocked ? does copy pasting in the role prompt window does that ? should i write directly in it ?

thx for this well done work nonetheless.

captainkatsura said:

Hi,
it works generally well for me on a lot of differents scenes. I only have an issue where VAM companion always use the same role prompt even with a different one in place and save restart server...something to do with the cache ? where does the role prompt are stocked ? does copy pasting in the role prompt window does that ? should i write directly in it ?

thx for this well done work nonetheless.

Click to expand...

If you want to create your own role, you need to enter it in this field. Naturally, you should first delete the existing text, and then you can decide for yourself who the neural network should role-play. It could be a forest fairy, a courtesan, or anyone else you want to interact with. It all depends on your imagination.


But I believe I already explained all of this in the instructions.

Dlesser said:

View attachment 606675

If you want to create your own role, you need to enter it in this field. Naturally, you should first delete the existing text, and then you can decide for yourself who the neural network should role-play. It could be a forest fairy, a courtesan, or anyone else you want to interact with. It all depends on your imagination.


But I believe I already explained all of this in the instructions.

Click to expand...

hi again,
yes you've made it very clear in the instructions and in this answer but that's not my issue. i may have stated it badly so i do it again here. Vam companion always use the same "rôle prompt" (screen on the right) even if another one is settled and save using the "save prompt/settings" button (left on this screen). Si i was figuring that using copy/paste in the "role prompt" windows was triggering this issue. Again if i make a new "role prompt" and use the "saveprompts/settings " button it will load and use the ancient one ....Also it's always the ancient one that comes up when using the "load setting" button enceforth my question about where are stocked those prompts that are loaded by this button ?

thx for your kindness i hope i have been more precise here

captainkatsura said:

hi again,
yes you've made it very clear in the instructions and in this answer but that's not my issue. i may have stated it badly so i do it again here. Vam companion always use the same "rôle prompt" (screen on the right) even if another one is settled and save using the "save prompt/settings" button (left on this screen). Si i was figuring that using copy/paste in the "role prompt" windows was triggering this issue. Again if i make a new "role prompt" and use the "saveprompts/settings " button it will load and use the ancient one ....Also it's always the ancient one that comes up when using the "load setting" button enceforth my question about where are stocked those prompts that are loaded by this button ?

thx for your kindness i hope i have been more precise here

Click to expand...

captainkatsura said:

hi again,
yes you've made it very clear in the instructions and in this answer but that's not my issue. i may have stated it badly so i do it again here. Vam companion always use the same "rôle prompt" (screen on the right) even if another one is settled and save using the "save prompt/settings" button (left on this screen). Si i was figuring that using copy/paste in the "role prompt" windows was triggering this issue. Again if i make a new "role prompt" and use the "saveprompts/settings " button it will load and use the ancient one ....Also it's always the ancient one that comes up when using the "load setting" button enceforth my question about where are stocked those prompts that are loaded by this button ?

thx for your kindness i hope i have been more precise here

Click to expand...

Was I able to help you?

Hi, I finally had some time to try this out. And I guess I'm doing something wrong. The connection is established, LM Studio is constantly processing, and the AI is busily taking screenshots, but I'm not getting any feedback—neither text-based nor audio. The XTTs model is loaded, and the server shows “Ready.” What am I doing wrong? So far, I've only tried it with the default scene.



Best regards

Bplus said:

Hi, I finally had some time to try this out. And I guess I'm doing something wrong. The connection is established, LM Studio is constantly processing, and the AI is busily taking screenshots, but I'm not getting any feedback—neither text-based nor audio. The XTTs model is loaded, and the server shows “Ready.” What am I doing wrong? So far, I've only tried it with the default scene.



Best regards

Click to expand...

How long does the AI remain unresponsive? Could you observe it for about ten minutes? You mean my program, right? You mentioned that you were working on something similar too.

Yeah, exactly, your program. I think it took at least 5 to 10 minutes. But I think the AI already responded—I could see in the server log that text was being generated, but I couldn't see or hear any of it in VAM. So no speech bubble or audio.

I need to think about the best way to handle this. I’m facing a dilemma. In version 2, I send screenshots with every request the server makes to the model, so it can understand what is happening around it each time and make decisions based on that. I also added instructions related to controlling FuckingReach and using triggers.
I need to figure out how to speed all of this up. I’m not sure I’ll be able to solve it quickly, because it also depends on which models you use. If the model loaded in LM Studio is large and Think mode is enabled in the server dialogue, that will increase the response time as well.

Very nice plugin, nice functions, thank you. I use poses in CheesyFX Body Language, they sometimes are broken, when thrusting is turned on by the ai ... I think, this is something I can controll with the harness. But it seems that it doesn't read out the arrousal data from the CheesyFX plugin suite and has its own data?

mmn123 said:

Very nice plugin, nice functions, thank you. I use poses in CheesyFX Body Language, they sometimes are broken, when thrusting is turned on by the ai ... I think, this is something I can controll with the harness. But it seems that it doesn't read out the arrousal data from the CheesyFX plugin suite and has its own data?

Click to expand...

My client allows you to create triggers and connect them to different in-game functions, so the AI can control them. I’m fairly familiar with CheesyFX’s Body Language plugin.
At the moment, the client lets the AI control things through triggers, but I haven’t yet implemented triggers that can read values from other sliders or triggers. I’ve already been thinking about adding this. It would be a different type of trigger and would need a separate button in the interface.
Could you also tell me how quickly the AI responds for you? Does it take seconds or minutes? The results vary greatly between users, and the response speed depends on many things, including the performance and VRAM capacity of your GPU or GPUs, as well as the AI model you load in LM Studio.
I simply don’t have enough data yet to understand how the program performs for different people or what difficulties they encounter.
I would also be glad to receive feedback about your experience using it. For example, do you dislike that the AI speaks too often or constantly, or do you feel that something else is missing?

Dlesser said:

My client allows you to create triggers and connect them to different in-game functions, so the AI can control them. I’m fairly familiar with CheesyFX’s Body Language plugin.
At the moment, the client lets the AI control things through triggers, but I haven’t yet implemented triggers that can read values from other sliders or triggers. I’ve already been thinking about adding this. It would be a different type of trigger and would need a separate button in the interface.
Could you also tell me how quickly the AI responds for you? Does it take seconds or minutes? The results vary greatly between users, and the response speed depends on many things, including the performance and VRAM capacity of your GPU or GPUs, as well as the AI model you load in LM Studio.
I simply don’t have enough data yet to understand how the program performs for different people or what difficulties they encounter.
I would also be glad to receive feedback about your experience using it. For example, do you dislike that the AI speaks too often or constantly, or do you feel that something else is missing?

Click to expand...

I've tried some LLMs and am using gemma-4-e4b-uncensored, which seemed the best option for speed and versatility (16 tokens/s) ... But I am using a second computer (my Ryzen AI 9 hx 370 Laptop) for LM Studio as server and getting a generated response fast, in some seconds because i set a token maximum for responses. The voice generation is only on CPU on my main PC and takes a while, so the answer or the reaction to the submitted pictures takes a while at this last step ... I have a RX 9070 XT and a 9800X3D on my PC ... And I am still experimenting to get the most out of this hardware.

RAM is no problem at all on both Computers, I maxed them out rather economically right before the RAM apocalypse ... Thank god.

Prompt and harness profiles could be an idea in the exe, but it really is not very much troublesom to save those settings in an editor for the moment.

Keep up the great work!