Log in
Register
  • Hi Guest!

    We are extremely excited to announce the release of our first Beta for VaM2, the next generation of Virt-A-Mate which is currently in development.
    To participate in the Beta, a subscription to the Entertainer or Creator Tier is required. Once subscribed, download instructions can be found here.

    Click here for information and guides regarding the VaM2 beta. Join our Discord server for more announcements and community discussion about VaM2.
koboldlink

Plugins + Scripts koboldlink

Download [<1 MB]
Overview Updates (11) Reviews (5) History Discussion
bot1789 said:
Thank you very much for continuing to test this! If you notice any glitches or errors, please let me know about them as well.
I hope that I will be able to work on improving this plugin again soon.
Click to expand...
If you dont mind I would write a longer post later. Overall it works well, and I just love this brilliantly lightweight approach to AI connection. I hoped that an earlier plugin called impostor would turn to be something like this, but sadly that was axed.. So there are only thoose bulky overcomplicated bloaty options aside from this. So in short its a godsend. Please keep it up! This is the way. :)

I have some scenarios which I would like to test first to see if works or not, then understand what could be the problem and what would teoretically fix easiest.
 
I worked a few days getting XTTS and getting SPQR to work properly. I still have some problems but it's working at a lot better state. I don't have a ton of exposure to python and that was primarily what I got stuck on. I like the plugin a lot but I was disappointed it didn't take the XTTS speakers I put together and improved. My usage was to give myself as much control of my content as possible and this plugin helps a lot. The issue I left off on is the wav file does not play when and I get an error message.
 
mostvanvege said:
If you dont mind I would write a longer post later. Overall it works well, and I just love this brilliantly lightweight approach to AI connection. I hoped that an earlier plugin called impostor would turn to be something like this, but sadly that was axed.. So there are only thoose bulky overcomplicated bloaty options aside from this. So in short its a godsend. Please keep it up! This is the way. :)

I have some scenarios which I would like to test first to see if works or not, then understand what could be the problem and what would teoretically fix easiest.
Click to expand...
I agree. I also started making this plugin because I just wanted to connect Kobold to VaM without any additional complications.
 
dinkthepowerful said:
I worked a few days getting XTTS and getting SPQR to work properly. I still have some problems but it's working at a lot better state. I don't have a ton of exposure to python and that was primarily what I got stuck on. I like the plugin a lot but I was disappointed it didn't take the XTTS speakers I put together and improved. My usage was to give myself as much control of my content as possible and this plugin helps a lot. The issue I left off on is the wav file does not play when and I get an error message.
Click to expand...
Can you please show the error log so that I can understand when it occurs and how to fix it?

I also started to break down the python code of the SPQR tool to add the functionality that I wanted, but I haven’t managed to finish it yet.
 
bot1789 said:
Can you please show the error log so that I can understand when it occurs and how to fix it?

I also started to break down the python code of the SPQR tool to add the functionality that I wanted, but I haven’t managed to finish it yet.
Click to expand...
Sure!
VAMx Output
===============================
Error during mp3/wav import Could not open: file://./TextAudioTool-0.4/SPQR.TextAudioTool/files/tts_output.wav

SPQR output
=========================
Received text to speak ...
127.0.0.1 - - [25/Jan/2025 07:29:30] "POST /speak HTTP/1.1" 200 -

Kobold AI Output
====================
Processing Prompt [BLAS] (244 / 244 tokens)
Generating (25 / 100 tokens)
(EOS token triggered! ID:2)
[07:29:30] CtxLimit:270/2048, Amt:25/100, Init:0.02s, Process:103.75s (425.2ms/T = 2.35T/s), Generate:14.76s (590.4ms/T = 1.69T/s), Total:118.52s (0.21T/s)



XTTS (might be outside of the scope of what you are doing but I still will mention that XTTS didn't log anything for this attempt. I really hope I can get it to work with this though because this is remarkably close to me having control over everything.)
=================
no output

VAMx Message
===============================
KoboldLink preferences loaded
Atom type: Person
Audiofile path:TextAudioTool-0.4/SPQR.TextAudioTool/files/tts_output.wav
{"max_context_length": 2048, "max_length": 100,"memory":"You are in a never ending roleplay and play a role of the Sasha character. You are in a dialogue with Dink. Below is the description of the Sasha character. \n\nSasha is a woman \n\n[Author's note: Keep the roleplay. Your response must contain only the Sasha's short chat-style answer (one or two sentences) to Dink and nothing else. Don't use any emoji.]\n\n", "prompt":"\n### Instruction:\nHey, how are you feeling?\n### Response:\n", "quiet":"true", "stop_sequence":["### Instruction:", "### Response:"], "rep_pen": 1.1, "rep_pen_range": 256, "rep_pen_slope": 1, "temperature": 0.5, "tfs": 1, "top_a": 0, "top_k": 100, "top_p": 0.9, "typical": 1}
Kobold Processing succeeded
Response_: {"results": [{"text": "I'm always ok with you Dink!", "finish_reason": "stop", "logprobs": null, "prompt_tokens": 245, "completion_tokens": 23}]}
prompt17: \n### Instruction:\nHey, how are you feeling?\n### Response:\nI'm always ok with you Dink!
pathh: Custom\PluginData\koboldlink\files\chat_history.txt
I'm always ok with you Dink!
{"message":"I'm always ok with you Dink!","voice":"Microsoft Zira Desktop - English (United States)","output":"files/tts_output.wav"}
TTS Processing succeeded
Response: {"status": "done", "saved": "files/tts_output.wav"}
path1: TextAudioTool-0.4/SPQR.TextAudioTool/files/tts_output.wav
path: TextAudioTool-0.4/SPQR.TextAudioTool/files/tts_output.wav
Audio Loading seccesful
Audio Timer succesful
TimerCallback1
Playing Audio

I may have to reinstall something. I haven't looked into it since getting this far.
 
For spqr textaudio with coqui just use this: https://github.com/marcoseduardopm/TextAudioTool_Coqui-ai_TTS

Delete the existing files, download from the link and start as usual, it will set you up coqui too with nice voices in folder. Leave model tiny, as that is the fastest.
You will need 2-3x restart the app tough.
Also do not forget to correct the folder access in kobold config. I had to, because it wanted to look for the output audio file in the wrong place. I assume you might have some similar issue.

I do not know what the exact windows dependencies, since I tried to setup alltalk before and made the prerequisites for that. Also I have an rtx card now, but as I understood that should work on cpu as well by default.
 
mostvanvege said:
For spqr textaudio with coqui just use this: https://github.com/marcoseduardopm/TextAudioTool_Coqui-ai_TTS

Delete the existing files, download from the link and start as usual, it will set you up coqui too with nice voices in folder. Leave model tiny, as that is the fastest.
You will need 2-3x restart the app tough.
Also do not forget to correct the folder access in kobold config. I had to, because it wanted to look for the output audio file in the wrong place. I assume you might have some similar issue.

I do not know what the exact windows dependencies, since I tried to setup alltalk before and made the prerequisites for that. Also I have an rtx card now, but as I understood that should work on cpu as well by default.
Click to expand...
Worth a shot. I'll let you know how it goes.
 
@bot1789: I am afraid spqr has turned his back on vam and he ceased his work. As far I see he already deleted his github works.
I know you posted a version in description, and also the fork I mentioned is still available, but for the future do you have any other option for tts in mind?
 
mostvanvege said:
@bot1789: I am afraid spqr has turned his back on vam and he ceased his work. As far I see he already deleted his github works.
I know you posted a version in description, and also the fork I mentioned is still available, but for the future do you have any other option for tts in mind?
Click to expand...
I didn't know that SPQR decided to quit VAM. Thank you for the news. For myself, I will continue using SAPI voices, anyway. It is not difficult, I've already managed to learn the SPQR's python code, there's nothing really difficult about it. Besides, no one forbids using existing forks of the tool.
I'd also like to adapt some free neural network voice libraries, for example Silero tts. I hope that this TTS will be not too heavy to run. I want to do this during vacation, along with reworking the plugin for the actual version of VAM (now the plugin does not work with new versions of the game because of the security updates).
 
bot1789 updated koboldlink with a new update entry:

version 8

Update:
- Error 'The type 'System.Net.Request' is prohibited...' has been fixed;
- Sentiment (emotion) classification option has been added;
- Custom trigger classification option has been added (please see the plugin description for more detail).

! Before installing this updated version, please, delete all files of the old version from the Custom/PluginData/koboldlink directory, otherwise it may cause an error.

Click to expand...

Read the rest of this update entry...
 
Hey! Is it currently possible to send custom messages through triggers, like having a trigger send a specific custom text message to Kobold? It would be useful to make touch triggers and have the AI respond to touch or other stimuli in a way.

Personally I think it would be nice to have a 'Send Custom Message' setting along with a text box for the input as one of Koboldlink's triggers, but other than that, great stuff! Love the lightweight plugin
 
smthsmththrowaway said:
Hey! Is it currently possible to send custom messages through triggers, like having a trigger send a specific custom text message to Kobold? It would be useful to make touch triggers and have the AI respond to touch or other stimuli in a way.

Personally I think it would be nice to have a 'Send Custom Message' setting along with a text box for the input as one of Koboldlink's triggers, but other than that, great stuff! Love the lightweight plugin
Click to expand...
Hey!
Thank you for commenting. I appreciate any feedback, suggestions or bug reports about this plugin.
Now (version 9), it is possible to send a message to Kobold using either STT, typing it in a UIText atom input field, or typing it in the plugin UI ("Test input" page). Also, it is possible to use UI buttons (then a text attached to the UIButton will be send as a message).
About touching triggers, good idea. I will try to implement it in future versions. It will be similar to UIButton triggers, but the attached text will be sent after touching some part of the character body.
About sending a custom message, you can put this action on a UIButton (please, see the picture). There are 3 different triggers: 'Send message (manual)' will send the text from the "Test input" page, 'Send message (UIText)' will send the text typed in the UIText input field, 'Send message STT' will send the STT recognition result.
h46.jpg
 
Thanks for the reply, really appreciate it! Thanks for listening to me about this too! 😄

I was thinking about this when I tried setting up vagina, glutes, and anus touch triggers to send a custom text message tailored to each trigger to Kobold. Basically, I've set up 2 actions in each trigger in order;

1. Change the text on a UIText atom (named $AI_Text and updated triggers within the plugin to recognize it) to a custom message ( eg; "*penetrated vagina*, *penetrated anus*, *touched ass*" ).
2. Send message (UIText) to Kobold.

The first trigger worked as intended; the trigger updated the was-blank UIText with an attached message and sent it to Kobold. But when I tried activating a different trigger, it doesn't update the UIText with a new message but sends whatever's already on the UIText atom message to Kobold. I was under the impression that actions fire from top to down so in theory the UIText atom should be updated before sending the message, but I'm fairly new to tweaking stuff like this in VAM so I could be wrong. Do forgive me if I misunderstood anything 😅

Which is why I was wondering if it's possible to implement a custom message action that solely works within triggers, but if there's currently a way to create a similar function please do let me know!
 
Last edited:
smthsmththrowaway said:
Thanks for the reply, really appreciate it! Thanks for listening to me about this too! 😄

I was thinking about this when I tried setting up vagina, glutes, and anus touch triggers to send a custom text message tailored to each trigger to Kobold. Basically, I've set up 2 actions in each trigger in order;

1. Change the text on a UIText atom (named $AI_Text and updated triggers within the plugin to recognize it) to a custom message ( eg; "*penetrated vagina*, *penetrated anus*, *touched ass*" ).
2. Send message (UIText) to Kobold.

The first trigger worked as intended; the trigger updated the was-blank UIText with an attached message and sent it to Kobold. But when I tried activating a different trigger, it doesn't update the UIText with a new message but sends whatever's already on the UIText atom message to Kobold. I was under the impression that actions fire from top to down so in theory the UIText atom should be updated before sending the message, but I'm fairly new to tweaking stuff like this in VAM so I could be wrong. Do forgive me if I misunderstood anything 😅

Which is why I was wondering if it's possible to implement a custom message action that solely works within triggers, but if there's currently a way to create a similar function please do let me know!
Click to expand...
I think I understand your idea. Please check the updated version. Now the "external trigger prompt" and "external trigger" options are also available. Use the "external trigger prompt" option to define your custom message and use the "external trigger" option to send it to Kobold. Please tell if it works as intended.
I tested it together with CheesyFX.TouchMe plugin (from BodyLanguage). It works for me. You also can include {{user}} and {{char}} symbols to your predifined message. It will be replaced to the user and character names, correspondingly.
 

Attachments

  • a1.jpg
    a1.jpg
    157.8 KB · Views: 0
  • a2.jpg
    a2.jpg
    151.6 KB · Views: 0
  • a3.jpg
    a3.jpg
    110.9 KB · Views: 0
  • a4.jpg
    a4.jpg
    115.7 KB · Views: 0
  • a5.jpg
    a5.jpg
    106.9 KB · Views: 0
Thanks! It works as intended for me 😄 Being able to set up custom messages for each trigger makes things a breeze, thank you!

Though my VaM often crashes when I sent messages through the external responses thing, though I'm pretty sure I was just using a fairly heavy model for Kobold haha, will let you know if I run into anything.
 
smthsmththrowaway said:
Thanks! It works as intended for me 😄 Being able to set up custom messages for each trigger makes things a breeze, thank you!

Though my VaM often crashes when I sent messages through the external responses thing, though I'm pretty sure I was just using a fairly heavy model for Kobold haha, will let you know if I run into anything.
Click to expand...
Thank you for testing and feedback!

As for the crashes, yes, I know about this issue, but I haven't been able to fix it yet. I think they happen equally with any options of setting the prompt, not just with the "external trigger", since they appear in the part of the code that all these options use.
I think the crashes may appear because the plugin loads the TTS generated wav audio file in a way that VaM doesn't like. I will try to fix this part of the code in future versions to reduce the probability of the crashes.

In the current version, to reduce the probability of a crash when loading a wav file by the plugin, you can try the following:
- Run VaM and Koboldcpp on different computers (see “LAN connection” and “cloudflare tunnel” sections in the plugin overview); this helps, as the probability of crashing seems to depend on the PC load;
- Increase the "Audio playback delay, s";
- Uncheck the SpeechOn toggle and play the wav after some time has passed after it has been loaded manually (see the “Tips and known bugs and restrictions” section in the plugin overview);
- Uncheck the SpeechOn toggle, play wav using TextAudioTool and transfer this sound to the atom's head audio source using the virtual audio cable (it is possible with my patch for TextAudioTool: https://github.com/Musa-step/kobold-sharing/
how to use virtual audio cable see here, section “Voice and Lip Synchronization”: https://hub.virtamate.com/resources...tar-for-ai-running-through-sillytavern.42634/).

Once again, I understand that all the above are just “crutches” 😄 I will try to figure out how to solve the problem with crashes in future versions.
 
Last edited:
bot1789 said:
Thank you for testing and feedback!

As for the crashes, yes, I know about this issue, but I haven't been able to fix it yet. I think they happen equally with any options of setting the prompt, not just with the "external trigger", since they appear in the part of the code that all these options use.
I think the crashes may appear because the plugin loads the TTS generated wav audio file in a way that VaM doesn't like. I will try to fix this part of the code in future versions to reduce the probability of the crashes.

In the current version, to reduce the probability of a crash when loading a wav file by the plugin, you can try the following:
- Run VaM and Koboldcpp on different computers (see “LAN connection” and “cloudflare tunnel” sections in the plugin overview); this helps, as the probability of crashing seems to depend on the PC load;
- Increase the "Audio playback delay, s";
- Uncheck the SpeechOn toggle and play the wav after some time has passed after it has been loaded manually (see the “Tips and known bugs and restrictions” section in the plugin overview);
- Uncheck the SpeechOn toggle, play wav using TextAudioTool and transfer this sound to the atom's head audio source using the virtual audio cable (it is possible with my patch for TextAudioTool: https://github.com/Musa-step/kobold-sharing/
how to use virtual audio cable see here, section “Voice and Lip Synchronization”: https://hub.virtamate.com/resources...tar-for-ai-running-through-sillytavern.42634/).

Once again, I understand that all the above are just “crutches” 😄 I will try to figure out how to solve the problem with crashes in future versions.
Click to expand...
Haha, no problem man, just a heads up. Other than that, you're doing god's work dude, this is gonna be the best lightweight, no nonsense plugin for AI integration on VaM.

Take your time and keep it up! 🤘
 
DEADMAN2000 said:
any plans for other tts , koboldcpp has built in tts why not use those instead or using another app with koboldcpp it supports voice clone too
Click to expand...
Hi!
Yes, you are right, the latest version of koboldcpp does indeed have TTS support (OutetTTS) and the corresponding API endpoint (/v1/audio/speech). Then I will add this option next. Thank for suggestion!
About adding SileroTTS in the previous update, I personally just like it. Plus it generates pretty fast.
 
bot1789 said:
Hi!
Yes, you are right, the latest version of koboldcpp does indeed have TTS support (OutetTTS) and the corresponding API endpoint (/v1/audio/speech). Then I will add this option next. Thank for suggestion!
About adding SileroTTS in the previous update, I personally just like it. Plus it generates pretty fast.
Click to expand...
looking forward to it thank you for the great work , any chance youll add ui adjustments i use DetachedUI with this i wish it had a monochrome theme instead of pink hope you consider this also
 
DEADMAN2000 said:
looking forward to it thank you for the great work , any chance youll add ui adjustments i use DetachedUI with this i wish it had a monochrome theme instead of pink hope you consider this also
Click to expand...
Is DetachedUI a plugin name? I use Plugin UI Undocker by Stopper.
Regarding the customizable background color, good idea. I will try adding it.
 
You must log in or register to reply here.
Back
Top Bottom