Plugins koboldlink

[mostvanvege]

Thank you! I'm also interested in it. I will try

Hi there!

It is http simply. I used the community template. The solution was to move the ised ports in template settings to TCP field. That way you will get an IP, not just a proxy and the forwarded ports for them.

It tested them since then with ooba and spqr alive plugin works too. I think A1111 should do too. So you can get a full fledged AI experience for vam for about a $ per hour.

Since I use amd gpu I hoped it could work for a long time, but never really digged in.

If you need more advice let me known

 

[bot1789]

bot1789 updated koboldlink with a new update entry:

v6

A couple of small fixes:
- A bug related to text processing has been fixed;
- The "SpeachOn" toggle to turn off the speaking has been added. This can be useful to improve the stability of the plugin (see the plugin description for more details.

Read the rest of this update entry...

 

[giraffeman2006]

Hello, I am having an issue with running the plugin.
When I try to generate a message to the AI. It keeps returning with a remote server error saying: [500] Internal Server Error. I am running the KoboldCpp 1.74.yr0-ROCm. The plugin and VAM is up to date. I am running KoboldAI locally.
This plugin was working fine just before. What could be the issue and is there a way to solve the problem? Many thanks.

 

[bot1789]

Hello, I am having an issue with running the plugin.
When I try to generate a message to the AI. It keeps returning with a remote server error saying: [500] Internal Server Error. I am running the KoboldCpp 1.74.yr0-ROCm. The plugin and VAM is up to date. I am running KoboldAI locally.
This plugin was working fine just before. What could be the issue and is there a way to solve the problem? Many thanks.

Hello
Some symbols in the prompt can cause a koboldcpp server error. Could this be the case?
For example, I noticed that in the current version (v6), the presence of quotation marks ("...") in the message or in the memory causes an error. If you change the memory contents to the default (to do this, you need to delete all files from the directory ".../Custom/PluginData/koboldlink" and update the memory or reload the plugin) and send the default message "Hello! How are you recently?", does the server also return an error?

 

[giraffeman2006]

Hello
Some symbols in the prompt can cause a koboldcpp server error. Could this be the case?
For example, I noticed that in the current version (v6), the presence of quotation marks ("...") in the message or in the memory causes an error. If you change the memory contents to the default (to do this, you need to delete all files from the directory ".../Custom/PluginData/koboldlink" and update the memory or reload the plugin) and send the default message "Hello! How are you recently?", does the server also return an error?

Its appears you are correct.
The memory I copied over from another card appeared to have the quotation marks you mentioned and it seemed to cause the problem.
I have removed the quotation marks and now it seems to work just as intended. I appreciate your help. Many thanks.

 

[bot1789]

Its appears you are correct.
The memory I copied over from another card appeared to have the quotation marks you mentioned and it seemed to cause the problem.
I have removed the quotation marks and now it seems to work just as intended. I appreciate your help. Many thanks.

Thank you for your feedback! I'm going to fix this bug in the next version.

 

[SkyGirl]

Hey,

Is there likely to be tutorials or documents for stupid people? Im about 1/2 hours in and I...Am...struggling...

 

[bot1789]

Hey,

Is there likely to be tutorials or documents for stupid people? Im about 1/2 hours in and I...Am...struggling...

You have the point. I plan to make a more detailed and complete tutorial, as well as make the plugin interface more easy-to-understand, after the release of the next version.
Maybe you can tell me which aspects of using the plugin need to be explained first?
For now you can check videos on my YouTube channel with examples of using the KoboldLink plugin. Maybe they will give some idea:

Abe Takakazu

Share your videos with friends, family, and the world

www.youtube.com

 

[mostvanvege]

Hi,
if you still cooking this may I ask if it is possible to use multiple instances? I mean using the plugin on 2 characters in the scene and with different prompts? I know it will not be a group chat per definiton, but still could be handy.
I tried alive, voxta already, but keep coming back to this due to simplicity

 

[bot1789]

Hi,
if you still cooking this may I ask if it is possible to use multiple instances? I mean using the plugin on 2 characters in the scene and with different prompts? I know it will not be a group chat per definiton, but still could be handy.
I tried alive, voxta already, but keep coming back to this due to simplicity

Hi,
I'm happy that you're still here : )
Do you want two characters to talk to you and each other in turn, like a group chat in SillyTavern? Yes, I think this could be a cool feature and it wouldn't be that hard to implement.

I know it's been a while since there's been an update, but I plan to continue working on this over the Christmas break

 

[mostvanvege]

Hi,
I'm happy that you're still here : )
Do you want two characters to talk to you and each other in turn, like a group chat in SillyTavern? Yes, I think this could be a cool feature and it wouldn't be that hard to implement.

I know it's been a while since there's been an update, but I plan to continue working on this over the Christmas break

Something like that yes. Or just let edit which memory file and chatlog to use the plugin. Right now made some rar for different characters and just unpack and reload, but I am wondering if that could be made easier.
I am glad you did not abbandon the project!

 

[bot1789]

bot1789 updated koboldlink with a new update entry:

version 7

User interface and file management system updates:
- Now it is possible to use in-game UItext atoms to enter the player's message and display the response (please put the SceneInput plugin to an UItext atom to be able to input the text; see the plugin description for more details);
- It is possible to use in-game UIButton atoms to send messages and active other plugin triggers (see the plugin description for more details); sound repeat trigger is now available for external triggers
- New...

Read the rest of this update entry...

 

[mostvanvege]

bot1789 updated koboldlink with a new update entry:

version 7



Read the rest of this update entry...

Thank you very much!!

 

[BopDobop]

Is there a way how to make this work with KoboldCCP without need for SPQR plugin?
I have AMD card so Im using this release: https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr1-ROCm
Its standalone exe and I can run with it language model, image model, whisper STT and microsoft natural voice TTS
I tried to use this plugin with it but it doesnt work even with correct url specified.

 

[bot1789]

Is there a way how to make this work with KoboldCCP without need for SPQR plugin?
I have AMD card so Im using this release: https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr1-ROCm
Its standalone exe and I can run with it language model, image model, whisper STT and microsoft natural voice TTS
I tried to use this plugin with it but it doesnt work even with correct url specified.

Thank you for your comment! As far as I know, vanilla koboldcpp does not have STT and TTS built in. For this reason, the current version of koboldlink (v7) uses the SPQR.TextAudioTool to provide STT/TTS. Koboldlink can still work without SPQR.TextAudioTool, there will just be no voice. I will try to look into the ROCm release you recommended (just have no idea about it yet) and add support of its built-in STT/TTS, if this is not too difficult.

 

[BopDobop]

As far as I know, vanilla koboldcpp does not have STT and TTS built in.

You welcome.
Latest version of koboldccp, at least one which I posted above.
can have both. I'm using ggml-large-v3.bin whisper model for stt, it works perfect
And for tts Im using Microsoft TTS, with hack app so that it can use Neural voices, which are much much better than default Windows 11 one.
So it can TTS as well, rather fast.
Note that I dont use install from git one, I use standalone exe which can run many things, and this one utilizes AMD card:
https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr1-ROCm
Im running it on RX 7900 XT

 

[sikici3162]

Thanks for your works ıs there any plugin same like this but for SillyTavern?

 

[bot1789]

You welcome.
Latest version of koboldccp, at least one which I posted above.
can have both. I'm using ggml-large-v3.bin whisper model for stt, it works perfect
And for tts Im using Microsoft TTS, with hack app so that it can use Neural voices, which are much much better than default Windows 11 one.
So it can TTS as well, rather fast.
Note that I dont use install from git one, I use standalone exe which can run many things, and this one utilizes AMD card:
https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr1-ROCm
Im running it on RX 7900 XT

Thank you. I checked this release. Now I understand your comment better. The thing is that the application that accesses TTS and STT modules is not KoboldCPP (server, backend), but KoboldAI Lite (browser application, client/frontend). Koboldcpp-server works only with text variables, it does not generate sound files (they are generated by the KoboldAI Lite after it receives the LLM response from the koboldcpp server). Koboldlink plugin is just also a simple frontend for koboldcpp. It simply sends a "generate" request to the koboldcpp backend. However it cannot use any of KoboldAI Lite's features, including TTS/STT, since the latter does not have an API. It seems that it is not possible to implement SST/TTS call inside a VaM script (VaM imposes quite strict restrictions on interaction with other applications), so SPQR decided to implement a separate HTTP server (SPQR.TextAudioTool) to add this functionality to VaM, which I also use.

 

[bot1789]

Thanks for your works ıs there any plugin same like this but for SillyTavern?

Thank you for the question. I also tried to use SillyTaven with VaM at first. However, the thing is that SillyTavern is a frontend for Koboldсpp and other similar LLM engines. And as far as I know, SillyTavern does not have an API for handling requests from other applications.
However, you can still use SillyTavern with VaM in the way I described here:

Using VaM to create a 3D avatar for AI running through SillyTavern - Guides -

What is it? This guide is about using VaM to create a 3D avatar for an AI running via SillyTavern. The SillyTavern application is a client for backend applications with inference engines such as oobabooga/text-generation-webui api and KoboldAI...

hub.virtamate.com

 

[ryohei3d]

Thanks for the plugin! I think AI integration in VaM will soon be essential and you'll be a forerunner. I wanted to ask if you’d consider adding support for KoboldCPP launched without a model (possible in the latest release), hooked up to OpenRouter (or other APIs). Since VaM is heavy to run, offloading the LLM seems like a good option.

I’ve been testing this setup (KoboldCPP hooked to OpenRouter via API, which works fine), but when using it with your plugin, KoboldCPP processes the input but doesn’t generate an output. The Kobold console says, “Warning: KCPP text generation not initialized!” while VaM error log shows “Server refused the connection.”

Do you think this could be resolved through the plugin, or is it a KoboldCPP limitation? Either way, thanks again for your work, it’s much appreciated!

 

[bot1789]

Thanks for the plugin! I think AI integration in VaM will soon be essential and you'll be a forerunner. I wanted to ask if you’d consider adding support for KoboldCPP launched without a model (possible in the latest release), hooked up to OpenRouter (or other APIs). Since VaM is heavy to run, offloading the LLM seems like a good option.

I’ve been testing this setup (KoboldCPP hooked to OpenRouter via API, which works fine), but when using it with your plugin, KoboldCPP processes the input but doesn’t generate an output. The Kobold console says, “Warning: KCPP text generation not initialized!” while VaM error log shows “Server refused the connection.”

Do you think this could be resolved through the plugin, or is it a KoboldCPP limitation? Either way, thanks again for your work, it’s much appreciated!

Hello! Thank you for your support!
I still believe that using AI with VaM will remain a narrow niche. Moreover, there are much more advanced AI plugins (Voxta, Alive, AI Chat Plugin by vamX). I think KoboldLink plugin is more for just getting into the AI topic.

It seems that KoboldCPP got so many updates in 2024 that it is not easy to keep track of everything. Regarding KoboldCPP launched without a model, I checked the latest release (koboldcpp-1.80.3), but unfortunately I did not find such an option. Can you, please, explain which buttons to press to run KoboldCPP this way? I also want to test it.

If the option you are talking about is only available in the browser interface (KoboldAI Lite) and not in the server settings window, then it is not possible to use it with this plugin, since the plugin can only send requests to the KoboldCPP server and cannot communicate with the browser frontend part.

Currently, in order not to overload the PC running VaM with the AI calculations, I run KoboldCPP with a model on another PC and connect it to the first one either via a local network or via a Cloudflare tunnel (checkbox "Remote tunnel" on "Network" tab in KoboldCPP).

 

[ryohei3d]

Hello! Thank you for your support!
I still believe that using AI with VaM will remain a narrow niche. Moreover, there are much more advanced AI plugins (Voxta, Alive, AI Chat Plugin by vamX). I think KoboldLink plugin is more for just getting into the AI topic.

It seems that KoboldCPP got so many updates in 2024 that it is not easy to keep track of everything. Regarding KoboldCPP launched without a model, I checked the latest release (koboldcpp-1.80.3), but unfortunately I did not find such an option. Can you, please, explain which buttons to press to run KoboldCPP this way? I also want to test it.

If the option you are talking about is only available in the browser interface (KoboldAI Lite) and not in the server settings window, then it is not possible to use it with this plugin, since the plugin can only send requests to the KoboldCPP server and cannot communicate with the browser frontend part.

Currently, in order not to overload the PC running VaM with the AI calculations, I run KoboldCPP with a model on another PC and connect it to the first one either via a local network or via a Cloudflare tunnel (checkbox "Remote tunnel" on "Network" tab in KoboldCPP).

Thank you for the quick response!

In KoboldCPP 1.80.3, there’s a checkbox in the Model Files tab that allows launching without a local model loaded (screenshot attached). However, this setup requires starting Kobold Lite in the browser to configure OpenRouter or other APIs after launch in order to chat. Based on your reply, I suspect this might be the root of the issue, and it may not be possible to offload LLMs this way. Bad luck!

However, one additional detail I noticed is that, in this setup, the KoboldLink plugin in VaM still sends the request and displays the text bubble after sending an input, but the bubble itself is empty. It seems like something interesting might be happening here regarding connections. I've attached a screenshot showing the general situation in VaM's log.

Thanks again for taking the time to look into this!

 

[bot1789]

Thank you for the quick response!

In KoboldCPP 1.80.3, there’s a checkbox in the Model Files tab that allows launching without a local model loaded (screenshot attached). However, this setup requires starting Kobold Lite in the browser to configure OpenRouter or other APIs after launch in order to chat. Based on your reply, I suspect this might be the root of the issue, and it may not be possible to offload LLMs this way. Bad luck!

However, one additional detail I noticed is that, in this setup, the KoboldLink plugin in VaM still sends the request and displays the text bubble after sending an input, but the bubble itself is empty. It seems like something interesting might be happening here regarding connections. I've attached a screenshot showing the general situation in VaM's log.

Thanks again for taking the time to look into this!

Thank you for the information. I checked it. Indeed, connecting to an external API service using this plugin is only possible if it is Kobold AI Remote API (i.e. remote KoboldCPP server). If it is another service, such as AI Horde or OpenRouter, it will not work. As far as I understand, this is a fundamental limitation imposed by the VaM engine. The only way to solve this problem is to use an intermediate local proxy server, as Voxta does (VaM connects to Voxta, and Voxta will connect, for example, to the AI Horde). I want to try to make such a proxy-server as a development of this plugin, but I still have no idea when it will be accomplished...