This simple plugin allows you to send requests from VaM to a locally running (on the same PC or on another PC in the same LAN) koboldcpp and display and voice the responses using game audio sources by means of SPQR TextAudioTool. There are basic settings for the AI rolelay. The plugin can be added to either a Person atom or an AudioSource atom. Manual and voice text input is available. Another feature is that AI requests can be sent not only by pressing the “Send Message” button in the plugin menu, but also by pressing in-game UI Buttons. In this case, the text attached to the button is sent as a request. This allows the AI to comment on the player's actions in the scene (for example, when changing poses or clothing).
It is assumed that koboldccp runs in Instruct mode. If koboldccp is running on the same computer as VaM, then in the KoboldAI URL field you need to enter the address “http://localhost:5001/api/v1/generate”. If the program is running on another computer in the local network, you need to enter the address of that local computer, for example, "http://192.168.0.107:5001/api/v1/generate".
To voice the AI responses, you need to install SPQR TextAudioTool in the directory with VaM (tested on version 0.4, the TextAudioTool-main folder is saved in the root directory of VaM). The tool can be downloaded from the following link:
https://github.com/5PQR/TextAudioTool
Unzip the archive with the TextAudioTool directly into the VaM folder. The path that the plugin uses to access the audio file created by the TextAudioTool can be edited in the pref.json file. For Text-to-Speech (TTS) to work, you need to run the file "SPQR.TextAudioTool - (standalone).bat" (before or after starting VaM).
Note: as of August 2024, due to a bug in the new PyTorch update, the TextAudioTool may be installed with an error (OSError: [WinError 126] fbgemm.dll or ModuleNotFoundError: No module named 'pyttsx3' or something like this) when using the GitHub link. As I can figure out, this error occurs because the contents of the "installer_files" folder are incompatible with the application. If the error still occurs, you can download the working build of the TextAudioTool using the link below:
drive.google.com
The character's memory and author's note can be changed by editing the corresponding text files in the Custom/PluginData/koboldlink/ directory. Symbol {{char}} changes to character name, {{user}} changes to player name (specified in pref.json file).
Some plugin parameters are not available in the plugin interface window, but can be changed in the file "Custom/PluginData/koboldlink/pref.json" (for example, the path to the audio file that the TextAudioTool creates).
Tips and known bugs and restrictions:
- You can use Plugin UI Undocker by Stopper to free up screen space when using this plugin's UI menu for text input;
- You may need to disable the "Auto Jaw Mouth Morphs" in the Auto Behaviours/Auto Systems menu to get the correct lip-synch animation;
- Also disable the "Enable auto-jaw animation" option for the VAMMoan plugin if it is used in the Person atom;
- You can change the pitch of the voice using the Pitch slider on the Head Audio tab;
- The voices available for the TextAudioTool can be checked by entering the request "http://127.0.0.1:7069/voices" into the browser search bar;
- Before installing this updated version, please, delete the configuration files of the old version from the Custom/PluginData/koboldlink directory, otherwise it may cause an error;
- Tested with TheBloke/Kunoichi-7B-GGUF model and KoboldCpp v1.66.1;
- Avoid using special characters and emojis in your requests and appearing them in AI responses. This may cause errors. I added "Don't use any emoji" instruction to the author's note, which reduces the probability of emoji appearing in AI responses;
-If your PC or file system is slow (e.g. high system load or slow hard drive), it is possible that the audio file with the new AI response will not be able to load in time and the audio file with the previous response will be played instead. In this case, it is recommended to increase the playback delay set by the slider "Audio playback delay, s";
- If glitches and crashes continue despite the maximum audio playback delay being set, turn off the "SpeechOn" toggle. In this case, the audio file will not be played immediately after loading, which should improve the stability of the plugin. In this case, to listen the voiceing of the last response, you can click the "Repeat AI response" button after the loading of the new audio file is apparently complete;
- Only connects to a local Koboldcpp API, cannot connect to the Horde API.
Example of using
The 110 Animated Sex Loops Beach scene by sxs4 is used in this example.
Another example of using
In this video, AI requests are sent both from the plugin interface and by pressing various UI buttons in the scene. The example in the video uses the SPQR Alive plugin to make the character follow the player. The character look is 2B by Xspada. The environment is from NieR and MvC ENV pack by neci.
This is a video guide on how to set up an UI button to send a generate request to Kobold:
This example uses one of the scenes from HZM Demos by hazmhox.
Another example of using
In this video, AI requests are sent both from the plugin interface and by pressing various UI buttons in the scene. The example in the video uses the SPQR Alive plugin to make the character follow the player. The character look is 2B by Xspada. The environment is from NieR and MvC ENV pack by neci.
This is a video guide on how to set up an UI button to send a generate request to Kobold:
Koboldcpp is one of the most popular and easy-to-use programs for communicating with text generating artificial intelligence (large language model, LLM):
https://github.com/LostRuins/koboldcpp
koboldcpp Quickstart guide:
https://github.com/LostRuins/koboldcpp
koboldcpp Quickstart guide:
It is assumed that koboldccp runs in Instruct mode. If koboldccp is running on the same computer as VaM, then in the KoboldAI URL field you need to enter the address “http://localhost:5001/api/v1/generate”. If the program is running on another computer in the local network, you need to enter the address of that local computer, for example, "http://192.168.0.107:5001/api/v1/generate".
From a performance point of view, it is better to run Kobold and VaM on different PCs in the same local network (then these two resource intensive applications will not slow each other down). You can connect two PCs to a local network using a cable, a Wi-Fi router or a mobile phone with a Wi-Fi router mode. When connecting both computers to a phone or a router, you need to make sure that they see each other (the names of both PCs must be displayed in the "Network" windows menu). Then, to send a request to the Koboldcpp server on another PC, you need to enter the IP-address of that local computer to the "KoboldAI URL" instead of "localhost", for example, "http://192.168.0.107:5001/api/v1/generate". You can check the IP-address of a PC using the "ipconfig" command in the Command Prompt (cmd).
To voice the AI responses, you need to install SPQR TextAudioTool in the directory with VaM (tested on version 0.4, the TextAudioTool-main folder is saved in the root directory of VaM). The tool can be downloaded from the following link:
https://github.com/5PQR/TextAudioTool
Unzip the archive with the TextAudioTool directly into the VaM folder. The path that the plugin uses to access the audio file created by the TextAudioTool can be edited in the pref.json file. For Text-to-Speech (TTS) to work, you need to run the file "SPQR.TextAudioTool - (standalone).bat" (before or after starting VaM).
Note: as of August 2024, due to a bug in the new PyTorch update, the TextAudioTool may be installed with an error (OSError: [WinError 126] fbgemm.dll or ModuleNotFoundError: No module named 'pyttsx3' or something like this) when using the GitHub link. As I can figure out, this error occurs because the contents of the "installer_files" folder are incompatible with the application. If the error still occurs, you can download the working build of the TextAudioTool using the link below:
TextAudioTool-0.4.zip
drive.google.com
The character's memory and author's note can be changed by editing the corresponding text files in the Custom/PluginData/koboldlink/ directory. Symbol {{char}} changes to character name, {{user}} changes to player name (specified in pref.json file).
Some plugin parameters are not available in the plugin interface window, but can be changed in the file "Custom/PluginData/koboldlink/pref.json" (for example, the path to the audio file that the TextAudioTool creates).
Tips and known bugs and restrictions:
- You can use Plugin UI Undocker by Stopper to free up screen space when using this plugin's UI menu for text input;
- You may need to disable the "Auto Jaw Mouth Morphs" in the Auto Behaviours/Auto Systems menu to get the correct lip-synch animation;
- Also disable the "Enable auto-jaw animation" option for the VAMMoan plugin if it is used in the Person atom;
- You can change the pitch of the voice using the Pitch slider on the Head Audio tab;
- The voices available for the TextAudioTool can be checked by entering the request "http://127.0.0.1:7069/voices" into the browser search bar;
- Before installing this updated version, please, delete the configuration files of the old version from the Custom/PluginData/koboldlink directory, otherwise it may cause an error;
- Tested with TheBloke/Kunoichi-7B-GGUF model and KoboldCpp v1.66.1;
TheBloke/Kunoichi-7B-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
-If your PC or file system is slow (e.g. high system load or slow hard drive), it is possible that the audio file with the new AI response will not be able to load in time and the audio file with the previous response will be played instead. In this case, it is recommended to increase the playback delay set by the slider "Audio playback delay, s";
- If glitches and crashes continue despite the maximum audio playback delay being set, turn off the "SpeechOn" toggle. In this case, the audio file will not be played immediately after loading, which should improve the stability of the plugin. In this case, to listen the voiceing of the last response, you can click the "Repeat AI response" button after the loading of the new audio file is apparently complete;
- Only connects to a local Koboldcpp API, cannot connect to the Horde API.
