Plugins + Scripts koboldlink

[DEADMAN2000]

Hello!
I think I finally fixed this issue in the latest version (v18). At least, I tested it for a while, but this issue didn't reoccur. It seems the problem was that I was trying to play audio in an async request handler, which is not a good idea in Unity (nor is invoking atom triggers in those handlers). This could lead to errors such as "mono.dll caused an Access Violation" and "Read from location XXX caused an access violation". Now I moved the audio playback to the Update() function and that helped.
...I'm not sure how optimal this solution is, although. If anyone knows how to threat it in a correct way, I'd be happy to receive advice

dunno why but with this update the ui dosent show up and i get this error
!> Exception during plugin script Init: System.InvalidCastException: Cannot cast from source type to destination type.
at KoboldLink.KoboldLink.AtomNameReplace1 (SimpleJSON.JSONClass json_proc) [0x00000] in <filename unknown>:0
at KoboldLink.KoboldLink.Init () [0x00000] in <filename unknown>:0
at MVRPluginManager.CreateScriptController (.MVRPlugin mvrp, DynamicCSharp.ScriptType type) [0x00000] in <filename unknown>:0
!> Exception during LateRestore of plugin#12_KoboldLink.KoboldLink: System.NullReferenceException: Object reference not set to an instance of an object
at KoboldLink.KoboldLink.LateRestoreFromJSON (SimpleJSON.JSONClass jc, Boolean restorePhysical, Boolean restoreAppearance, Boolean setMissingToDefault) [0x00000] in <filename unknown>:0
at Atom.LateRestore (SimpleJSON.JSONClass jc, Boolean restorePhysical, Boolean restoreAppearance, Boolean restoreCore, Boolean isSubSceneRestore, Boolean setMissingToDefault, Boolean isTheSubSceneAtom) [0x00000] in <filename unknown>:0
!> Error during attempt to load assetbundle vecterror.ColorCorrect.1:/Custom/Assets/colorcorrect.assetbundle. Not valid
!> Exception during plugin script Init: System.InvalidCastException: Cannot cast from source type to destination type.
at KoboldLink.KoboldLink.AtomNameReplace1 (SimpleJSON.JSONClass json_proc) [0x00000] in <filename unknown>:0
at KoboldLink.KoboldLink.Init () [0x00000] in <filename unknown>:0
at MVRPluginManager.CreateScriptController (.MVRPlugin mvrp, DynamicCSharp.ScriptType type) [0x00000] in <filename unknown>:0
!> Exception during plugin script Init: System.InvalidCastException: Cannot cast from source type to destination type.
at KoboldLink.KoboldLink.AtomNameReplace1 (SimpleJSON.JSONClass json_proc) [0x00000] in <filename unknown>:0
at KoboldLink.KoboldLink.Init () [0x00000] in <filename unknown>:0
at MVRPluginManager.CreateScriptController (.MVRPlugin mvrp, DynamicCSharp.ScriptType type) [0x00000] in <filename unknown>:0

 

[bot1789]

dunno why but with this update the ui dosent show up and i get this error
!> Exception during plugin script Init: System.InvalidCastException: Cannot cast from source type to destination type.
at KoboldLink.KoboldLink.AtomNameReplace1 (SimpleJSON.JSONClass json_proc) [0x00000] in <filename unknown>:0
at KoboldLink.KoboldLink.Init () [0x00000] in <filename unknown>:0
at MVRPluginManager.CreateScriptController (.MVRPlugin mvrp, DynamicCSharp.ScriptType type) [0x00000] in <filename unknown>:0
!> Exception during LateRestore of plugin#12_KoboldLink.KoboldLink: System.NullReferenceException: Object reference not set to an instance of an object
at KoboldLink.KoboldLink.LateRestoreFromJSON (SimpleJSON.JSONClass jc, Boolean restorePhysical, Boolean restoreAppearance, Boolean setMissingToDefault) [0x00000] in <filename unknown>:0
at Atom.LateRestore (SimpleJSON.JSONClass jc, Boolean restorePhysical, Boolean restoreAppearance, Boolean restoreCore, Boolean isSubSceneRestore, Boolean setMissingToDefault, Boolean isTheSubSceneAtom) [0x00000] in <filename unknown>:0
!> Error during attempt to load assetbundle vecterror.ColorCorrect.1:/Custom/Assets/colorcorrect.assetbundle. Not valid
!> Exception during plugin script Init: System.InvalidCastException: Cannot cast from source type to destination type.
at KoboldLink.KoboldLink.AtomNameReplace1 (SimpleJSON.JSONClass json_proc) [0x00000] in <filename unknown>:0
at KoboldLink.KoboldLink.Init () [0x00000] in <filename unknown>:0
at MVRPluginManager.CreateScriptController (.MVRPlugin mvrp, DynamicCSharp.ScriptType type) [0x00000] in <filename unknown>:0
!> Exception during plugin script Init: System.InvalidCastException: Cannot cast from source type to destination type.
at KoboldLink.KoboldLink.AtomNameReplace1 (SimpleJSON.JSONClass json_proc) [0x00000] in <filename unknown>:0
at KoboldLink.KoboldLink.Init () [0x00000] in <filename unknown>:0
at MVRPluginManager.CreateScriptController (.MVRPlugin mvrp, DynamicCSharp.ScriptType type) [0x00000] in <filename unknown>:0

Thank you for testing it.
This initialization error may be due to the plugin attempting to use configuration files remaining from older versions of the plugin, or because other information from older versions of the plugin remains in the scene.
Can you, please, try deleting all files from the [VaM location]/Custom/PluginData/koboldlink folder and restarting the plugin?
You can also try running the plugin in a new scene that did not use previous versions of the plugin.

 

[DEADMAN2000]

Thank you for testing it.
This initialization error may be due to the plugin attempting to use configuration files remaining from older versions of the plugin, or because other information from older versions of the plugin remains in the scene.
Can you, please, try deleting all files from the [VaM location]/Custom/PluginData/koboldlink folder and restarting the plugin?
You can also try running the plugin in a new scene that did not use previous versions of the plugin.

thank you its working now no problems

 

[bot1789]

thank you its working now no problems

Great! Thank for your feedback!
I will try to avoid the initialization error caused by incomplete settings files from the previous version in future versions.

 

[Battou]

Hi. Great asset! But I have a question. Is it possible to use STT with Koboldcpp TTS? I see that Koboldcpp has STT using whisper. But the record mic in VAM plugin UI gives me an error.

 

[bot1789]

Hi. Great asset! But I have a question. Is it possible to use STT with Koboldcpp TTS? I see that Koboldcpp has STT using whisper. But the record mic in VAM plugin UI gives me an error.

Hello!
Thank you for trying it!
Unfortunately, in the current version (v18), STT (speech-to-text) only works with TextAudioTool (which also uses Whisper):

GitHub - Musa-step/koboldlink-tools: KoboldLinkTools project repository.

KoboldLinkTools project repository. Contribute to Musa-step/koboldlink-tools development by creating an account on GitHub.

github.com

But thanks for your suggestion, I will try to add Koboldcpp STT support in the next version.

 

[Battou]

Function calling is not working for me for some reason.
First I get an error in Koboldcpp:

Body Err: b'{"max_context_length": 4096, "max_length": 18,"memory":"", "prompt":"\\n### Instruction:\\nGiven the dialog between Master and Lizi, determine the best available function from the list that Lizi should execute.If none of the available functions match, choose \'fn.noop()\'. Your response should only contain the name of the function and nothing else.\n\nBased on the following, determine which function to call from the list:\n\nMaster:\nPlease sit on your knees\nLizi:\nI like that Master it feels so nic\n\nThe list of functions available to Lizi:\n- \'fn.noop()\': None of the other functions matching the reply\n- \'fn.sitdown1()\': when Master asks Lizi to sit on her knees\n\n\nPlease do not introduce your own new functions. Select only functions from the list above.\n\n\\n### Response:\\n", "quiet":"true", "stop_sequence":["### Instruction:", "### Response:"], "rep_pen": 1.1, "rep_pen_range": 256, "rep_pen_slope": 1, "temperature": 0.5, "tfs": 1, "top_a": 0, "top_k": 100, "top_p": 0.9, "typical": 1, "function_calling":"true"}'

Second according to this output in Kobold it see my button fn.sitdown1() with a condition "when Master asks Lizi to sit on her knees", and I have input exactly that "Please sit on your knees", but it says that this function is not matching the reply.

How am I supposed to use this to activate actions with prompts?

 

[bot1789]

Function calling is not working for me for some reason.
First I get an error in Koboldcpp:

Body Err: b'{"max_context_length": 4096, "max_length": 18,"memory":"", "prompt":"\\n### Instruction:\\nGiven the dialog between Master and Lizi, determine the best available function from the list that Lizi should execute.If none of the available functions match, choose \'fn.noop()\'. Your response should only contain the name of the function and nothing else.\n\nBased on the following, determine which function to call from the list:\n\nMaster:\nPlease sit on your knees\nLizi:\nI like that Master it feels so nic\n\nThe list of functions available to Lizi:\n- \'fn.noop()\': None of the other functions matching the reply\n- \'fn.sitdown1()\': when Master asks Lizi to sit on her knees\n\n\nPlease do not introduce your own new functions. Select only functions from the list above.\n\n\\n### Response:\\n", "quiet":"true", "stop_sequence":["### Instruction:", "### Response:"], "rep_pen": 1.1, "rep_pen_range": 256, "rep_pen_slope": 1, "temperature": 0.5, "tfs": 1, "top_a": 0, "top_k": 100, "top_p": 0.9, "typical": 1, "function_calling":"true"}'

Second according to this output in Kobold it see my button fn.sitdown1() with a condition "when Master asks Lizi to sit on her knees", and I have input exactly that "Please sit on your knees", but it says that this function is not matching the reply.

How am I supposed to use this to activate actions with prompts?

Thank you for testing it and reporting the error.
It looks like it is my mistake. I only tested this part with the horde. It seems the local koboldcpp does not accept the single quote character ( ' ) in the prompt, while the horde is okay with this prompt.
I think I should test any new functionality with both the local kobold and the horde.
I will fix it soon.