Hey there. I’ve tested various neural nets in games before, including koboldcpp. If I recall correctly, the author disabled NSFW content in koboldcpp. Overall, it’s a solid idea. You’re a trailblazer, and you’re definitely killing it! This could potentially be a replacement for Voxta. I’d recommend checking out Mantella—here’s the link:
https://art-from-the-machine.github.io/Mantella/. Alternatively, you could try LM Studio. LM Studio lets you load any text model from huggingface.co right from its interface.
It’d also be cool to add a feature for taking screenshots so the neural net can understand what’s happening on the screen and react to it. Mantella does this already. I’ve seen it implemented in Mantella with ChatGPT for Skyrim, for example. The screen could be captured every 5, 10, or 20 seconds, and the neural net could process what’s going on.
Thank you for reviewing and for your recommendations! I will check your links.
No, there is no NSFW filter in koboldcpp! You can also choose your model: any model if you are running koboldcpp locally, or from a list if you are using AI Horde.
I'm not trying to replace Voxta. Voxta is a high quality project with a user-friendly interface. KoboldLink is just an experimental, glitchy plugin to try new ideas :D
About multimodal input, yes, great idea. I hope to add this feature someday.