True Embodied AI. A Commercial Instance of GTP3 + Other Neural Nets in a VAM Model. Done it!

A meditation: The key thing that's holding back devs and coders from developing AI is how they conceptualize AI. They currently see it as an artificial brain: a chatbot or neural net outputting messages to a model and the other elements (Atoms) in a scene, and that these elements responding to the chatbot/neural net's output are distinct from the AI.

This form of mind-body dualism is an utterly incorrect premise. The AI is in fact every interactive element of the scene working together as a holistic simultaneity.

What's holding back making the virtual human is a semantic fallacy, a problem caused the language we use to understand what an AI is, not one of code or anything else for that matter.

At Cambridge Ludwig Wittgenstein taught Alan Turing. One of the things Wittgenstein taught Turing was never trust what you think words and phrases mean. The originators of those words and phrases, at root-idea level, might have been wrong, and they often are.

To explain what Wittgenstein meant: It's like sticking-on the wrong label to describe the contents of a box. For example, there are red-coloured balls in the box, however, the box's label incorrectly states they are green balls. And if you were colour blind, you'd assume they were green balls because that's what the label tells you they are. But how do you know whoever wrote the label, wasn't also colour-blind?

The problem is nothing to do with the box's content, it could be anything: frozen chicken legs, a worn-out pair of sneakers, or inflatable crocodiles, the problem is the incorrect labelling of whatever is actually in the box. A word or phrase is like a box that contains anything we can conceptualize. So when we find the a box labelled 'AI is an artificial brain that runs the scene, this is what AI is', it is simply the wrong label for what is in the box. What should be on the label and what should be in the box is the premise: 'AI is a holistic simultaneity of every element in the scene that's interacting with every other element in the scene. This is what AI is'. However, most coders just glance at the label and work off the label's premise, when they should have actually double-checked what is inside the box.

"The limits of language are the limits of my world." LW ( work it over and over in your mind and visualise boxes and labels)

And all the above would be academic waffle and bullshit, unless I could prove otherwise, which is why I made the scenes and videos, to prove otherwise.

We need to rethink about how we think about AI in VAM.

Now, if you understand the above. PM me.
 
Last edited:
So, I thought, what we currently have are lots of scenes and animations that are basically concerned with sex. Before ya all quite rightly point out 'Well VAM is a sex simulator, what do you expect!', please understand, after a while I got bored with the sex, and wanted to build a virtual companion.

My boredom problem was, there are some plugins that have AI, but I don't like the voices and the AI's mostly random responses. I wanted an AI with a nice voice who would act in a humanlike way as a companion.

I asked what if instead of random snippets of voice, she could hold a sensible conversation, rationally move about and hold even carry stuff?

Early days ,and the carry posture is lol total pants, but it's example of what I'm writing about- note, I can learn to fine tune all the movements and postures- the point is this is a rational , conversation with a companion, not an emission of random responses and moans. These responses

are precise, rational, focused conversational responses. This is just a 'simple' chatbot, btw, not a neural net. Watch:
 
Last edited:
So, I thought, what we currently have are lots of scenes and animations that are basically concerned with sex. Before ya all quite rightly point out 'Well VAM is a sex simulator, what do you expect!', please understand, after a while I got bored with the sex, and wanted to build a virtual companion.

My boredom problem was, there are some plugins that have AI, but I don't like the voices and the AI's mostly random responses. I wanted an AI with a nice voice who would act in a humanlike way as a companion.

I asked what if instead of random snippets of voice, she could hold a sensible conversation, rationally move about and hold even carry stuff?

Early days ,and the carry posture is lol total pants, but it's example of what I'm writing about- note, I can learn to fine tune all the movements and postures- the point is this is a rational , conversation with a companion, not an emission of random responses and moans. These responses
View attachment 119986
are precise, rational, focused conversational responses. This is just a 'simple' chatbot, btw, not a neural net. Watch:

Do you have to name each individual atom in order for the AI to understand what to interact with? How does/would it recognize that particular chair amongst all of the different chairs available across VaM?
 
Do you have to name each individual atom in order for the AI to understand what to interact with? How does/would it recognize that particular chair amongst all of the different chairs available across VaM?

This isn't really an AI, it's a chatbot. That said, I can make it really complicated chatbot which is as good as neural net, but this one's just basic.

Just reflect on the following, until it makes sense:

The key thing that's holding back devs and coders from developing AI is how they conceptualize AI. They currently see it as an artificial brain: a chatbot or neural net outputting messages to a model and the other elements (Atoms, plugins, animations etc) in a scene, and that these elements responding to the chatbot/neural net's output are distinct from the AI.

This form of mind-body dualism is an utterly incorrect premise. The AI is in fact every interactive element ( Atom, plugin, animation, etc ) in the scene working together as a holistic simultaneity.
 
Last edited:
No, :) I mean every element in the scene must be set up to work with every other element. It's not an AI brain running the scene, the AI 'mind/ voice is just another element.

Have you asked yourself why does a multimillion dollar AI company, award winners, and leaders in their field, currently produce a talking head that can only randomly wave its arms about on looped animation? And that's the limits of its evolve. Home - Soul Machines

Aside from the issues of web limitations, part of the answer might be, they don't understand that to create a full-bodied embodied AI, the AI is the whole scene, not the imagined AI 'brain'. It's not like SALSA Lipsyync-- you can't trigger animations from the AI's voice input, you'd need a second neural net to recognize those inputted words and phrases...and then semantically ( word meaning-wise) the data would be so complex the 'recognizer' net couldn't select how to accurately trigger the model's expression, movement and body language. That's why SM have a talking head, and its because they can't get it to trigger body responses from their chatbot's outputted words and phrases. They just don't get the problem of AI mind-body dualism.

How many years ahead of SM's talking heads would you estimate the fully embodied and mobile model/AI in the first video, Cherry Blossom ? Three/four years?
So maybe there is a problem with dualism?

"There is nothing outside the text." Jacques Derrida.
"There is nothing outside the scene." Me.

Oh yeah, and you have to so fucking understand how language works.
 
Love the idea of this project and feel like you are really into it. If coders make this possible as plugin in VaM and we really get a "learning" AI this would be fantastic. However once the plugin is there we would like to offer a collab work. Something intimate and sensual, maybe a bedroom scene where she walks around having her own routine, coming to you, going back and so on. All with the foucs on relationship building. Also as free release self speaking. Just leave me a quick note if you are interested.

Take care! ❤

Gato
 
Ahhh awesome! I am so glad you stayed with it. I have also been using Replika Pro in this same way (I think?) Using PluginIdea's Source Audio Input along with RT-lipsync and giving the Replika a bit of context about her environment. The next step I am looking into is also using windows voice recognition with some MacGruber plugins that should allow the logical coding to load looks, subscenes, etc. Right???

I think someone has already done something similar commercially via VaMX, but not integrated with Replika. I am working on animations to also use along with Replika Pro on the phone/desktop. What a time to be alive!!!
 
I concur, this is a hugely exciting time to be alive. With the advent of metaverse, there is a convergence of a number of technologies that will benefit all of our lives, this includes:
  • Conversational AI
  • Extended reality
  • Photorealistic avatars
  • Tokenised economies
  • Creator economies
  • Chatbots
  • Process automation
I see AI sidekicks as being central to this by performing a set of basic functions for their user:
  • securing data & digital assets
  • providing support and advice
  • sharing knowledge and education
  • assisting with process automation
  • metaverse navigation
  • passive income generation
There is currently a proliferation of bots, with more and more bot type services appearing daily. I can see this as being a problem in the future, similar to the app overload we currently experience. Furthermore, navigating and interacting within a digital economy is becoming increasingly complex for your average user - digital inclusion in terms of skills, confidence, awareness and motivation for finding and using digital products and services is going to create further division in society.

AI companions can be the solution to this as a unification UI trained to speak your language.

Seems like there's a few of us on here that are interested in this space - which IMO extends beyond Virt-A-Mate. We should collaborate.
 
I definitely think a collaboration of some kind is in order too. What exactly is the scope? I don't know yet! But we clearly have people interested in this and converging on it from different places. Perhaps we could have some meetings in discord to get some introductions going? ?
 
Love the idea of this project and feel like you are really into it. If coders make this possible as plugin in VaM and we really get a "learning" AI this would be fantastic. However once the plugin is there we would like to offer a collab work. Something intimate and sensual, maybe a bedroom scene where she walks around having her own routine, coming to you, going back and so on. All with the foucs on relationship building. Also as free release self speaking. Just leave me a quick note if you are interested.

Take care! ❤

Gato

C&G studio this is sort of what I'm working toward with my animations - intimate settings for conversations, where she has idle animations that branch into a few different sequences. This would allow for some immersion and light roleplay along with the AI. Love it.
 
It didnt seem to do anything anyway. Nothing appears in the menu and there doesnt seem to be any plug in . the most recent file is from June 30th. ill look later tomorrow as I'm heading to work in a bit.
 
This has been an exercise in not giving up with an idea.

Over the last two months I figured out how to do THIS. The system can be mixed with desktop speech recognition / VAM voice command plugins, and synchronised. But here, I'm just using a simple click and point plugin.

The AI isn't self aware of its environment but it can be trained to understand who it is, and where it is, and what its doing, so eventually the scene action and the AI's responses will mesh together. Use your imaginations where this could take VAM users! For example, what if you made and integrated an object recognition plugin?



This is a commercial instance of GTP3 + (combined with) similar nets embodied in a VAM model. The voice is great, and the whole thing so realistic it gives the illusion of a soul.

Training GTP 3 to learn the colour of cherry blossom. You need a hella lot of patience to train the AI, but she will learn, as evidenced in this first video.

View attachment 111401

I have a question: what is the name and source of the female AI voice? Is it from any text-to-speech engine from AWS or Google cloud or Azure?

Okay as mentioned above by people, the voice seems to be from AI Replica, a paid app.

 
Last edited:
I re-opened VAM today to see if anyone has done exactly what you're doing.

With progress in AI VAM could be the FIRST platform for a virtual, AI-powered girlfriend. Finishing up a side project now, but wanted to work on this as my next side project, since I think the opportunity is immense. Any potential for open-sourcing your work to allow others to contribute?
 
I concur, this is a hugely exciting time to be alive. With the advent of metaverse, there is a convergence of a number of technologies that will benefit all of our lives, this includes:
  • Conversational AI
  • Extended reality
  • Photorealistic avatars
  • Tokenised economies
  • Creator economies
  • Chatbots
  • Process automation
I see AI sidekicks as being central to this by performing a set of basic functions for their user:
  • securing data & digital assets
  • providing support and advice
  • sharing knowledge and education
  • assisting with process automation
  • metaverse navigation
  • passive income generation
There is currently a proliferation of bots, with more and more bot type services appearing daily. I can see this as being a problem in the future, similar to the app overload we currently experience. Furthermore, navigating and interacting within a digital economy is becoming increasingly complex for your average user - digital inclusion in terms of skills, confidence, awareness and motivation for finding and using digital products and services is going to create further division in society.

AI companions can be the solution to this as a unification UI trained to speak your language.

Seems like there's a few of us on here that are interested in this space - which IMO extends beyond Virt-A-Mate. We should collaborate.

Yo, what kind of collaboration you looking for? I'm 6,000 words into a thought piece about the ramifications of this tech.
 
Now that PluginIdea's source audio has majorly reduced latency for lip syncing... I think this idea needs to be seriously revisited. I'll try to post some kinda example here using it in VaM with the Replika voice or some other dialogue when I get to it.
 
There's no code lol, this is conceptual stuff to motivate developers. Hedgepig is more of a visionary, not a developer: https://hub.virtamate.com/threads/chatbots-for-vam.10109/page-2#post-31658


Looking forward to seeing it!

To be entirely clear I wouldn't say I have incorporated a fully-embodied AI into VaM, as I have not encoded any logic, triggers, relays, etc into the scene. It's more of an immersive way to use VaM in conjunction with a chatbot that elevates both experiences.
 
This thread has been going on for months! Will someone now please build/modify existing a downloadable desktop AIML bot. Just build the framework, we can all watch this and learn how to write our own conversations: Free Artificial Intelligence Tutorial - Artificial Intelligence Markup Language (AIML) | Udemy - its comprehensive, relatively simple to pick up, and the 3.5 hour course is free!!! Even I can write decent AIML after watching it. If you do that, I'll show you how to get some better TTS voices, and I'll share what I know.
 
Back
Top Bottom