Question LipSync plugin for ANY LANGUAGE using AUDIO ?

[VirtualWishX]

Hi All,
I'm looking for a general LipSync plugin that I will be able to control based on the Mouth / Lips based on my own AUDIO FILES
So it won't matter what language I use as an Audio Source.

What is the best Plugin for 2023 (most up to date?)
Can you recommend one that you've tried, give some tips or how to use it for best most-accurate results?

Thanks ahead!

 

[atani]

Still this one:

Plugins - Realtime Lipsync

This is the VAR version of my RT_Lipsync plugin updated for VaM v1.20.0.7 RT LipSync is a real-time lip-syncing script for VaM. After attaching the plugin to a character, any speech playing through the character's head audio source will produce...

hub.virtamate.com

 

[VirtualWishX]

Still this one:

Plugins - Realtime Lipsync

This is the VAR version of my RT_Lipsync plugin updated for VaM v1.20.0.7 RT LipSync is a real-time lip-syncing script for VaM. After attaching the plugin to a character, any speech playing through the character's head audio source will produce...

hub.virtamate.com

Thanks I'll give it a try! ?

 

[VirtualWishX]

Hi @atani
Any idea how can I make the volume not depand to how the camera close to the character?
I want to have the same volume no matter how close or far away.

EDIT:
I think I found it, under the "Head Audio" tab of the character,
Min Distance to 10.00

 

[atani]

There's a spatialize audio or something like that as a checkbox. You want it unchecked to not be affected by distance.
However, normally audio coming from a person's head is spatialized to mimic reality when the person says something. If you're using it for music or something not person related, you should use instead a audiosource atom.

 

[VirtualWishX]

There's a spatialize audio or something like that as a checkbox. You want it unchecked to not be affected by distance.
However, normally audio coming from a person's head is spatialized to mimic reality when the person says something. If you're using it for music or something not person related, you should use instead a audiosource atom.

Thanks!
I'll try to find that spatialize audio, probably hiddnen somewhere in the maze of tabs and menus, unless it's on the same one and I missed it.

For rendering video, in some cases I don't need stereo or depth to visualize it in 3D space, so I need it to be equal at least for the Lip-Sync audio part.

Now I'm trying to find a good way to RENDER OUT the video without capturing manual with OBS.
I'm afraid that my machine can't really render from within VAM without loosing frames, I'll ahve to try...

I also look for a way to RENDER the video with Transparent Background,.. not sure if there such a plugin I'm still looking around.

 

[atani]

Spatialize checkbox on the left

 

[VirtualWishX]

Spatialize checkbox on the left

Thanks again, I guess I missed it. that's VERY helpful! ?

 

[void]

From my mediocre experience: You have to put some effort on the Jaw Movement / max Jaw Movement Angle etc on how loud your audio sample is. When you pitch up / down the voice it's just being played faster / slower. In Adobe Audition f.e. you can change the playback speed, if you pitch up the voice by 0,1 then let Audition extend the length by 10%.

Again any tiny change in volume can make the mouth / lip morph going a bit crazy from extreme opening to mouth still shut. But it works despite it looks more like a puppet mouth open / close with no exact forming of the sillables, expecially tongue or lip movement.
Still amazing effect and the best we got so far AFAIK.

 

[EasyVam]

Thanks again, I guess I missed it. that's VERY helpful! ?

Hey my guy, i hope youre happy with the plugin youve found. I just wanted to throw in that theres LipSync present in VaM without the need to use PlugIns. This might be better suited to your needs if you want to create resources (or keep your scenes dependency-light), since the Author of RT.Lipsync still hasnt set any Licensing yet.

To use this, just set it to active on the Auto-Behaviour tab @ personrootcontrol.

 

[abjohn]

Hey my guy, i hope youre happy with the plugin youve found. I just wanted to throw in that theres LipSync present in VaM without the need to use PlugIns. This might be better suited to your needs if you want to create resources (or keep your scenes dependency-light), since the Author of RT.Lipsync still hasnt set any Licensing yet.

To use this, just set it to active on the Auto-Behaviour tab @ personrootcontrol.

new audio feature is not very straight forward in VAM to use, it will be great if someone could write a plugin like adam real time lip sync which is easy to use,instead of going through creating triggers etc.