VaM 1.x VaM Character Generator

[CraSh.it]

Hey gang
I wanted to give you a sneak peek into something I've been working on for the past few weeks.
The model is still in training — updates and releases will follow as results improve.

"VaM Character Generator" - (Drop in a photo > Get a VaM face)

VaM Character Generator analyses a reference photo using MediaPipe's 478-point 3D face mesh, extracts 28 precise facial measurements from both front and side views, and maps them to a VaM appearance preset using a K-Nearest Neighbour model trained on 160,000 VaM renders so far.
Each training sample is a real VaM render with randomised face morphs — the model has seen virtually every combination of face shape the engine can produce.
The result is a .vap preset that loads directly into VaM with matching face morphs, skin tone, lip colour, and eye colour extracted from the reference photo.

This is a face/head generator only.
Body, clothing, hair and expressions are yours to customise. All generated morph values are fully accessible as sliders in the app. You can fine-tune before export.

Stay tuned...

 

[murn]

What next, auto scene creating

 

[LeChuck99]

hi. sounds good, but this the same like generate a face morph and face diffuse texture for genesis 2 models with the tool facegen?

 

[Demorfeus]

I'm also wondering how this option differs from facegen.

 

[Juno]

The main difference between this and facegen is that this is using an ai trained model using vams builtin morphs to do the job granularly where as facegen is using basic image analysis and user placed pins to create a singular morph which you then need to simply drop into vam.

Lets be real here, Facegen is an ancient outdated method to generate morphs and textures from a photo and frankly the results it gives are subpar and that's being extremely generous. Morphs are never even remotely close to the source and textures are a blurry mess and often not even placed correctly even if you provide perfect high res images. The only reason its still used at all is because it requires no 3d knowledge of its users and every other method, and there are many, absolutely does require the user to learn multiple software suites to get the job done. Myself I stopped using FG in 2020 cos I was just fed up of spending my time polishing a turd and committed to learning things properly. I suspect it won't take much for this to bury facegen.

 

[LeChuck99]

The main difference between this and facegen is that this is using an ai trained model using vams builtin morphs to do the job granularly where as facegen is using basic image analysis and user placed pins to create a singular morph which you then need to simply drop into vam.

Lets be real here, Facegen is an ancient outdated method to generate morphs and textures from a photo and frankly the results it gives are subpar and that's being extremely generous. Morphs are never even remotely close to the source and textures are a blurry mess and often not even placed correctly even if you provide perfect high res images. The only reason its still used at all is because it requires no 3d knowledge of its users and every other method, and there are many, absolutely does require the user to learn multiple software suites to get the job done. Myself I stopped using FG in 2020 cos I was just fed up of spending my time polishing a turd and committed to learning things properly. I suspect it won't take much for this to bury facegen.

hmm okay, i understand it. Eagerly awaiting the result. But one thing. That means, you need for the outcome from your tool, already the morphs in VAM bc. it adjusting the morphs from a list of VAM morphs?
Means, if youre using for the created Face, also the custom morphs, which are used for the training, you need them too? Or are you using only VAM Stock morphs?

btw i am using facegen since`... a long time. idk. which version you used in 2020, i am using pro V3. The morphs are really fine for me and also are the textures placed exactly where they should. Also the nostrils. The trick is, where you placing the points for the nostrils. The textures are for me also very sharp and not blurry. The main benefit is, that it adjusts the whole texture set, so there are no border lines in the brightness etc between face, and torso textures.
btw. I also trained a lora for flux a long time ago, to create nativ genesis 2 face textures directly. Meanwhile i using the portrait tools and dropping them into facegen. Its also not that difficult to enhance blurry gen2 textures via AI. But what you do is really interesting.
One more question. If youre using, like in your picture above, 320 Morphs for the face, whats about the Performance in VAM? Or are you baking at the end also one morph from all the adjustments?

 

[hazmhox]

One more question. If youre using, like in your picture above, 320 Morphs for the face, whats about the Performance in VAM? Or are you baking at the end also one morph from all the adjustments?

That'll probably up to you to merge the final model. Merging upfront would be counter-productive as you could not adjust manually the character.

The morphs are really fine for me and also are the textures placed exactly where they should. Also the nostrils. The trick is, where you placing the points for the nostrils. The textures are for me also very sharp and not blurry. The main benefit is, that it adjusts the whole texture set, so there are no border lines in the brightness etc between face, and torso textures.

You don't have published any looks but one thing for sure, people using FG and releasing on the hub unless there's a massive work afterwards to make this looks good... are generally kind of "meh" compared to any manually created character. A direct output from FG is on the very low quality end of my spectrum.

 

[hazmhox]

I'm also wondering how this option differs from facegen.

Not using a proprietary, close source, paid, and wayyyyy too pricy for what it does... software?
(assuming that one is free)

 

[Juno]

hmm okay, i understand it. Eagerly awaiting the result. But one thing. That means, you need for the outcome from your tool, already the morphs in VAM bc. it adjusting the morphs from a list of VAM morphs?
Means, if youre using for the created Face, also the custom morphs, which are used for the training, you need them too? Or are you using only VAM Stock morphs?

btw i am using facegen since`... a long time. idk. which version you used in 2020, i am using pro V3. The morphs are really fine for me and also are the textures placed exactly where they should. Also the nostrils. The trick is, where you placing the points for the nostrils. The textures are for me also very sharp and not blurry. The main benefit is, that it adjusts the whole texture set, so there are no border lines in the brightness etc between face, and torso textures.
btw. I also trained a lora for flux a long time ago, to create nativ genesis 2 face textures directly. Meanwhile i using the portrait tools and dropping them into facegen. Its also not that difficult to enhance blurry gen2 textures via AI. But what you do is really interesting.
One more question. If youre using, like in your picture above, 320 Morphs for the face, whats about the Performance in VAM? Or are you baking at the end also one morph from all the adjustments?

I'm not the creator of this plugin just an experienced Vam creator with his opinion so many of those points should be directed to @CraSh.it but I did see some discussion on the Vam discord and it is supposedly only using the inbuilt morphs for training and morphing the end model. I stand by the statement Fg textures are turds, they are just not comparable to many of the lovingly created texture sets made by the real texture gods, serviceable after a lot of extra work but not in the same league.

What I would like to see here is the actual result in Vam rather than just a shot of the plugin/app ui

 

[LeChuck99]

That'll probably up to you to merge the final model. Merging upfront would be counter-productive as you could not adjust manually the character.




You don't have published any looks but one thing for sure, people using FG and releasing on the hub unless there's a massive work afterwards to make this looks good... are generally kind of "meh" compared to any manually created character. A direct output from FG is on the very low quality end of my spectrum.

ah ok. then its possible to merge them if youre running into performance problems. Thats a good fallback if you need it.

i tested it right now with facegen. created with AI an 8k Portrait with upscale as input with noises on the skin then created the texture with facegen. The output is a 4096 face texture. Thats the outcome from facegen.
but at standard, i am dont using such detailed pictures as input, because i use for skin details more the other texture layers. Alpha, spec etc. The skin is not realistic, its for the demonstration why it looks that weird.

 

[LeChuck99]

I'm not the creator of this plugin just an experienced Vam creator with his opinion so many of those points should be directed to @CraSh.it but I did see some discussion on the Vam discord and it is supposedly only using the inbuilt morphs for training and morphing the end model. I stand by the statement Fg textures are turds, they are just not comparable to many of the lovingly created texture sets made by the real texture gods, serviceable after a lot of extra work but not in the same league.

What I would like to see here is the actual result in Vam rather than just a shot of the plugin/app ui

yah sure. I hope, he can finish it and it works. A free tool like face gen, with some improvements is alway the better one.
Excellent textures are mostly not on the fly out of the box. I also know, you can waste hours of time to adjusting morphs.

 

[Clownmug]

This seem like it could be another great tool for creators and increase overall accessibility for everyone.

I saw this other thing below posted on Reddit the other day that seems relevant.
From Image to Fully Rigged Character in UE5 with 3D AI Generation

It seems impressive yet the bullet points show their process involves a whole bunch of cleanup and refinement work.

Anyways, my point is we shouldn't kid ourselves about which tool or method works better. The skills and experience of the creator matters but the results are just a nicer looking caricature in the end.

 

[DJ]

I know nothing about AI modeling, so maybe this is a painfully naive question, but...

Even if you limit each morph to a value of either zero or one, 320 morphs could be used to generate 2^320 = 2.13 x 10^96 different faces. But your training set uses only 160,000 images. How is this enough to even approximate all the facial possibilities?

 

[hazmhox]

Even if you limit each morph to a value of either zero or one, 320 morphs could be used to generate 2^320 = 2.13 x 10^96 different faces. But your training set uses only 160,000 images. How is this enough to even approximate all the facial possibilities?

Yeah, and it's not 0/1... Assuming everything is -1 / +1 value.
A single morph is 1 * 200 values. ( with a floating point of 2, 0.001 ). That's 200 values x 320 morphs.

Now, the training pool is as good as it gets depending on your hardware. Just like the average face generation today, is based mostly on porn (as reported by several papers)... this base will also lean on the best possible outcome it can do with the dataset it has. But due to the diffusion process, it could technically average the result to get to a face outside of the original dataset it was trained on (even tho it's far stretched, I agree).

One of the massive caveat of the AI today, everyone knows it (at least in the field), no one talks about it: there is not enough data to make the model better... we're at a plateau. And in the case of that project, hoping to train on millions on source images is definitely out of the scope

But if @CraSh.it does a good job, it should lend pretty solid results even with a small/medium pool for the training.

EDIT: on top of that, it hardly can't be worst than FG.

 

[Paulj]

Very much interested in this project! Like OP said.. Facegen is ancient and we desperately need a replacement. I'm looking forward to seeing how this goes.