VaM 1.x VaM Character Generator

[CraSh.it]

Hey gang

Small update on where things stand with the project.

Honestly, not much progress over the last couple of months.
I've been in the middle of an international family relocation which turned out to be a massive undertaking and consumed pretty much everything I had in terms of time and energy.

Things are starting to settle down now and I'm hoping to get back to proper development within the next couple of weeks. There's actually some genuinely exciting stuff in the pipeline. I moved away from the KNN approach entirely, built a Jacobian least-squares solver from systematic single-morph sweep calibration data, and the early results are the best I've seen so far. Still work to do but the direction feels right.

Looking forward to finally being able to share some actual results with you all soon.

Stay tuned..

 

[CraSh.it]

One additional thought.

Personally, I think a more realistic long-term direction would be to train a specialised AI model that predicts vertex positions directly on the DAZ Genesis 2 base mesh used by VaM, rather than trying to reconstruct a face by blending existing morphs. Morphs are ultimately an artist-friendly control system, not necessarily the ideal representation for machine learning.

Even then, I would still see that as a way to generate a solid starting point rather than a finished character. At least with today's technology, I think there would still be a significant amount of manual sculpting and refinement afterwards.

More importantly, this still doesn't solve what I see as the biggest challenge. Whether you're predicting morph weights or directly predicting mesh geometry, you still need a highly specialised model capable of reconstructing accurate 3D facial geometry from very limited 2D information. That means building an enormous, carefully curated training dataset, designing and tuning the model, handling domain adaptation between real photos and the DAZ/VaM mesh, and training with a level of compute that is well beyond what most individuals, and probably even an ordinary software company, could realistically afford.

So in my opinion, switching from morph prediction to direct mesh prediction makes more technical sense, but it doesn't really change the scale of the AI problem. The biggest bottleneck is still the specialised training pipeline, the data, and the engineering effort required to make it work reliably.

Really appreciate the thoughtful breakdown and honestly, I agree with most of it.

You're absolutely right that a reliable one click photo-to-VaM system at production quality is nowhere near to what I'm building,

What I'm actually building is closer to your "60-80% starting point", an automated tool that gets you a plausible face shape from a photo, saves you the bulk of the manual morph work, and leaves the fine-tuning to the artist. That's a much more realistic target and still genuinely useful.

The current approach deliberately avoids the neural network route for exactly the reasons you describe (data requirements, domain adaptation, training cost).
Instead it uses a Jacobian solver built from systematic single-morph sweep calibration data, which sidesteps the data problem entirely. It's not a black box, it's a direct measurement-to-morph mapping. The results aren't perfect but they're consistent and explainable.

Your point about direct mesh prediction is actually already on the roadmap as a longer-term direction. Reconstruct geometry from the photo, fit it to the G2 base mesh, export as a custom morph rather than blending existing ones. You're right that it doesn't solve the fundamental 2D-to-3D problem, but it does bypass the morph crosstalk issue entirely.

The gap between prototype and production is real. But a useful prototype that saves creators significant manual work is still worth building.