Not A Bug VAM 800MB per-character-instance memory consumption easily fixable (1.x/2.x)

[newmark42]

I'm a computer programmer, and I'm reaching out to help you fix a really easy and important problem in VAM.

The 800MB per-character instance memory consumption problem. It's not textures, it's morph data, and it's trivial to fix.

After fixing in my own DAZ/Unity project, character instances take < 80MB of memory each.

Let me first explain why the problem occurs.. Unity blend shapes are always for the "full mesh", because Unity has no "partial blend shape" support. Which means a DAZ morph that moves 100 verticies in the nose, gets expanded to a morph for all ~20k vertices in the mesh. This makes 70MB of DAZ morph data bloat into 800MB of unity blend shape data. Also, Unity doesn't have a way to share blend-shapes across multiple loaded mesh instances. Each one gets its own copy of the blendshape data. (Thus the 800MB per-instance, instead of per-gender) Not only that, but using Unity's morph system for this causes it to recompute all active morphs for the character every frame, which is a huge waste of CPU/GPU time, and the 800MB is a huge bloat that Unity puts in GPU ram for GPU morph computation.

It took me several days to find this problem, but only four hours and only about 60 lines of code to fix it. All I did was stop using Unity blend-shapes for DAZ character morphs. Instead I extracted the DAZ moprhs into my own "partial morph dataset" onto a "per gender morph proxy", and then wrote my own morph application code (which is tiny). The code re-generates a unique per-instance character mesh for each character-instance when that character's morph state changes. This way, each instance only takes the size of the post-generated mesh, which is quite small. The DAZ morph data is also only stored once (per gender), no matter how many character instances there are.

I'm happy to point you to my example code, or talk about this further to explain.

 

[newmark42]

Here is an example screenshot of our Unity-based project, with 100 DAZ Genesis 3 figures spawned, all custom morphable and posable. Each taking about 80MB of memory.

Of course VAM will not be able to do this many characters, because it also has softbody simulation and clothing sim, and lots of other things to do per character. However, this is just a quick example to demonstrate that it's pretty trivial to fix the 800MB per character instance problem.. And when it's fixed, it radically reduces GPU memory pressure, which improves rendering framerate dramatically.

 

[newmark42]

And just for a quick idea of what my code looks like to apply a "partial morph" to the per-instance mesh, it's really really simple and tiny..

(you can ignore the "AffectedVertexAABB" code, that's computing the bounding box of every affected vertex of a morph, to setup a camera for automatic morph thumbnails)

 

[Jarny]

@meshedvr, any thoughts ?

I know nothing about any of this...only that this post from @newmark42 is hella intriguing.

@newmark42, you mention G3F. does this work for G2F as well ?

 

[meshedvr]

Here is the response I already gave to this on DIscord:

Hi - neither 1.X or 2.X versions of VaM use Unity's BlendShape system. 1.X only stores the actual partial mesh deltas, not full mesh deltas. In 1.X I used a custom morph system that supports formulas, bone movements, bone orientation changes, etc. that Unity does not support. 1.X also only demand loads morph deltas if the morph is used to save memory. It can still use a lot if the user has 1000s of morphs because we keep a lot of data structures for fast access and there is memory overhead for that. 2.X will be similar although I'm not feature matching DAZ morphs directly any more because I'm making 2.X more generic. In any case, there isn't anything I'm going to do to 1.X at this point to address memory issues. 2.X is just getting started and performance and memory usage are a top priority.

 

[newmark42]

Okay, sorry for the multi-posting, and thanks for your reply.

I do hope the 2.x instance memory usage can be dramatically reduced. A DAZ mesh is about 80MB, so at some level it should be very practical to have a character instance use only about that much memory, textures aside. Which means three characters with the same textures would only require 80MB each. And with 2.x hopefully you can use virtual texturing to avoid loading high rez textures on the GPU.

 

[meshedvr]

2.X is going with higher resolution meshes than baseline DAZ mesh, but we won't require as many copies as 1.X does because 1.X did runtime grafting (mesh merging) as well as several other things that requires the mesh in multiple stages to cover all the needs of the program. 2.X will also avoid preload of texture that are not being actively used by using a different asset management system. I don't think we will use virtual texturing because VaM is meant for small scenes where nearly everything is on screen and is in view in most cases. This application is not meant for large scale scenes where a lot of textures are off screen and can be pulled in when needed. Memory usage is a priority and we already have some improvements over what 1.X is doing.

 

[TToby]

although I'm not feature matching DAZ morphs directly any more

Sounds great, all of it, but please please don't screw that DAZ morph support and custom morph creation, cause that is what I am doing 80% of the time in VaM: look creation with 1000s of morphs. I know you won't, because I've read your last update message, but saying this only to be on the safe side.
I am still sure, that the large-scale support of DAZ stuff was the key to success! VaM being something like "DAZ-VR", with all that trillions of DAZ items to use, is like having a world full of options... wether we realy use it or not. Don't mistake those shared stuff with that tons of copyright protected DAZ stuff we have bought and converted for ourself. But I think I charge an open door and you know all of this very well. Thank you!

 

[meshedvr]

@TToby Certain kinds of morphs that Daz3d uses are no longer going to be supported in 2.X, at least for now. Formula morphs and bone rotation (posing) morphs, as a couple of examples, will no longer be supported. Bone movement (structure) morphs will still be supported as those are important for changing overall character size and proportions. VaM will still support custom morphs from the user community or conversion of DAZ Studio morphs. The method for import will be different than 1.X, due some complexities of supporting HD mesh and morphs and the removal of runtime grafting from the VaM engine, which was necessary to get HD to work at all and also support custom grafting. 2.X will still support 1000s of morphs with minimal impact to memory or performance, by using a demand-load system just like 1.X has.