If you haven't read our previous VaM2 progress update, please read that 1st as it contains a lot of information we don't want to re-post here as this post is already going to be a giant wall of text:
https://hub.virtamate.com/threads/vam-2-x-progress-update.8374/
We are 3 months further along on VaM2 development since the last progress update at the end of May, and a lot has been accomplished, but the journey of VaM2 has just begun. Some things have taken more time than expected, but some things were easier due to our previous experience on VaM and the ability to have some reference code and models from that earlier project. Since the last progress update, we made several important decisions after careful evaluation of many options. We decided to take the time to carefully weigh options in various areas in order to avoid long term issues in VaM2 that could affect performance or usability. Improved performance and usability are 2 of our 4 main goals for VaM2. Some decisions in VaM were rushed which led to later issues in both these areas. We are trying to learn from that and avoid similar hasty decision making on VaM2. This does come at a cost of increased time to development.
This progress update does not include any new screenshots or videos due to time constraints. We were attempting to make it far enough along to provide a video demonstrating a physically rigged and poseable model showing the smooth skinning and joint bends using the now fully completed joint-controlled-morph (JCM) system. We did not quite get there in time for this progress update. It has been too long since the last progress report, so we decided to give a lengthy text-only update now to explain what we have been working on. A video demonstrating everything accomplished so far on VaM2 will be coming in the next few weeks once the final pieces have come together. And within a few weeks after that we expect to release the 1st playable alpha. This alpha will be released to all Patreon backers to try out. Thank you for your patience!
Summary of some of the work since the last update:
High Resolution Skinning Issue Fix
DAZ Studio does not have a way to export HD (4X - 16X high resolution) skinning data making it a challenge to support high resolution Daz3d models in Unity and VaM. The base resolution models are too blocky for use in VaM (see the last update for comparison). If you export a high resolution model from DAZ Studio to Unity and pose the model, it does not work. The reason DAZ Studio does not export HD skinning data is because it does not generate or use it internally. DAZ Studio uses a process called mesh subdivision and smoothing to make HD skins. Mesh subdivision and smoothing is a fairly complex and computationally expensive operation. Instead of implementing this kind of subdivision system in VaM2, we found a faster computational method of using interpolated skinning data. In our last update we found a way to generate the HD skin data needed for this system which is what was shown in the screenshots, but this method turned out to be flawed and produced artifacts and inaccurate skinning in some regions of the skin. This was not going to be acceptable in a final product, so we rewrote how this HD skin creation was done, and the results are even better on all areas of skin than our previous method. The skinned mesh now very closely matches what is seen in DAZ Studio but without the computational cost and complexity of the subdivision system. This rework took several weeks to complete. We strongly believe the HD skin and HD morphs in VaM2 will separate it from the competitors so this was a very important item to complete. We are 100% happy with the skinning system in VaM2 now and don't expect further changes to the implementation going forward other than to implement the final GPU variant of the code to further improve performance.
Evaluation and Decisions On Physics Engines
Physics is one of the most important parts of VaM. It is what provides realism in movement and reaction. For VaM2 we want to vastly improve both physics quality and performance. To do that, we have to make the right decisions early on in the development process. Unity has several new options for physics engines. We decided to evaluate each option in order to make the best decision for what VaM2 needs.
Evaluation and Decisions On Model Importing and Usage
We looked at 2 possible ways to import and handle models from Daz3d and other sources.
Fixed issue with bone import to allow joint controlled morphs (JCMs) to exactly match DAZ Studio implementation.
The previous progress update showed off JCMs, but they were not yet 100% accurate due to how DAZ Studio was exporting bones with incorrect orientation. We have fixed this so the JCMs and other joint-angle controlled systems can get accurate joint angles. The result is now 100% match with what is seen in DAZ Studio. This took a bit of time to get right, but having a 100% match to original intent of these morphs made it worth the time spent.
Redid the morph system and partially completed bone movement morphs.
In combination with the choice of using FBX importer, the special HD morph importer, and the need for bone movement morphs, we redid the morph system as previously shown in the last update to now account for all of these factors. Bone movement morphs are now importing correctly, but we have not yet finished the system to apply the bone movement morphs. Modifying bone movement morph values in VaM could result in severe performance degradation. These were all the morphs marked in red in VaM. We are hoping to improve the runtime performance of using these morphs so they no longer affect performance. We believe this will be achievable with this new morph system in combination with our choice in physics implementation.
Created initial person physics rig to allow person to be posed and react in a natural way
This is still a work in progress and needs more tweaking. We are experimenting with a setup similar to VaM as well as a new setup using the new articulation joints in the newer version of Physx as these promise more stability and realism for long chains of joints like human models have. Our goal is to improve upon the realism as compared to VaM if possible. Joints did not always act in a natural way in VaM even though we spent months getting those results back when that rig was initially created. It is quite difficult to get the settings set in an optimal way. We will be providing a reference physics rig to the community so making different person atoms with different settings and different grafts (genitals, tails, wings, horns, etc.) will be possible.
Project reorganization
This was done for better development efficiency and future setup for community creation of custom objects (atoms). We now have separate core engine and asset creation projects. The asset creation project will be provided to the community to allow making content for VaM with unprecedented possibilities. We will be using this same asset creation project internally to make the built-in atoms, including the person atoms.
VaM2 Alpha
Goals for the first playable alpha:
The goals for the 1st playable alpha have changed slightly since the last update in order to get this out as soon as possible. Here are the current goals.
https://hub.virtamate.com/threads/vam-2-x-progress-update.8374/
We are 3 months further along on VaM2 development since the last progress update at the end of May, and a lot has been accomplished, but the journey of VaM2 has just begun. Some things have taken more time than expected, but some things were easier due to our previous experience on VaM and the ability to have some reference code and models from that earlier project. Since the last progress update, we made several important decisions after careful evaluation of many options. We decided to take the time to carefully weigh options in various areas in order to avoid long term issues in VaM2 that could affect performance or usability. Improved performance and usability are 2 of our 4 main goals for VaM2. Some decisions in VaM were rushed which led to later issues in both these areas. We are trying to learn from that and avoid similar hasty decision making on VaM2. This does come at a cost of increased time to development.
This progress update does not include any new screenshots or videos due to time constraints. We were attempting to make it far enough along to provide a video demonstrating a physically rigged and poseable model showing the smooth skinning and joint bends using the now fully completed joint-controlled-morph (JCM) system. We did not quite get there in time for this progress update. It has been too long since the last progress report, so we decided to give a lengthy text-only update now to explain what we have been working on. A video demonstrating everything accomplished so far on VaM2 will be coming in the next few weeks once the final pieces have come together. And within a few weeks after that we expect to release the 1st playable alpha. This alpha will be released to all Patreon backers to try out. Thank you for your patience!
Summary of some of the work since the last update:
- Fixed a major issue with the skinning system on high resolution meshes that was causing rough distortions and inaccurate skinning in some areas.
- Evaluation and decisions on physics engines. This took an extensive amount of time.
- Evaluation and decisions on model importing and usage. This took an extensive amount of time.
- Fixed issue with bone import to allow joint controlled morphs (JCMs) to exactly match DAZ Studio implementation.
- Redid the morph system and partially completed bone movement morphs.
- Created initial person physics rig to allow person to be posed and react in a natural way.
- Project reorganization for better development efficiency and future setup for community creation of custom objects (atoms).
High Resolution Skinning Issue Fix
DAZ Studio does not have a way to export HD (4X - 16X high resolution) skinning data making it a challenge to support high resolution Daz3d models in Unity and VaM. The base resolution models are too blocky for use in VaM (see the last update for comparison). If you export a high resolution model from DAZ Studio to Unity and pose the model, it does not work. The reason DAZ Studio does not export HD skinning data is because it does not generate or use it internally. DAZ Studio uses a process called mesh subdivision and smoothing to make HD skins. Mesh subdivision and smoothing is a fairly complex and computationally expensive operation. Instead of implementing this kind of subdivision system in VaM2, we found a faster computational method of using interpolated skinning data. In our last update we found a way to generate the HD skin data needed for this system which is what was shown in the screenshots, but this method turned out to be flawed and produced artifacts and inaccurate skinning in some regions of the skin. This was not going to be acceptable in a final product, so we rewrote how this HD skin creation was done, and the results are even better on all areas of skin than our previous method. The skinned mesh now very closely matches what is seen in DAZ Studio but without the computational cost and complexity of the subdivision system. This rework took several weeks to complete. We strongly believe the HD skin and HD morphs in VaM2 will separate it from the competitors so this was a very important item to complete. We are 100% happy with the skinning system in VaM2 now and don't expect further changes to the implementation going forward other than to implement the final GPU variant of the code to further improve performance.
Evaluation and Decisions On Physics Engines
Physics is one of the most important parts of VaM. It is what provides realism in movement and reaction. For VaM2 we want to vastly improve both physics quality and performance. To do that, we have to make the right decisions early on in the development process. Unity has several new options for physics engines. We decided to evaluate each option in order to make the best decision for what VaM2 needs.
Choice 1: Physx
Physx is what VaM uses for everything except clothing and hair. VaM uses Physx version 3.3.3. Since then, Unity has updated the Physx version many times with various improvements to features, stability, and performance. Unity is now using the latest released version of Physx, version 4.1. While performance is improved, Physx in Unity still cannot handle the skin collision system used in VaM without a huge performance bottleneck. The biggest performance issue in VaM was the skin collision physics running on Physx. Many other aspects of Physx, like the joint system, work very well. We spent time trying out the new articulation joints which promise more stability as compared to the configurable joints used in VaM. Physx is the winner of all the options when it comes to joints. Physx, however, will not work for the massive collision system needed in VaM2 for skin, clothing, and hair. So something else is needed.
Choice 2: DOTS Unity and Havok
We were hopeful these new physics systems were far enough along to be feasible for VaM2 because they promise incredible performance with lots of physics objects. Unfortunately after careful evaluation we found several problems that will prevent us from using either of these choices for now. These packages are still in preview and are not really ready for production projects, especially ones using character models. The joint system is not fully functional and missing some key features, like joint drive, that are very important for realistic joints on a character model. The DOTS model is also not user friendly. We are intending to allow community creators to be able to make custom models using the same process we do, and the model setup in DOTS is a lot more complex and makes many things more difficult to achieve at runtime once the model is running in game. Since we want the character model to be easy to extend and be able to dynamically interact with it in game, DOTS physics (both Unity and Havok) is just not a great option for now.
Choice 3: Full Custom
This option did not require a lot of evaluation. We cannot take the time to implement a full physics engine. What we can do, however, is use custom physics in places where the built-in physics engine options come up short. We could use custom physics for skin collision, soft-body simulation, clothing simulation and collision, and hair simulation and collision. This system would be similar to the clothing and hair physics found in VaM, but with improved accuracy and performance.
Choice 4 and Final Decision: Mix of Physx and Custom
Physx will be used for joints and body dynamics. A custom physics system will be used for all collision, soft-body, clothing, and hair. This should allow us to meet the goals of improved performance, accuracy, and realism as compared to VaM physics.
Evaluation and Decisions On Model Importing and Usage
We looked at 2 possible ways to import and handle models from Daz3d and other sources.
Choice 1: Reuse most of the VaM systems and code, reading Daz3d files like duf and dsf natively, but add support for HD skin and HD morphs
The benefit of selecting this choice would be speeding up time of development as most of the code could just be ported over to VaM2. VaM used the Daz3d duf and dsf files to import models. This method imported each base mesh, skin, and morph separately, and then later used a runtime system to combine the morphs, meshes, and skinning data into the final combined result. Sections of this process were computationally expensive. Mesh grafting is a system used by Daz3d to combine multiple meshes into a single mesh. Grafts are used for things like genitalia, wings, horns, etc. While this system is powerful, it is somewhat computationally expensive to do real-time. VaM did a single graft on the female person and a double graft on the male person for the genitalia. This system was never perfect and could result in weird distortions when using certain morphs and joint bends. You could not get a 1-to-1 match to what DAZ Studio was showing with the same model. This is because Daz3d does not provide code or even a description of how this graft system works. We spent months reverse engineering it for VaM, and the result was just good enough for the models we used. For VaM2 we want to support custom models with custom grafting, and the quality level of the VaM system is not good enough for generic use. The VaM system also came at a performance cost and additional complexity to the character skinning flow. For VaM2 we wanted to avoid that and make the core engine not specific to Daz3d models in particular. In summary, mesh grafting was one roadblock to using the VaM duf/dsf import approach in VaM2. Another roadblock is HD skin and HD morphs. As mentioned earlier, Daz3d does not provide HD skin data as it is not used internally by DAZ Studio. Daz3d also does not provide HD morph data in a readable format. They used a proprietary file format that they did not intend to share. This was recently cracked so HD morph extraction could be possible, but the format of the data is quite complex and oriented towards a system that uses subdivision, which, as mentioned earlier, VaM2 will not use. Overall, to use this choice in VaM2 would mean a compromise on quality and performance, and an additional difficulty in bringing the resolution up to HD quality which is something we don't have an easy solution for.
Choice 2: Create new import system using industry standard FBX format to exchange files
This choice is more generic and supports models from other vendors as FBX is a widely used exchange format supported by dozens of programs and model providers. As mentioned earlier, Daz3d did not provide the HD skin data in the FBX format, but they did provide the combined HD mesh itself. And there is also a way to get combined HD mesh HD morphs exported from DAZ Studio through a special method we will share later once we open up creator content. The HD mesh and HD morphs using this choice have also been run through DAZ Studio's grafting system internally already, resulting in a single skinned mesh and set of morphs rather than a set of meshes and morphs that have to then be combined into a single smooth mesh. This is a huge benefit as it allows us to not have to run mesh grafting at runtime. It is also 100% accurate to the results inside of DAZ Studio which choice 1 is not. The downside of this choice is additional development work as none of the VaM import system can be used.
Final Decision:
We are using choice 2 and have already begun implementation using this approach. We have HD skin and HD morph import systems completed and working with the HD runtime skin system shown in last update. This is the best choice for performance. This is the best choice for model flexibility and does not lock us in to only using Daz3d models. It will make it easier for community creators to create new models with custom grafting or even use models from other sources. This choice does come with the downside that importing morphs will be more complex than just copying over some dsf files from the DAZ Studio install area. Morphs are now specific to the full combination of the base mesh and all the graft meshes and have to be imported through a slightly more complex system. We will detail this process later and might even make a standalone morph converter program since morphs are a popular part of VaM add-ons.
Fixed issue with bone import to allow joint controlled morphs (JCMs) to exactly match DAZ Studio implementation.
The previous progress update showed off JCMs, but they were not yet 100% accurate due to how DAZ Studio was exporting bones with incorrect orientation. We have fixed this so the JCMs and other joint-angle controlled systems can get accurate joint angles. The result is now 100% match with what is seen in DAZ Studio. This took a bit of time to get right, but having a 100% match to original intent of these morphs made it worth the time spent.
Redid the morph system and partially completed bone movement morphs.
In combination with the choice of using FBX importer, the special HD morph importer, and the need for bone movement morphs, we redid the morph system as previously shown in the last update to now account for all of these factors. Bone movement morphs are now importing correctly, but we have not yet finished the system to apply the bone movement morphs. Modifying bone movement morph values in VaM could result in severe performance degradation. These were all the morphs marked in red in VaM. We are hoping to improve the runtime performance of using these morphs so they no longer affect performance. We believe this will be achievable with this new morph system in combination with our choice in physics implementation.
Created initial person physics rig to allow person to be posed and react in a natural way
This is still a work in progress and needs more tweaking. We are experimenting with a setup similar to VaM as well as a new setup using the new articulation joints in the newer version of Physx as these promise more stability and realism for long chains of joints like human models have. Our goal is to improve upon the realism as compared to VaM if possible. Joints did not always act in a natural way in VaM even though we spent months getting those results back when that rig was initially created. It is quite difficult to get the settings set in an optimal way. We will be providing a reference physics rig to the community so making different person atoms with different settings and different grafts (genitals, tails, wings, horns, etc.) will be possible.
Project reorganization
This was done for better development efficiency and future setup for community creation of custom objects (atoms). We now have separate core engine and asset creation projects. The asset creation project will be provided to the community to allow making content for VaM with unprecedented possibilities. We will be using this same asset creation project internally to make the built-in atoms, including the person atoms.
VaM2 Alpha
Goals for the first playable alpha:
The goals for the 1st playable alpha have changed slightly since the last update in order to get this out as soon as possible. Here are the current goals.
- Working on both desktop and VR modes
- Pose-able physically rigged characters
- Morph-able characters
- Light movement and parameter control
- Complete the node interaction system for both VR and desktop. This will allow controlling and moving objects and joints in the scene.
- Finish tweaking the initial character physics rig
- Complete enough of the parameter system to allow morph and light interaction. The parameter system allows controlling properties like morphs, light color, etc.. Some of the parameter system work is already done and is partially ported from VaM but will include new improvements like undo/redo and support for animation.
Last edited:
