[Static Person Preview]Create a frozen duplicate of the character.

Assets + Accessories [Static Person Preview]Create a frozen duplicate of the character.

Download [<1 MB]

ldfec

Member
Joined
Apr 19, 2024
Messages
19
Reactions
37
ldfec submitted a new resource:

[Static Person Preview]Create a frozen duplicate of the character. - Create a frozen duplicate of the character.

This is a plugin I created using Codex, and its performance is even better than I expected.

It can create a visual clone of a character and record the character’s state. The clone has no collision or physics, but it still blocks light. This can significantly reduce the hardware load caused by multiplayer scenes without sacrificing visual quality.

It can also be used to create highly customizable background crowds. Clothing and hair can be made optional, and cosmetic effects applied to the...

Read more about this resource...
 
Very nice! Any chance we will be able to save a atom preset with the clone in a later update?
 
Very nice! Any chance we will be able to save a atom preset with the clone in a later update?
Actually. Not being able to save the clone is also one of the trickiest parts for me. If I could get that working, I think it could be useful for a lot of scene creators.

That said, this plugin was originally made just for screenshots. That’s also why it’s so small. I know basically nothing about programming, so I’ve had to let Codex do most of the heavy lifting. 😂

It was originally just a personal-use plugin, but the results turned out way better than I expected. So I figured there might be other people who would find it useful too… though I honestly have no idea how many.

If enough people really want the ability to save the clones, I’ll make Codex burn through its tokens and see what we can do. 😂
 
Actually. Not being able to save the clone is also one of the trickiest parts for me. If I could get that working, I think it could be useful for a lot of scene creators.

That said, this plugin was originally made just for screenshots. That’s also why it’s so small. I know basically nothing about programming, so I’ve had to let Codex do most of the heavy lifting. 😂

It was originally just a personal-use plugin, but the results turned out way better than I expected. So I figured there might be other people who would find it useful too… though I honestly have no idea how many.

If enough people really want the ability to save the clones, I’ll make Codex burn through its tokens and see what we can do. 😂
Well.... I kinda already burned the tokens lol. Very cool idea and great results. I implemented the save file as referencing like an appearance preset vs storing everything to make it portable as that would be large save files. I attached my code if you want to take a look and post it if you like. Codex + VAM = god tier. I have sooo many plugins for stuff now. Haven't gone through any real testing other than good enough for me just fyi. I can't attach the script, but it's here: https://mega.nz/folder/F2lTgLJK#o2dSTeMPYaTG5n0XCY2oJQ might need a touch more work done to get perfect. hair looks like it might be slightly off.
Clone is on the right reloaded from save file.
1791096116401.png
 
Last edited:
Hey man, you might actually want to release this yourself. It looks really good!

I don't know if the Hub has any kind of co-release or collaboration feature, but now that it can save and reload the clones, I think a lot of scene creators could find this useful.

I originally made this because I wanted to create something similar to the MV for “Storm”. And seeing your example, I can already imagine how cool it would look with a bunch of different goblins jumping and running around in the background with different appearances.

This is seriously awesome.
Well.... I kinda already burned the tokens lol. Very cool idea and great results. I implemented the save file as referencing like an appearance preset vs storing everything to make it portable as that would be large save files. I attached my code if you want to take a look and post it if you like. Codex + VAM = god tier. I have sooo many plugins for stuff now. Haven't gone through any real testing other than good enough for me just fyi. I can't attach the script, but it's here: https://mega.nz/file/grdl0ATL#X4CA-T31b7QZ93gQzfYmXcsyJ733ZfU61nLo4BrOTgY might need a touch more work done to get perfect. hair looks like it might be slightly off.
Clone is on the right reloaded from save file.
View attachment 628969
 
Honestly, I’m kind of surprised nobody had made a plugin like this before. Of course, I know basically nothing about programming, so maybe there are reasons or technical considerations that professionals would have thought of that I haven’t.

And about the hair you mentioned — yeah, some clothes with physics also won’t come out perfectly. But for using them as background stand-ins, I think the results are already pretty much perfect.

You were thinking the same thing I was: Codex + VAM is a surprisingly powerful combination. I’m honestly pretty excited about it. Just imagine how many crazy ideas the community could come up with once people start experimenting with this.
 
Honestly, I’m kind of surprised nobody had made a plugin like this before. Of course, I know basically nothing about programming, so maybe there are reasons or technical considerations that professionals would have thought of that I haven’t.

And about the hair you mentioned — yeah, some clothes with physics also won’t come out perfectly. But for using them as background stand-ins, I think the results are already pretty much perfect.

You were thinking the same thing I was: Codex + VAM is a surprisingly powerful combination. I’m honestly pretty excited about it. Just imagine how many crazy ideas the community could come up with once people start experimenting with this.
I'm actually trying to fix the hair now lol OCD. Eyes also seem a touch off when loaded back in when i tried it on a human. I think the idea of this is just unique and sounds difficult. I really dig it, but now that I'm thinking about it memory could be an issue if it's all loaded there depending how big these end up. kudos coming up with it. Skies the limit. I'll post a new version if I can flush this out and let ya know. thanks for your work pretty cool.
 
I'm actually trying to fix the hair now lol OCD. Eyes also seem a touch off when loaded back in when i tried it on a human. I think the idea of this is just unique and sounds difficult. I really dig it, but now that I'm thinking about it memory could be an issue if it's all loaded there depending how big these end up. kudos coming up with it. Skies the limit. I'll post a new version if I can flush this out and let ya know. thanks for your work pretty cool.
Absolutely, I’m looking forward to seeing the polished version
 
I think it got it now. Was kind of a pain (for codex) to get everything referenced in the save file. probably not perfect but pretty spot on now I can't find flaws at 1am lol. I updated the file in the mega if you want to try it out. Forgot to turn off Neural Rending, but even that worked out great. hair and eyes both had to be fixed so might be some code that needs cleaning up in it. Didn't run it through any performance or cleanup runs. I just checked mouth, teeth, tongue were all looking good.

1791101458667.png
 
Last edited:
Okay well we've evolved... 10fps 5 second clip as a poc, but holy crap potential.

 
Okay well we've evolved... 10fps 5 second clip as a poc, but holy crap potential.

View attachment 628988

Oh my god, what a leap! One question though: how demanding would this be on the computer? I imagine it would massively reduce the load on the CPU and GPU, but would it use a lot more RAM? If the memory usage doesn’t go up too much, this would be absolutely insane.

Once we get the frame rate right, we could even record several scenes with this and put them together. It would open up so many more possibilities for scene creators. I can imagine switching back and forth between scenes and animations as easily as switching videos—or even having dozens of people in the same scene at once.

This is fucking incredible. Hats off to you, mate!

PS: How many tokens did you burn through to get this working?
 
oh wow i've been hoping for a plugin like this for years, if it could just save the data with the scene it would be perfect!
 
Oh my god, what a leap! One question though: how demanding would this be on the computer? I imagine it would massively reduce the load on the CPU and GPU, but would it use a lot more RAM? If the memory usage doesn’t go up too much, this would be absolutely insane.

Once we get the frame rate right, we could even record several scenes with this and put them together. It would open up so many more possibilities for scene creators. I can imagine switching back and forth between scenes and animations as easily as switching videos—or even having dozens of people in the same scene at once.

This is fucking incredible. Hats off to you, mate!

PS: How many tokens did you burn through to get this working?
I spent a few hours last night playing with it and just got back on. But this is somewhat technical to get setup and trying to see if I can make it feasible. There is a performance hit obviously since we are using real textures, hair etc and bringing in light. Each frame being rendered. It's honestly going to take a little time to get a good working plugin, but hopes are high. Right now the hard disk size is too large for short clips, we have filesize limits so had to break it up into multiple parts storing the frames separately and textures on their own which is the big size eater. But simple skinned person atoms with default skin presets and simple hair could be really good just haven't tried. I was using 8k skin, long thin hair, obviously all the worst things lol.

I think where this could really excel would be simple blinking, swaying, breathing, slight movements and short clips. Essentially making them appear alive like a person atom vs full out animations, but I'm sure with time we can get there too. What I would love to do is probably setup a plugin for short alive type movement, but have maybe multiple animations it could play? Maybe getting ahead of myself here, but I think for a crowd where you have one person who can just be idle, then react or something could be really sweet.

I also gotta figure out saving all this to scene as well so we don't have to reload them all. The fps drops really bad right now when i go to record. I'm on a 5090 and I go from 200 to 20 for those 5 seconds while recording so having to shut off lighting and do other things to get that recording clean.
 
Last edited:
I also gotta figure out saving all this to scene as well so we don't have to reload them all. The fps drops really bad right now when i go to record. I'm on a 5090 and I go from 200 to 20 for those 5 seconds while recording so having to shut off lighting and do other things to get that recording clean.
Quite excited about the direction this is going;

There's a couple ways I think this could go:

1. I wonder if this can combine with Shadow Venom's Puppeteer plugin, where the recorded static model can be convert it into a rigged CUA for his plugin? Not sure what to do about the hair and clothes though.

2. As for the FPS drop; instead of trying to increase FPS to get a clean recording, perhaps the solution would be similar to ZHFX's render to movie plugin, where it literally freezes frame, allows the physics to calculate and everything to render properly, and then record, and then proceed to the next frame.

3. Wondering if there will be an issue with playback; there might have to be a way to sync the recording to game-time if FPS fluctuates.
 
I spent a few hours last night playing with it and just got back on. But this is somewhat technical to get setup and trying to see if I can make it feasible. There is a performance hit obviously since we are using real textures, hair etc and bringing in light. Each frame being rendered. It's honestly going to take a little time to get a good working plugin, but hopes are high. Right now the hard disk size is too large for short clips, we have filesize limits so had to break it up into multiple parts storing the frames separately and textures on their own which is the big size eater. But simple skinned person atoms with default skin presets and simple hair could be really good just haven't tried. I was using 8k skin, long thin hair, obviously all the worst things lol.

I think where this could really excel would be simple blinking, swaying, breathing, slight movements and short clips. Essentially making them appear alive like a person atom vs full out animations, but I'm sure with time we can get there too. What I would love to do is probably setup a plugin for short alive type movement, but have maybe multiple animations it could play? Maybe getting ahead of myself here, but I think for a crowd where you have one person who can just be idle, then react or something could be really sweet.

I also gotta figure out saving all this to scene as well so we don't have to reload them all. The fps drops really bad right now when i go to record. I'm on a 5090 and I go from 200 to 20 for those 5 seconds while recording so having to shut off lighting and do other things to get that recording clean.
That is great to hear. Can I post the mega link in the overview, since people really want to save the clone, after you release the formal version I will change it into hub link.
 
That is great to hear. Can I post the mega link in the overview, since people really want to save the clone, after you release the formal version I will change it into hub link.
Man I though I had that part dialed in, but was trying to fix the saving path etc and noticed there were still textures not being loaded right under the right lighting. I'd say it's 99% there I just copied up a new script, but it looks a little different than yours as I was bringing in fixes from the animation plugin. Here is the problem I found ( had to write a plugin for this too lol, to take a look at cpu gpu ram vram. You script in the capture state does a great job not consuming much using the person atom for textures, etc. When you save it off and load it on it's own all of that has to be put into memory. I'm seeing about a 3G bump in RAM usage when loaded from a preset on its own. Not a big deal for a couple of these, but a crowd we have issues. I can't really sink too much more time into this as I have a bunch of other plugins I was working on as well as life. Maybe someone can tweak it to be better, but I fixed the texture issues, made saving explorer window better, and simplified the UI. It's working just eats a bit of RAM. Also I put in logic so it can save to a scene. I haven't had time to test that out fully, but worked when i checked it.

As for the animation oh man at least for my implementation has been a headache. It's consuming 5G of VRAM right now as well for 1 instance and that scales to the rest. I could clean it up getting rid of matching frames and maybe sharing textures between frames to reduce size, but this would take some time. I don't know how feasible this would be as I have a GPU with 32G, but most are probably in the 8G-12G boat. It works, but I had to do a lot like integrating it with timeline to go frame by frame in sync. I wouldn't say this is ready to answer to yet lol and I think I might table it for a sec. I started working on a video version similar to Freeze2d plugin but with video. Thats working somewhat without all the performance issues but you can walk around it or anything like that its flat video.

For now take a look at what I posted should be slightly better. It was about 95% before, but torso, head, and clothing needed a touch more work. Unfotunately this isn't my realm of expertise so I'm just learning as I go.
 
Wow, this opens so many possibilities! But most of the things I think of now, need it to be saved. I really really hope you can get it working to save the capture in a scene (or preset, whatever). Less important (for me): Would it ever be possible to give it colliders?
 
It's good, the quality is really exceed my expectation. However, if it can not be saved or loaded, what is the use case?
 
Well.... I kinda already burned the tokens lol. Very cool idea and great results. I implemented the save file as referencing like an appearance preset vs storing everything to make it portable as that would be large save files. I attached my code if you want to take a look and post it if you like. Codex + VAM = god tier. I have sooo many plugins for stuff now. Haven't gone through any real testing other than good enough for me just fyi. I can't attach the script, but it's here: https://mega.nz/file/grdl0ATL#X4CA-T31b7QZ93gQzfYmXcsyJ733ZfU61nLo4BrOTgY might need a touch more work done to get perfect. hair looks like it might be slightly off.
Clone is on the right reloaded from save file.
View attachment 628969
It's really cool!!!. Will you make it to this plugin as an extender?
It will be quite useful.
 
Many people have been asking about the storage issue. Actually, JustADude has already made a usable version. Hat off to his work.

However, at this stage, storage clones still put a heavy burden on the hardware. Someone previously asked about CUAPuppeteer, and that gave me some inspiration. I think it might be possible to directly export the characters and their movements into CUA.

What I mean is that the original version can already achieve this, but it requires the use of tools like Unity and various other components. I might try moving in that direction in the future, but it would probably become a separate plugin altogether.

Once I have a working version, I’ll update this thread and let everyone know.
 
Many people have been asking about the storage issue. Actually, JustADude has already made a usable version. Hat off to his work.

However, at this stage, storage clones still put a heavy burden on the hardware. Someone previously asked about CUAPuppeteer, and that gave me some inspiration. I think it might be possible to directly export the characters and their movements into CUA.

What I mean is that the original version can already achieve this, but it requires the use of tools like Unity and various other components. I might try moving in that direction in the future, but it would probably become a separate plugin altogether.

Once I have a working version, I’ll update this thread and let everyone know.
I could be talking out of my butt since I'm not a coder. This thread had gotten me interested in vibe coding with Codex / Claude, lol.

The issue that CUA Puppeteer ran into for converting a character in VAM to CUA, per Shadow Venom, is with hair and clothes, not with the character model itself.

It seems like your clone captures clothes as well as hair, which, if it can be paired with the locations of the person Controllers, theoretically gives a working solution IF we can export your clone into a CUA to be animated using CUA Puppeteer (bonus if it'll work with CUA Color Vamifiers). It would even remove the need to create new bones / Controllers since it would inherit the existing.

I'm thinking that how the clothes bend may need to be adjusted by recording different poses and comparing which portions bent with respect to the primary controllers; no idea how to resolve the hair issue, but if the clothes can be overcome then even static short / shoulder length hair for background characters would do so much to bring scenes to life.
 
Just want to start out here by saying while I am in software development this stack of tools and game related development is all coming from playing VAM lol. I rely on codex to spin up the code, just guiding it along the way and have some knowledge of C#. So I could be totally dense to something that is easier.

There are a lot of ways to take this. It really comes down to how we intend to use it. Right now this approach could possibly be flushed out into a reference only model which would be lighter than saving, but I haven't tried and at the end of the day you still have textures, hair, clothing, etc which will all take resources in scene. So there is some decent load for each clone you make (simplifying those items would surely decrease the load). Maybe creating a bunch of person atoms, cloning, then turning them off is the right approach. Don't have time to really test that. Those with lots of RAM could use the saved presets and have quite a few people in scene.

ShadowVenom is doing a lot of work in the CUA area adding soft physics, clothing, hair, etc. So I'd rather leave that to them as they spend a lot of time on it. I played around with creating a goblin then gave it controllers. It technically works, but needs more work to make it more stable. it looks goodish. Then last night I tried creating a person into a CUA. That looks great til I start to animate it lol with the exception of hair that was just bad.

The other thing I was playing with was Freeze2D plugin adapting it for video vs. image so I could have more of a living model at the cost of a video. For my use I am not walking around in the environment or anything so I don't think I will use this much, but may instead just do a single rotating plan or less degrees of video. I notice some hitching and keeping animation on pace is a little funky which makes sense. It is fairly low cost though.

Here is basically what I'm talking about... From left to right. CUA goblin i made from a look (was just in the scene), person atom (source), 16 sides of video of person atom, CUA of person without hair because that was absolutely horrendous.



All in all I think it's best we almost vote on what is important to everyone. I personally will keep playing around and of course share if I find a solution, but don't want to be dedicated to it. I think ShadowVenom's upcoming solution is probably the best, but will need a lot of models (CUAs) that are compatible before widely used.

This video is just to demonstrate the roadblock I had with my implementation of animated clone based on this plugin. You can see my VRAM almost maxed out with 6 clones, and decreasing a lot with each removal. This is why I don't even really see it viable to post it. We need to go about it another way, but this was loaded from disk as a preset on each which was the heavier solution plus the FPS impact as well. It would help a lot to share the resources and have each talk to one another, but that is only practical if all of them look the same. In my case I want variety. note: this was all saveable in scene and loaded this way.
 
Last edited:
Just want to start out here by saying while I am in software development this stack of tools and game related development is all coming from playing VAM lol. I rely on codex to spin up the code, just guiding it along the way and have some knowledge of C#. So I could be totally dense to something that is easier.

There are a lot of ways to take this. It really comes down to how we intend to use it. Right now this approach could possibly be flushed out into a reference only model which would be lighter than saving, but I haven't tried and at the end of the day you still have textures, hair, clothing, etc which will all take resources in scene. So there is some decent load for each clone you make (simplifying those items would surely decrease the load). Maybe creating a bunch of person atoms, cloning, then turning them off is the right approach. Don't have time to really test that. Those with lots of RAM could use the saved presets and have quite a few people in scene.

ShadowVenom is doing a lot of work in the CUA area adding soft physics, clothing, hair, etc. So I'd rather leave that to them as they spend a lot of time on it. I played around with creating a goblin then gave it controllers. It technically works, but needs more work to make it more stable. it looks goodish. Then last night I tried creating a person into a CUA. That looks great til I start to animate it lol with the exception of hair that was just bad.

The other thing I was playing with was Freeze2D plugin adapting it for video vs. image so I could have more of a living model at the cost of a video. For my use I am not walking around in the environment or anything so I don't think I will use this much, but may instead just do a single rotating plan or less degrees of video. I notice some hitching and keeping animation on pace is a little funky which makes sense. It is fairly low cost though.

Here is basically what I'm talking about... From left to right. CUA goblin i made from a look (was just in the scene), person atom (source), 16 sides of video of person atom, CUA of person without hair because that was absolutely horrendous.

View attachment 629679

All in all I think it's best we almost vote on what is important to everyone. I personally will keep playing around and of course share if I find a solution, but don't want to be dedicated to it. I think ShadowVenom's upcoming solution is probably the best, but will need a lot of models (CUAs) that are compatible before widely used.

This video is just to demonstrate the roadblock I had with my implementation of animated clone based on this plugin. You can see my VRAM almost maxed out with 6 clones, and decreasing a lot with each removal. This is why I don't even really see it viable to post it. We need to go about it another way, but this was loaded from disk as a preset on each which was the heavier solution plus the FPS impact as well. It would help a lot to share the resources and have each talk to one another, but that is only practical if all of them look the same. In my case I want variety. note: this was all saveable in scene and loaded this way.
View attachment 629685
I don't think Freeze2D will be a good reference since it really eats FPS while the cam move to another angle. In fact VS is creating an APP to convert VaM character into CUA. I think the 1st step to improve this plugin is to enable the feature of saving and loading a static visual person.
 
I don't think Freeze2D will be a good reference since it really eats FPS while the cam move to another angle. In fact VS is creating an APP to convert VaM character into CUA. I think the 1st step to improve this plugin is to enable the feature of saving and loading a static visual person.
Yeah I'm noticing that as well there is definitely a little hitch in there that isn't pleasant with the multi sided video.

I put a new iteration out in the mega link with _LATEST at the end. Here is a release notes on the changes. I had 10 cloned empties running well all loaded from presets and saved in the scene. Below are the changes.
  • Lower VRAM use: Loaded presets now compress large textures by default. You can turn this off with Compress loaded textures and reload to compare quality. Fresh captures are unaffected.
  • Smaller repeat saves: Presets reuse matching textures from one shared folder instead of copying them into every save.
  • Less duplicate memory: Multiple loaded previews using the same preset can share texture objects.
  • Trigger support: Load Preset From Path loads the selected preset. Show Loaded Preview and Hide Loaded Preview let triggers reveal it without loading it again.
  • Smoother static loads: Removed an automatic material check that caused GPU readbacks.
  • When the plugin can identify a texture’s original file path, the preset references it, including textures inside installed .var packages. Textures without a reliable path are saved once in the shared texture folder and referenced from there.
Older presets still load. New presets depend on Saves/PluginData/StaticPersonSharedTextures, so include that folder in backups.

This is about as far as I can take it. I'm out of ideas on more lol. I think this is pretty good now so taking a break. Capture still uses uncompressed, but presets will save and load compressed if the checkbox is checked. Seems to help a lot with little loss.
 
Back
Top Bottom