Solved Too many heap sections

[Vesolye]

I have 64gb ram now, so there should be no problems theorically, but it keeps crashing my game.

Yesterday, it requires 10-13 scenes to crash, but today only one scene with timeline is enough to crash with this message.

Maybe it was caused by my enormous amount of addons? I cannot guess...

 

[Case]

I'm not sure if this works or makes any difference on your system.. I'm able to add up to 5 girls and 1 guy no problem if no one has any Spec, gloss or Norm textures on.
I'm not sure what the deal with the extra maps is on my system and I've even created my own gloss, spec and norm maps all as jpgs. Even those files that are 90kbs crash my scenes the same way the pngs do... If anyone here happens to have a way to fix that I'd be interested..

I have an older EyeBuyPower with 32 gigs of ram. My graphics cards a gtx 1060ti. Its not amazing but it still works awesome for me. I set the render scale to 85. I cant really see a difference in appearance from 100, but it sure loads up faster. I'm not sure what the deal with the shaders is from my end and why those other maps are so buggy for me.

* Pretty sure that your issue is smth different -sounds rather like something to do with the rendering pipeline or GPU drivers than a memory-related issue (it's pretty much accepted wisdom by now that the heap sections CTD is memory-related) - also pretty sure that you'd like to have @hazmhox advice.

* Five ppl is ... extremely ambitious, for ppl with hardware three geneerations ahead of yours. Unless you're into slideshow's that is.

However, the crashing sounds ... ominous.

I've no advice I could give you but this: Most likely, asking for help in this thread isn't the best best way for you to optimize the chances of a helpful response - you want to attract the attention of ppl who have either encountered your problem before, or those who have a very good understanding of Vam's rendering pipeline (there are a couple of those active here)

TL;DR - Best thing you can do IMO is create your own thread.

 

[le_hibou]

Kudos on the find that morph-dupes also count. That being said, some "nagging":
* Your script "double.-counts" morphs - it counts both the .vmi and the .vmb for each morph as separate morphs. They're not. Not really important (If x>y, then 2x > 2y, too), but still.
* There's a limit to how usefull this can be: The problem with the .var system isn't "getting rid of bad vars" per se (otherwise, we'd solved it years ago) - it's the references to bad .vars.

For example, look at your own whitelist - you've whitelisted both the "V2.___UniversalMorphCollection___.100" and "vecterror._MorphCollection_.26". If you take a closer look, you'll see that "vecterror._MorphCollection_.26" is completely duplicated within "V2.___UniversalMorphCollection___.100" - unsuprisingly, since the latter was meant as a replacement for all prior packs. So why are both in your whitelist? (SPOILER: Bcs vecterror is smart, but his Vam's packagemanager is not ... eg. grab his Lana-look. Both packs are referenced in the .vap ... Why? I've actually tried editing the references manually & repackaging that look - Vam just adds the references again. Probably bcs they're both listed in my cache.)

Same applies for other morphpacks - the "import_reloaded" pack(s) are duplicated in another common package ... etcetc

Thanks for your feedback, and sorry for the late answer.
* You're absolutely right about the double count. I'll try to fix that at some point.
* And yeah, I agree that because of dependencies there's a limit to what we can do... But at least now, we know that it's the morphs that cause this issue, and hopefully creators will pay more attention to the number of morphs they package (and maybe also avoid depending on bloated VARs).

 

[PiddlyD]

A problem with this script - or really, a problem with the hub - is that if some of these bloated vars with tons of morphs are hub hosted dependencies - you're going to have to run the script and delete those dependency vars every time you update dependencies in VAM.

But it does seem to work. i7, 64GB, RTX2080, crashing all the time. Ran this, removed almost all vars with 1000+ morphs - running far more stable now.

 

[le_hibou]

A problem with this script - or really, a problem with the hub - is that if some of these bloated vars with tons of morphs are hub hosted dependencies - you're going to have to run the script and delete those dependency vars every time you update dependencies in VAM.

Or you could give up on those and whitelist them, that's what I did... But yeah, it's not ideal.

 

[PiddlyD]

Or you could give up on those and whitelist them, that's what I did... But yeah, it's not ideal.

I suppose if you made empty files with the .var extension with those names in your addons folder, it wouldn't download the dependencies again after you deleted the actual vars. That would be a nifty option to the script - "delete var and create dummy file as placeholder".

 

[adamgali]

Cutting edge solution without disabling vars, morphs, ..etc

Disable caching. The impact is not very noticeable since I have nvme/i9/64 memory. I have over 1.5TB vam and using caching requires additional 1TB of free space, eats a lot of memory, heap, and causes a lot of crashes. The difference between caching/no-caching loading time is not a big deal.
You will have slower VAM but no more crashes, no deleting vars.

Also, caching will reduce the lifetime of your ssd/nvme.

 

[SlimerJSpud]

I think I found a minor issue with the python script. In the past, one way to eliminate errors from broken dependencies was to unpack the .var file contents to a .var directory with the same name, then edit the files within. The problem here is, the python script is trying to unzip a folder to check it, which doesn't work. It throws an error and ignores the content. In my AddonPackages, this is a folder, not a file:
ERROR: .\Ark1F1.Violet_Update_1.1.var: [Errno 13] Permission denied: '.\\Ark1F1.Violet_Update_1.1.var'

 

[Boogaloo1]

Thanks for your feedback, I'm glad it helps, I was really happy when I figured that out.

Here's my script, it's my first time writing Python, so bear with me ;-)

You'll have to rename it from morphbloatscanner.txt to morphbloatscanner.py, because the forum wouldn't let me upload a file with the .py extension.

To execute it, you can just run it from your AddonPackages folder (or any folder with VARs) like this:
python.exe C:\<YOUR_PATH_HERE>\morphbloatscanner.py
Of course, you'll need python.

If you want to customize the max number of morphs by VAR allowed you can use "-m".
If you want to execute the script from anywhere and specify the path to a VAR folder you can use "-p"
Here's an example with both:
python.exe C:\<YOUR_PATH_HERE>\morphbloatscanner.py -m 900 -p C:\<ANOTHER_PATH>\virt-a-mate\AddonPackages

I've only tested with Windows PowerShell

Also, in the script, there's a whitelist of VAR files that are ignored, that was mostly for my use, but that you can customize by editing he script.

Hi there,
Can you please tell me where the results of the scan are saved to?
I can see it has ran successfully but can't find where the results have been saved to.

 

[FreeMaster69]

Thanks for your feedback, I'm glad it helps, I was really happy when I figured that out.

Here's my script, it's my first time writing Python, so bear with me ;-)

You'll have to rename it from morphbloatscanner.txt to morphbloatscanner.py, because the forum wouldn't let me upload a file with the .py extension.

To execute it, you can just run it from your AddonPackages folder (or any folder with VARs) like this:
python.exe C:\<YOUR_PATH_HERE>\morphbloatscanner.py
Of course, you'll need python.

If you want to customize the max number of morphs by VAR allowed you can use "-m".
If you want to execute the script from anywhere and specify the path to a VAR folder you can use "-p"
Here's an example with both:
python.exe C:\<YOUR_PATH_HERE>\morphbloatscanner.py -m 900 -p C:\<ANOTHER_PATH>\virt-a-mate\AddonPackages

I've only tested with Windows PowerShell

Also, in the script, there's a whitelist of VAR files that are ignored, that was mostly for my use, but that you can customize by editing he script.

I'm using powershell but it says python can't open the file.

 

[everlaster]

I'm using powershell but it says python can't open the file.

Run as admin?

 

[FreeMaster69]

Run as admin?

Thanks for the reply but it was me being a dumbass as I did not open powershell in addon packages folder first lol. I got rid of so many vars with 1,000s of morphs its unreal. I have a feeling that this script is not scanning my sub folders though as it says "error this file is not a zip" on a lot of packages.

 

[VRmatt]

I'm having the same problem. Too many heap sections.
Recently failed loading the default scene. Is MeshedVR or another dev available to take a look?
<<< Crash dump attached >>>

 

[everlaster]

I'm having the same problem. Too many heap sections.
Recently failed loading the default scene. Is MeshedVR or another dev available to take a look?
<<< Crash dump attached >>>

the file doesn't contain the text "too many heap sections". Where is that error?

Did you try reducing the number of morphs in your installation?

 

[VRmatt]

the file doesn't contain the text "too many heap sections". Where is that error?

Did you try reducing the number of morphs in your installation?

Sorry, the too many heap error is in the popup on the desktop.
It doesnt show in the logs.



This section looks off-putting.
ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF9E63694B4)
0x00007FF9E63694B4 (UnityPlayer) (function-name not available)
ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF9E6366910)
0x00007FF9E6366910 (UnityPlayer) (function-name not available)
ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF9E634CC37)
0x00007FF9E634CC37 (UnityPlayer) (function-name not available)
ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF9E60820F9)
0x00007FF9E60820F9 (UnityPlayer) (function-name not available)
ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF9E624B962)
0x00007FF9E624B962 (UnityPlayer) (function-name not available)
ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF9E625887A)
0x00007FF9E625887A (UnityPlayer) (function-name not available)
ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF9E68A188C)
0x00007FF9E68A188C (UnityPlayer) (function-name not available)

 

[VRmatt]

I'd be happy to send you a scene file or screen share with you to look at it in realtime.
perhaps you'd know where to look, better than I would.

 

[5DVoom]

I latest times i start more often catch this error too (64gb here), suppose this relate with with increased var's, but as already was noted above that in moment of crash the memory far from maximum load, well ok about half, but i don't think it's can cause problem in normal case. But ok t's not critical, just for fyu. If will explore some dependency (among var amount) then will post.

 

[Deleted member 70582]

so if I wanna get rid of these morphs individually in a file, how am i gonna be able to? Im already having trouble keeping phython up even. it runs and it reads but it doesnt really stay up long enough for me to read it.

 

[FreeMaster69]

so if I wanna get rid of these morphs individually in a file, how am i gonna be able to? Im already having trouble keeping phython up even. it runs and it reads but it doesnt really stay up long enough for me to read it.

If you are using python it won't stay up. Use powershell.

 

[VRmatt]

Well, I mean what about the case where you load a default person atom.
If you add 2 more - you'll crash with GC error.

So then, are these loading duplicate morphs? Or are they using the same stack?
I can't load any 3 people, at all. I have trouble believing its a 'morph count' bug. More like a memory leak bug.

I have 64 RAM, and lots of JVM ram available.
I will say, VAM thin twin works most of the time, without any crashing. But obviously its extremely hard to setup a viable working session with it.

 

[virtamater]

Could a folder for Clothes and things that suck up memory be referenced outside of the VaM folder? My VaM folder's 100 gigs.. I use a 8 terabite external hard drive for it, I'd say a good 60 gig of its clothes, does that all really need to b in the addon folder is it's just there for a few scene as presets?
a map to place things not physics or core game related outside of the VaM folder but still be able to be referenced for the use in a particular scene might help if this issue is in fact related to the amount of Vars in your add on folder.

 

[tokyoidolsafk]

Hi all,

I tried everything here from all the helpful people. I removed a whole bunch of Vars with high morphs and did everything but Produce 69 Season 2 just refused to load and got the Heap error no matter what I did. I may not have removed enough, as I have a huge folder of VARS (1TB).

What DID work in the end was to turn off cache. I turned it off. reloaded VAM and BOOM.. Produce 69 season 2 fully functional.
Thought I'd share that with u guys.


My system:
i7 12700K OC
RTX 3080Ti
64GB DDR5 RAM
Samsung 990 Pro NVME 2TB

Cheers.

 

[VRmatt]

I did everything in this thread as well, also cache off, and also with thin twin.
I'm still hitting this error at random times - probably based on model sizes or texture sizes (in simple scenes).

Anyone know if its possible to increase the JVM Heap size for Unity/Vam??
Any devs care to chime in here?

 

[VRmatt]

Why doesn't Meshed just release the V1 code to the community? There's a FAR better chance for us to fix it and make it amazing, than there is for V2 to be a fully fleshed out savior. Every project that becomes open-source inevitably becomes amazing, and bugs get fixed!
Gotta do it for the good of the community.

Honestly, I'm gonna answer the thing you're not gonna like: I would bet that meshed has never stressed tested VAM with 25K vars.
Even tho the game does not prevent you to hoard that much content, I'm convinced that the overall code is not perfectly made to handle that much data. And the fact that reducing the amount of vars fixes the problem does prove that this is the issue.

You have to understand that beyond a handful of devs around here that know exactly how to optimize their plugins/content, the rest does not. You will end up having content that is not perfectly optimized and could lead to memory issues or potential crashes.

So the only solution would be to have several builds with content spread across these.

Unless meshed optimize the game engine to be able to handle that much content, AND that all creators would release content that is optimized af... it's likely that the problem will never go away. ( And 1.x is not gonna receive that kind of heavy change anymore )

 

[hazmhox]

Why doesn't Meshed just release the V1 code to the community? There's a FAR better chance for us to fix it and make it amazing, than there is for V2 to be a fully fleshed out savior. Every project that becomes open-source inevitably becomes amazing, and bugs get fixed!
Gotta do it for the good of the community.

Why don't you ask meshed? Are you ready to dive yourself into the code and fix those bugs if it goes open source? And I wouldn't even put that under "bugs", that's optimization. And scaling the optimization for situations like this is (to me) kind or proposterous.

Open source is not something you decide one morning out of the blue. VAM has several licenced assets inside its core, meshed couldn't release it. Since the whole lighting system is based on that, you would need to reverse engineer the shader and build something that would work the same way.

And beyond that... not everyone wants to do open source. That's a business decision.

And to be honest, the last part of my message was a polite way of saying: "No one is ever gonna optimize a game for the three dudes that have a 3TB folder of stuff they never use".

 

[VRmatt]

Thanks for the reply, I did just throw that idea out there willy-nilly, but I am aware there's a huge amount of money and business property involved. Call it devils advocate, or you know just getting some ideas out in the open.
Anyway - I wouldn't want to take on a complete rewrite, but I definitely would like some visibility into the backend code, to find out where this GC error is stemming from.
As far as I can tell, there's multiple threads with people just randomly guessing at what the solution is. Which I definitely am thankful for, because I've tried every solution that was mentioned.
I'd definitely like to know what is happening under the hood, that's all I'm saying
I absolutely love VAM and I use the shit out of it, obviously.