for every float, it is not opening the file every time, but it calls the ZipLibrary, which in turn does an allocation of 4kb, reads 4kb from disk (the minimum amount in the zip library), decompresses 4kb (yes, thats the minimum amount it can decompress), truncates to 4bytes, casts 4bytes to a single float and returns the float. This happens FOR EVERY SINGLE FLOAT. You see millions of ReadFile(4kbytes) in procmon when loading a scene lol. The amount actually read from the harddrive is exactly 1024 times as large as needed. The MemoryStream isnt an optimal solution, but atleast it skips the uneeded expensive zip decompression. The perfect solution would be to somehow load the whole zip once from disk, keep it in memory and decompress the individual files from memory. Not sure if VaM loads all files from the .var or only the needed parts, so memory usage might increase if you do it that way. But atleast do the small fix I mentioned, it just skips the redundant decompression.Wait you are telling me ReadInt32() and ReadSingle() in the original code is actually opening the file every single time? That's insane and I had no idea the zip library would work like that.
C#:public void LoadDeltasFromBinaryFile(string path) { //Debug.Log("Loading deltas for morph " + morphName + " from "+path); try { using (FileEntryStream fes = FileManager.OpenStream(path, true)) { using (BinaryReader binReader = new BinaryReader(fes.Stream)) { numDeltas = binReader.ReadInt32(); deltas = new DAZMorphVertex[numDeltas]; for (int ind = 0; ind < numDeltas; ind++) { DAZMorphVertex dmv = new DAZMorphVertex(); dmv.vertex = binReader.ReadInt32(); Vector3 v; v.x = binReader.ReadSingle(); v.y = binReader.ReadSingle(); v.z = binReader.ReadSingle(); dmv.delta = v; deltas[ind] = dmv; } } } } catch (System.Exception e) { Debug.LogError("Error while loading binary delta file " + path + " " + e); } }
It's actually a common mistake in unity game dev lol.
I think the callstack is something like BinaryReader.ReadSingle() -> BinaryReader.ReadBytes(4) -> ZipLibrarySomething.ReadBytes(4) -> ZipLibrarySomething.ReadInternal(4096) -> ZipLibrarySomething.Decompress(4096)
You really need to do decompression in a block as big as possible, there is additional huge overhead in initializing the decompression state machine. You might get away with the OS caching disk reads, but many small decompressions are burning the CPU.
Last edited: