Cache optimization

[Kookie]

The current cache can get very large. For example I have 9 GB, this data is highly compressible, I can reduce the size by around 70% using transparent Xpress16K with the windows compress utility. I have not done any extensive testing in regards of speed but generally, at least on HDDs, the reduction of required bandwidth results in a performance increase regardless of the slight cpu load of decompressing.

Could this or similar compression be used in VaM itself? Either way the current cache system seems inefficient.

 

[meshedvr]

The ultimate goal of the cache system was to focus 100% on load times. I tested with both compressed and uncompressed files on both SSD and HDD and found uncompressed offered a significant advantage in all cases so that is what I went with.

 

[Kookie]

That particular compression is meant for transparent and fast decompression. At least Xpress8K usually ends up being faster on HDDs. This compression is used on the windows folder itself and is really decompressed file by file when it is accessed, so it is extremely performant. I don't know which algotrithms you tested, but I'd think using Xpress would be a good compromise at least as an optional setting. I do this compression manually for now and it works well. An optional setting would be appreciated, because the cache still provides massive improvements. I don't know which APIs are exposed for that algorithm, but you could use the built in compact.exe, compression phase of course takes much more time than decompression, but you wouldn't have to even manually decompress it when loading. It works transparently like NTFS compression but is much more efficient.