Hi,
I have a 5800x too and no crashes (there surely will be people who may say AMD is the cause of the crashes).
Your directory is not too big (AFAIK) mine is that big, too.
OK, now my educated guess:
Try to auto-repair VaM first. As you may know, there are some functions to test and repair in the VaM updater tool if you will click "options".
If that will not help, try to re-install VaM. This had helped a lot of users in the past.
If that is still not helping with the issue, then I fear it could be something hardware-related.
Those freezing issues are typical for at least two things: Issues with the RAM or heat.
Try to do a BIOS-update first, as this is the least expensive try.
-RAM: Try to do some recherches if the RAM modules you have are compatible with your new CPU and with your motherboard. There are usually whitelists around in the internet. Not very likely, but maybe you have not installed the RAM correctly? Try to remove and insert it again, if you know how. Don't kill your PC with static electricity from your fingers!
-Heat: Your new CPU is getting real hot after a while. More than many other CPUs, and VaM in VR is a real stress test! I did some free CPU tests and they all worked with no issues, but after 1h in VaM, I had heat issues, too. Now I have water cooling (I do not advice anyone to build a water cooling, unless you really know what you are doing and if you definitely know you will need it. Many normal air cooling solutions are better in some aspects). Try to optimize your old cooling solution! Maybe add some more fans. But the most important thing: Are you sure you have used (good) CPU heat grease correctly? Too thick? Too thin? Did you have fastened the screws of the heat sink correctly afterwards? This is not always as easy as it seems. Try to monitor your GPU and CPU heat with some tools.
EDIT: Something completely different... Is VaM crashing only on loading scenes (or on loading clothes)? This is unfortunately quite "normal". Though, not very often. I sometimes have crashes while loading big scenes, or scenes with many clothes and/or textures. If this happens, VaM does not freeze, but simply crashes to desktop (or to the VR starting screen). If this is the case, try to clear your VaM cache first, and/or don't use those crashing scenes.
Good luck, I hope this might help.