I apologize, I meant session plugins, which keep remain unloaded (except FasterVam) and I have to reload them manually. Default scene seems to load properly. There's another bug I believe, if I don't actually load default scene after starting (which I often do) and load another scene instead, it will miss things like clothing/hair on the person and also custom unity asset. When I load it again, it loads properly. Also, if I do load default scene at first (it also uses custom unity asset environment) and then load other scene, it loads right away with no missing assets.
So looks like having to load default scene is a requirement for other scenes to load properly (or load them twice). But still, all other session plugins are not working for me - looks like they're in queue for loading and can't even reload them right away, have to wait until they go past the queue, maybe that's the case with other scenes assets not loading.
I tested it with FasterVAM as a session plugin default, and other time with it being enabled only once - looks like as long as there is headers file in VirtualVars directory, plugin will work. For me it's difference of 5 seconds with 1034 var files, so I don't suffer for not having it (but still it loads 33% faster).