Playing a game in VR is somewhat similar to playing a game in 4k UHD resolution on your desktop monitor,
plus the need for high fps in VR because of constant head movement and severe stuttering.
VaM is known for not being optimized very well and in addition to that: it is easy to set some settings wrong in VR and instantly getting an insane rendering workload.
"Low FPS" is not saying much. It varys strongly with different scene complexity.
If you want to know more, please provide the results of the MacGruber Benchmark test to us.
If possible, please run it with an additionaly installed fresh VaM folder without a high count of custom stuff.
Only with those results, preferably one in VR and one in desktop mode, we can compare the overall performance.
Maybe we will find some issues and solutions.
But... yes, you most likely will have down to half the fps in VR, than in desktop full HD resolution.
It is true, that the CPU is (often) the bottleneck for this GPU in low resolutions. But the higher the resolution and the higher the GPU workload, the smaller this CPU bottlenck gets. Your CPU should be quite good in VR in most other games, but not that much in VaM.
VaM is doing a lot of CPU physics calculation, especially for hair!! clothes and soft body whobbeling... but it is doing this with only using one to three CPU cores, mainly because of the old Unity version. So all that counts in VaM is pure single core speed, not the ammount of cores. Please check if you can optimize your single core speed.
I would suggest to use some monitoring and logging tool to see how high your hardware is clocking on load, like maybe GPUz and CPUz.