I was using VAM with Oculus Quest 2 via the cable and SteamVR. The experience was... similar to yours.
On this forum someone said that there was a better way of using it, getting rid of SteamVR because it was causing the lag, especially if SteamVR settings were messed up.
So, now I am using it wirelessly via Virtual Desktop app (AFAIR it was $20 or so, must be bought in Oculus Store). You install the app on the Oculus Quest 2, and then you install the Desktop Stream application from the developer's website. Both Oculus Quest and your PC must be connected to the same WiFi network to work correctly. Please refer to YouTube tutorials how to optimize Virtual Desktop's streaming settings (resolution, framerate, etc.).
After that you have 2 choices:
1. Using your PC, create a shortcut on your desktop with the target:
"C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "
H:\VAM\VaM.exe"
I have my VAM folder on the drive
H:, change your target location to where you keep your VAM.
Then, put on your headset and launch Virtual Desktop App, connect to your computer using your Quest app. After that, you will be able to see your desktop in your Quest. Now just click on your shortcut and VAM should launch in VR mode, without SteamVR.
If it is too difficult for you, you have the 2nd option:
2. After launching both Stream App on your PC and Virtual Desktop App on your Quest, connect to your computer using your Quest app. After that, you will be able to see your desktop in your Quest. From there, long press the small orange taskbar icon of the Virtual Desktop Streamer and select "Launch Game..." and then point to your VAM.EXE file on your HDD. The app will launch on your quest without the stupid SteamVR causing problems and delays.
Keep in mind that VAM will lag anyway if the loaded scene is cluttered with objects, especially some large environments, lights, etc.
To increase FPS, you can go to User Preferences in VAM and set the Render Scale to 0.9 or even to 0,7 (it only works in VR). You will lose some resolution, but FPS will increase. You can also use a free VAM plugin called GiveMeFPS.
My PC is 8 years old and I my VAM experience is quite good.
Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
16GB RAM
GeForce GTX 1060
Here are my VAM settings:
Hope this helps
KG