Seems they know there has been a problem with the 540 drivers.... https://www.nvidia.com/en-us/geforce/forums/game-ready-drivers/13/530853/geforce-grd-54617-feedback-thread-released-111423/
Good thing I found this thread, exact same thing has been happening to me! So the best solution is to roll back to an older driver? Currently on Nvidia 546.17 and I get the mentioned freezes and slow downs.