Got it. The character's own fingers will require additional collider settings, and because of self-collision, this will start to cause unnecessary workload. So for now, it's either your own fingers in VR, another character, or toys.
To answer your second question—try adjusting the Break...