Yes, characters are very expensive. For frametime/fps tracking, you need to do like any gamedev would do, disable things one by one, or tweak the settings one by one to see what makes a difference.
Biggest offenders in VAM:
- Soft Body physics
- Hairs
- Clothes
Without excluding some that sometimes can kill your framerate: some environments (badly made), and lighting.