I don't know about 6000 & 7000 series, but starting with 23.7.1 drivers, tessellation mode is broken on 5000 series (only AMD optimized works).
That's why I asked about GPU, maybe you changed tessellation settings (override app. settings) inside adrenalin.
Make sure it's on "AMD optimized" then restart VaM to apply.
EDIT: