Credit where credit is due: some random forum post here.
Open regedit: ctrl+r and search regedit
Find the right file: on my computer it's Computer\HKEY_CLASSES_ROOT\.zip but you can ctrl+f and search ".zip" and it should pop up
Export the file: File > Export and save it wherever
Open and edit...