Backing up VaM to a local disk with Rsync
wrapped in a python script for easy to do backups
wrapped in a python script for easy to do backups
Why to backup VaM content in the first place?
Because shit happens, that's why. You delete stuff by mistake, your disk can go kaput, online stuff can be gone forever, your dog eats your homework and takes a shit on your PC.
WTF is Rsync?
rsync is a program that copies files from one folder to another, but, and this is the important bit, it only copies the differences in files between the source and destination. The destination folder becomes a exacy copy of the source folder, with rsync copying and removing only what is needed.
Rsync is a syncing process but it's not "live" and only works in a single direction, and as such you can use it as a real backup tool.
What about online storage options?
Well, that's up to you, it's definitely better than nothing. Keep in mind though that you probably have a ton of stuff and it will be expensive in the long run, you may be going against their Terms of Operations regarding content and be cut off, and have little control of how it all works.
Often the service includes a syncing process where your local files are in sync with the online storage, which means that if you mess up locally or online, the synced storage will also repeat those actions. A bi-directional syncing service is not a real backup service, it's mainly for data availability.
More about Rsync:
It is a linux command line program
...
...
...
For those that sticked around and didn't panic with the words linux or command line, thank you first of all.
In the end you can have a python script run all the needed commands with the press of a button, no crazy command line adventures anywhere.
This guide is not for coders or experts but may require you to practice a little and look up information elsewhere to learn more about them.
Linux on Windows (WSL) or a linux system
Rsync runs on linux, but fortunately there's such a thing as linux inside Windows, you don't need a dedicated linux OS anywhere.
Installing WSL is very easy, just follow the instructions.
BIOS settings for Intel based systems:
- Advanced>CPU Configuration>Intel (VMX) Virtualization Technology = Enabled
- Advanced>CPU Configuration>SVM Mode = Enabled
Bash:
sudo apt update && sudo apt install rsyncIf you went the route of dual booting Windows and a linux distribution, then you probably know well enough how to install rsync, mount windows drives, and that kind of stuff. If not, you have to look up guides about those, and welcome to the linux world.
| Rsync | Mount the needed partitions |
Mirroring a source and destination folder with rsync goes something like this:rsync -[options] sourceFolder destinationFolderThis simplicity hides a little the power of the operations that rsync can do. That power can be useful BUT also destructive, be careful. For that reason I use rsync inside a python program - read further down - to avoid typing commands repeatedly and be safe on my backup operations. Follow the link above to learn and test rsync, I will not go in detail on how to use it beyond the example below. | NTFS partitions for fixed disks are mounted automatically in WSL, for example: C:\ /mnt/c/D:\ /mnt/d/At the moment I don't know if USB NTFS disks are automounted, will have to test. |
Rsync command use example
Let's consider you want to do a real rsync backup operation with this example:
| Windows (source) | WSL (linux) | Backup external disk in Windows (destination) |
| VAM folder is in C:\VAM | VAM folder is in /mnt/c/VAM/ Backup VAM folder is in /mnt/d/VAMbackup/ | Backup VAM folder is in D:\VAMbackup |
rsync -[options] sourceFolder destinationFolderbecomes
rsync -arvP --delete /mnt/c/VAM/ /mnt/d/VAMbackup/| This means that rsync will archive, be recursive and verbose, show the progress and delete content while it mirrors /mnt/c/VAM/ into /mnt/d/VAMbackup/. When finished, C:\VAM should be identical to D:\VAMbackup, and only performed operations on what was different from C:\VAM. | archive: includes permissions and other stuff recursive: include all files and folders inside the source folder verbose: chatty progress: what was updated and details --delete: will delete files if not present in sourceFolder |
BE CAREFUL USING RSYNC I said it before, there is the potential to make a mess or even lose data. That's why I use the python script to do these operations, as I can do checks for mounted disks and no need to retype commands. Be safe by learning and testing before doing your first backup.
- Read the guide linked above or another to make sure you understand the rsync command
- Paths in linux use
/and not\like in Windows - The slash
/in front of a folder name, like/mnt/VAM/, means the content of the folder only and will ignore the folder name - Likewise, no slash
/means the folder and contents will be copied over to the destination folder (also check the slash) - Use
--dry-run or -nto test before doing a real copy procedure - DO NOT mistake and swap the source with the destination, unless you want to revert to your backup
- Linux paths are case-sensitive
Using a Python script to run Rsync and tests
because it's easy to fuck up and boring to write commands
because it's easy to fuck up and boring to write commands
Install Python 3
In a Ubuntu WSL:
Bash:
sudo apt install python3Using a Python script
This is not a Python guide, so if you don't know anything about python, well, the web is a big place and full of information.
Comments in the script tell where to change things, with the most important being the paths on top.
Other comments briefly explain the operation, and hopefully the remainder script makes sense or is easy enough to adapt. Bear in mind that I am a python beginner and the code may not be as good as it could be, so if you notice anything that I can improve, please let me know.
Essentially, copy the code, paste it to a text file named
vam-backup.py for example, and place it somewhere easy of reach when using WSL, like your (linux) profile folder. Make it a executable with chmod +x vam-backup.py.To run the script (after your changes) do
./vam-backup.py.
Python:
#!/usr/bin/env python3
import os
import time
# paths here are used in the rsync commands and script below, specify your own
path = {"VAM":"/mnt/c/VAM/", # your VAM folder
"backup-disk":"/mnt/d/VAM-backup/", # your USB backup disk for example
"backup-vam":"/mnt/d/VAM-backup/VAM/"} # the backup VAM folder in the above disk
# rsync command(s) using the paths above
rsync = {"VAM-backup":"rsync -avzh --progress --delete " + path["VAM"] + " " + path["backup-vam"]}
# Options menu for user input
def backup():
os.system("clear") # clean the command line area
print("It's time to do a backup!" + "\n")
print("""
Options:
(1) Backup VAM""" + "\n")
print(" (0) Exit the script" + "\n")
choice = input("Type the option number you want and press ENTER: ")
# VAM option
if choice == "1":
folder_vam = os.path.exists(path["VAM"]) # check if folder exists - assumed as a fixed disk
mount_backup_disk = os.path.ismount(path["backup-disk"]) # check if the mount point exists - assumed as a mounted USB disk
# if the above are not available
if folder_vam == False:
print("\n" + "The folder VAM is not available in '" + path["VAM"] + "'!" + "\n")
input("Press ENTER to go back to the options menu.")
backup()
elif mount_backup_disk == False:
print("\n" + "Your backup disk is not mounted in '" + path["backup-disk"] + "'!" + "\n")
input("Press ENTER to go back to the options menu.")
backup()
# when conditions are met to do a backup
else:
print("VAM and backup disk ready for backup:" + "\n")
input("Press ENTER to proceed with the backup")
os.system(rsync["VAM-backup"]) # rsync command
print("\n" + "VAM backup completed" + "\n")
input("Press ENTER to go back to the options menu.")
backup()
# if you only have one option you can end the script by commenting "backup()" and uncommeting the line below
# raise SystemExit
# Other results
if choice == "0":
print("Chosen option: Exit the script" + "\n")
print("See you at the next backup :)" + "\n")
raise SystemExit # exists the script
else:
print("Unrecognized option. Pick again." + "\n")
time.sleep(1) # delay to read the message above
backup()
# run the options menu function
backup()
