Arguably, one of the more important things you need to do with your server is back it up; if you have a recent backup, you can rebuild your server, no problem. But if you do NOT have any backups (which is surprisingly common), you're screwed... The rebuild will HAVE to be from scratch, and if you had any custom scripts or configurations there, here's to hoping you remember what they were. Most of the time, this is next to impossible to accomplish without extensive documentation on the rebuild process. While there are all kinds of products and applications that can accomplish this, there are some built-in Linux tools that can accomplish a backup quite easily. Here, we'll go over the general process of backing up a Linux system.
Before we get to the fine details, as the old saying goes, "there are many ways to skin a cat". So this process is a general idea of how *I* perform my backups. Feel free to suggest your own methods in the comments section as well. Just don't send out a lynch mob if your method isn't specifically here...
First, let us get into when and why you would use one or the other. Let us take an example of a basic Linux WebServer; we need to back up the entire hard drive. Now, we will examine a couple of sccenarios and discuss when/why one method may be better suited than the other,
Scenario 1:
You can boot the device, log in to it, and otherwise access the device. In this situation, you would be better off using rsync. Since dd makes a complete disk IMAGE (including free space), you can eliminate some of that extra, unneeded space. With rsync, you can back up the files in such a way as to ensure that they are not corrupted, rsync has a built-in file check. While not a cryptographically secure check, it's a basic check, and I have never personally seen it fail. You can also back up the files in such a way as to be accessible from another device; if you back up the files to an external USB HDD, you can then browse and open those files from another PC. You may wind up changing the filesystem that the files are stored on. For example, your Linux Server is likely to be ext3 or ext4, and an external HDD is likely to be NTFS. Be careful with this... Some filesystems (like fat16/32) have file size limits. If you try backing up a file that is too large for the target filesystem, you are going to wind up having issues.
Scenario 2:
You cannot get to a useable state as Scenario 1. Either the device doesn't boot, or you don't have a username/password, or something similar to this. As long as the HDD is not encrypted, you can get it to a live environment (boot a live CD/USB, or disconnect the HDD from that device and plug it into another), and do an rsync from there. Of course, this is also assuming that the HDD you want to back up is a filesystem that your live environment can mount/load.
Scenario 3:
You cannot get to a useable state as in Scenario 1, and the HDD is either encrypted, or a filesystem that your live environment cannot mount/load, contrary to Scenario 2. In this case, your only option would be to use dd. This way, you still back up all the data on the HDD, and keep it in it's encrypted state. Or, if it is a filesystem that the live environment can't mount, dd keeps the filesystem as-is. This method, however, does not have the same file checks as rsync; it just images the entire drive, byte-for-byte.
Please keep in mind that the steps below are not step-by-step instructions in any way. It is more of a general idea of how I typically go about these things. It is up to you to decide how and in what order to perform the steps/commands.
Mount USB Storage
So you've got some kind of USB Storage device you want to backup to... First thing you need to do is to mount it.
Before you plug it in, you will want to view the existing mounted devices and mountpoints:
df -h
Go ahead and plug it in. To find the new drive and filesystem type:
fdisk -l
lsblk -f
Depending on the exact details of your system and backup media, you may need to find out what filesystems the kernel can mount:
ls /lib/modules/$(uname -r)/kernel/fs
Create the mountpoint directory:
mkdir /mnt/directory
Mount the backup device:
mount -t <fstype> /dev/device /mnt/directory -o uid=1000,gid=1000,utf8,dmask=027,fmask=137
Backing up to a directly connected storage device isn't necessarily the ONLY way to accomplish this. You can just as easily mount an NFS Share or Windows Share:
Mount nfs:
mount 12.34.56.789:/remote/nfs/directory /mnt/directory
Mount smb/cifs:
mount -t smbfs -o username=userid,workgroup=workgroupname,password=XXXXX //ipadd/sharepoint /mountpoint/
Backup
So you have your backup directory mounted, ready to accept the backup, now it's time to do the backup. Now, there are ways we can accomplish this, either with rsync or dd commands. The difference, really, is how you plan on restoring said backup. The rsync backup should allow you to restore the backup onto another system entirely, as we are only actually backing up the data. The dd backup, however, is a complete disk image, so it can only really restored back to that same device. The other primary difference is how much space will be used... Let's say you have a 1TB HDD, with 500GB used; rsync will only copy over that 500GB of date, while dd will completely image the ENTIRE HDD, even the unused space, so that will work out to 1TB as well. Let's take a look at the commands and break them down.
rsync:
rsync -aHAXvP --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} /* /path/to/backup/folder
Flags:
-a : archive mode; equals -rlptgoD (no -H,-A,-X)
-r : recurse into directories
-l : copy symlinks as symlinks
-p : preserve permissions
-t : preserve modification times
-g : preserve group
-o : preserve owner (super-user only)
-D : same as --devices --specials
--devices : preserve device files (super-user only)
--specials : preserve special files
-H : preserve hard links
-A : preserve ACLs
-X : preserve extended attributes
-v : verbose
-P : same as --partial --progress
--progress : show progress during transfer
--partial : keep partially transferred files
The contents of /dev, /proc, /sys, /tmp and /run were excluded because they are populated at boot (while the folders themselves are not created), so we backup the folder itself, no contents.
The contents of /mnt and /media were excluded as these are typically only full of symbolic links to devices or other mount points.
/lost+found is filesystem-specific, so exclude it entirely.
If you plan on doing the rsync command often, you can easily make a file of all the exclusions you want to exclude. Below is an example of creating the file to exclude the same directiories as mentioned above, and the new command you would use for the file:
--exclude-from=FILE : read exclude patterns from FILE
echo "/dev/*
/proc/*
/sys/*
/tmp/*
/run/*
/mnt/*
/media/*
/lost+found" > exclude.txt
rsync -aHAXvP --exclude-from=exclude.txt /* /path/to/backup/folder
NOTE: Be sure to exclude the backup directory as well, otherwise you will enter an infinite backup loop, where rsync will keep backing up the contents of the backup folder to the backup folder, and ever increasing the size. You can combine both the exclusion file as well and the exclude flag.
dd:
Next up, is the dd command. Fortunately for all, there is a rather simple way to compress that disk image to save space, that way, in the example above, your 1TB disk image won't actaully work out to be 1TB in size (exact size depends on the compression).
dd if=/dev/sdX conv=sync,noerror bs=512 | gzip -c > /path/to/backup.img.gz
noerror: ignore all read errors. The default behavior for dd is to halt at any error.
sync: fills input blocks with zeroes if there were any read errors, so data offsets stay in sync.
If you are backing up to a FAT32 filesystem, you will need to split the backup into 2GB files:
dd if=/dev/sdX conv=sync,noerror bs=512 | gzip -c | split -a3 -b2G - /path/to/backup.img.gz
Unmount
Once the backup command has completed, you can now go ahead and unmount the backup directory. If you have a USB device, you *NEED* to do this prior to removing it, otherwise you risk losing data or corrupting something somewhere.
sudo umount /mnt/directory
And now your backup is done!
Restore
Oh snap, something happened to your server, and you need to restore the backup!? To do this, you will need to have the backup file(s) on a functioning, booted device. Just like doing the backup, you will need to bring the HDD from the downed server to this functioning one. You would use the same mounting techniques as described above. The restore commands, though still rsync and/or dd, will slightly differ.
rsync:
rsync -aHAXvP /path/to/backup/folder /path/to/mounted/HDD
dd:
gunzip -c /path/to/backup.img.gz | dd of=/dev/sdX
If you need to restore the dd split files:
cat /path/to/backup.img.gz* | gunzip -c | dd of=/dev/sdX
If you are looking for more info on the above, here are a few useful links:
https://wiki.archlinux.org/index.php/Full_system_backup_with_rsync
https://wiki.archlinux.org/index.php/USB_storage_devices
https://help.ubuntu.com/community/Mount/USB
https://wiki.archlinux.org/index.php/Disk_cloning
http://lmgtfy.com/?q=linux+backup
Lastly, there is a bootable Linux distro, specific for disk imaging, Clonezilla:
