Automatic Backup of Running KVM Virtual Machines
I am using KVM to run multiple servers as virtual machines. The host machine is running Ubuntu Server 12.04 LTS, as are the guests. I've described setting up the host and guests here.
Raw images are used for the guest file system. That is, on the host machine, the entire guest file system is simply a (sparse) file:
sgordon@host:~$ ls -lsh /var/vm total 76G 62G -rwxr-xr-x 1 libvirt-qemu kvm 90G Jun 5 17:51 guest1.img 15G -rwxr-xr-x 1 libvirt-qemu kvm 45G Jun 5 17:51 guest2.img
Above the two guests have file systems of 90GB and 45GB. But note that as the ext4 filesystem on the host supports sparse files, only 62GB and 15GB are occupied for the respective guests. That is, even though 90GB is allocated for guest1, only what is used - 62GB - is stored on the host disk.
One of my desired backup solutions is to simply copy the guest image files (to another disk and eventually another computer). Something like:
sgordon@host:~$ sudo cp /var/vm/guest1.img /mnt/bkp/guest1-2012-06-03.img
Copying the files is ok if the guest virtual machines are stopped. But if they are running its possible that as the copy is taking place changes are made within the guest, leading to an inconsistent copy. So to copy a running KVM virtual machine I make use of LVM snapshots.
The host system is using the Logical Volume Manager. There are two hard disks in the volume group:
sgordon@host:~$ sudo pvs PV VG Fmt Attr PSize PFree /dev/sda1 ictweb lvm2 a- 1.36t 1022.07g /dev/sdb5 ictweb lvm2 a- 1.36t 769.04g
and several logical volumes:
sgordon@host:~$ sudo lvs --segments -o +pe_ranges LV VG Attr #Str Type SSize PE Ranges bkp ictweb -wi-ao 1 linear 600.00g /dev/sdb5:176429-330028 home ictweb -wi-ao 1 linear 10.00g /dev/sda1:16688-19247 root ictweb -wi-ao 1 linear 18.62g /dev/sda1:0-4767 swap ictweb -wi-ao 1 linear 9.31g /dev/sdb5:7152-9535 tmp ictweb -wi-ao 1 linear 18.62g /dev/sdb5:2384-7151 var ictweb -wi-ao 1 linear 46.56g /dev/sda1:4768-16687 var ictweb -wi-ao 1 linear 300.00g /dev/sda1:19248-96047
My KVM guest images are stored on the var logical volume. I want to take a snapshot of that logical volume and then copy the images to a directory on the bkp logical volume (which is on a different physical hard disk to var). In simple terms the snapshot can be considered another logical volume that is identical to the original var volume. Once the snapshot is taken, any subsequent changes on the var volume, specifically in the guest images, are not reflected in the snapshot. Therefore I will copy the guest images from the snapshot instead of the original var volume. Once the copy is complete I will remove the snapshot.
First, create the snapshot of the original volume var. I'll call my snapshot varsnap. Make sure the size is large enough to fit the entire original volume:
sgordon@host:~$ sudo lvcreate -L 350G -s -n varsnap /dev/ictweb/var
Now create a directory for the snapshot and then mount the snapshot to that directory:
sgordon@host:~$ sudo mkdir /mnt/snapshot sgordon@host:~$ sudo mount /dev/ictweb/varsnap /mnt/snapshot
You can visit the /mnt/snapshot directory and now list the files - you will see everything from the original volume var. So lets now copy the guest images:
sgordon@host:~$ tar --create --sparse --file /mnt/bkp/guest1-2012-06-03.img.tar /mnt/snapshot/vm/guest1.img
I am using tar in this case because I don't want to worry about spasre files. Recall that the KVM image is a sparse file: although 90GB of hard disk is allocated (and visible within the guest OS) currently only 62GB of the host hard disk is used to store it. I am using tar to "convert" from a sparse file to a ordinary file so I can move it between different computers/OSes in the future. (See the comments below for further details about sparse files). The backup of the KVM guest image is now created:
sgordon@host:~$ ls -lh /mnt/bkp/ total 62G -rw-r--r-- 1 root root 62G Jun 3 05:07 guest1-2012-06-03.img.tar
Finally, the snapshop logical volume can (and should, to save space) be removed:
sgordon@host:~$ umount /mnt/snapshot sgordon@host:~$ lvremove -f /dev/ictweb/varsnap
Next time you want to backup the image than repeat the above steps (except no need to make the /mnt/snapshot directory if you didn't remove it). Of course its much easier to automatic this process using a cron job. Assuming the directory /mnt/snapshot exists, put all the commands in a Bash script, say called backup-kvm-images as follows:
#!/bin/bash
# Creates a snapshot of the volume storing the KVM images, and then
# using that snapshot copies the images. The benefit of using a snapshot
# (as opposed to copying the images) is that the VM does not need to be
# shutdown.
# Include appropriate paths (since cron doesn't use your normal environment)
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
# The date of the backup
bkpdate=`date +%Y-%m-%d`
# Create a LVM snapshot of the var LV
lvcreate -L 350G -s -n varsnap /dev/ictweb/var
# Mount the snapshot
mount /dev/ictweb/varsnap /mnt/snapshot
# Copy the KVM images
tar --create --sparse --file /mnt/bkp/guest1-${bkpdate}.img.tar /mnt/snapshot/vm/guest1.img
tar --create --sparse --file /mnt/bkp/guest2-${bkpdate}.img.tar /mnt/snapshot/vm/guest2.img
# Remove the snapshot
umount /mnt/snapshot
lvremove -f /dev/ictweb/varsnap
Save the Bash script in the appropriate cron directory for daily, weekly or monthly backups. For example, in /etc/cron.weekly for weekly backups. Make sure the script is executable:
sgordon@host:~$ sudo chmod a+x /etc/cron.weekly/backup-kvm-images
You now have automatic weekly backups of running KVM virtual machines. For those backups to be more useful, you should move them to another computer or even offsite.
Created on Thu, 07 Jun 2012, 7:04pm
Last changed on Thu, 10 Oct 2013, 11:52am