Reduce Proxmox LXC Backup Size and Time

  • 0

Reduce Proxmox LXC Backup Size and Time

Category : How-to

Get Social!

Proxmox backs up guests byte-for-byte in a compressed archive. Looking at LXC backups specifically, the file system is compressed into the target backup file with just a few exceptions – temp files aren’t included. You can also add your own exceptions by editing the vzdump.conf to exclude specific file patterns.

All that said, one of the biggest disk space wasters is the cache directory for apt which caches the installation packages for software you have installed. This can generally be safely removed on internet connected machines which will reduce your overall backup size.

For example, a newly created Debian LXC that’s been recently updated shows a total of 206MB of disk used.

After clearing this with the command apt-get clean we can see the space has mostly been freed.

Considering this whole container is only consuming approximately 1GB of disk space, 200MB is quite significant.

vzdump hooks

Now we can see how much space we can save, we need to make Proxmox issue the apt-get clean command before it creates the backup of our container.

vzdump, the utility which Proxmox uses to perform backups has the ability to call a script for various stages of the backup process – these stages are:

  • backup-start
  • backup-end
  • backup-abort
  • log-end
  • pre-stop
  • pre-restart
  • post-restart

We can use these hooks to run our own commands at any of these points of the backup. For the goal of this blog post, we want to run the apt-get clean command at the point of backup-start.

Create a script on your Proxmox host with the following content:

Now edit your vzdump.conf file and add the following line to point to your new script. Remember to change the location of where your script is – I’ve just saved mine in /root/.

 


  • 1

dd Cheat Sheet

Get Social!

dd is one of the most versatile IO tools available for Linux. It’s used in a variety of ways ranging from Disk Benchmarking through to creating SWAP files and copying downloaded disk images to physical disks.

dd takes the following common switches:

  • if is the input file name and location.
  • of is the name and location of the output file.
  • bs is the block size that will be used to read and/ or write the file. Increasing this can help with performance  or dictate how much data will be read or written.
  • count is the number of blocks that will be used.
  • seek is the number of blocks on the output file that will be skipped before writing any data.
  • skip is the number of blocks that will be skipped on the input file before starting to read data.
  • conv is a comma separated list of additional parameters that can be used. See the man dd for more information.

The below headings will list a few example uses of dd in a typical Linux environment.

Backup disk partition with dd

You can use dd to copy an entire disk partition to a virtual disk file. This can be useful for creating a backup or to clone the disk to another machine.

You can use this method to read a CD-ROM, USB drive or Flash disk to a file in the same way – just make sure the device is inserted and point the if= part of the dd command to the relevant /dev/ device.

You could also compress the image as part of the process with gzip.

Restore disk partition with dd

Similar to the above command, you can use dd to replace a disk’s partition with a virtual disk file.

If you compressed the image then you can decompress it first all in one go:

Create a fixed size file with dd

You can create a fixed size file with DD that will be created in the location you specify.

This will create a file in /root/test of 1024 bytes in size. Increase either bs or count to change the size of the file. The resulting size will be bs count. You can also use shorhand sizes such as K, M and G with bs, for example bs=1G,

Create a SWAP file with dd

dd can be used to create a SWAP file that can be used as a SWAP device by your computer. This is often needed with smaller instances on Cloud providers such as AWS.

The starting point is the same as the above command to create a file with the size that you’d like to use for swap. See my other blog post for more info.

Split a file with dd

dd can be used to read just part of a file, given offset and length coordinates. The below example will skip the first 100 bytes of the file and output the proceeding 10 bytes (byte 101 – 111).

You could repeat this process to split a large file into multiple smaller files, to be able to email it for example.

Merge multiple files with dd

You can merge multiple files into a single file with dd. Following on from the above split example, the below will rejoin the 3 file parts into a single file.

Convert text to lower case with dd

You can use the conv switch with dd to transform ascii text from upper case to lower case and visa-versa. Using lcase and ucase in the conv switch will instruct dd to convert the text as it’s written.

The below example will convert all characters in the filetoconvert.txt. file to lower case.

 


  • 10

Move Proxmox Container to Different Storage

Get Social!

2015-03-05 00_18_04-Proxmox Virtual Environment storageA task often required when new storage is added or removed, and containers grow over time is to move a container onto another storage device.

The  Proxmox  Web GUI does not give us the ability to migrate a container from one storage device to another directly. To move a container onto different storage we have to take a backup of the container and restore it to the same ID with a different storage device specified. This can be time laborious when working with several containers.

The below script allows you to move an OpenVZ container from one storage device to another. The process requires that the container be stopped, which the script will handle.

Save the below script into a file called migrate.

Set execution permissions on the script:

The script has several parameters which are detailed below:

  • -d is specified if you would like the script to delete the temporary backup after the process has completed. Leave this out if you would like the backup tar file to be kept, just in case anything goes wrong.
  • -s is required to specify the name of the target storage. You can find this from the Proxmox Web GUI.
  • -c is required for the container ID to migrate.

In addition, the script contains the variable TMP. This will be the location of the backup tar created as part of the migration process and must contain enough space to store the content of the container being migrated. You can change this to suit your environment.

Example command:

 


  • 0

Backup all Proxmox OpenVZ containers in one go

Category : How-to

Get Social!

proxmox logo gradThe below script is a bash script which works with Proxmox and the OpenVZ commands to backup all known containers to a specified folder.

You will need to set BACKUP_PATH to be the folder where you would like the backups to be stored and COMPRESS to a value which specifies the compression used. COMPRESS values can be:

  • 0 – no compression.
  • 1 – default compression (usually lzo).
  • gzip – gzip compression.
  • lzo – lzo compression.

Paste the below file into /bin/backup_all and make sure it’s executable.


  • 0

Restore a single Proxmox OpenVZ Container From The command Line

Get Social!

proxmox logo gradI mostly use Proxmox from the command line, or terminal, and I have created a few scripts to perform common and repetitive tasks.

The below script will restore a single OpenVZ container to the latest backup file available in the dump directory. The scripts takes a parameter for the container VMID to restore from backup. If the container exists, it will be stopped and removed before restoring the latest backup file available in the backup directory.

The script iterates through all of your backup files and only restores the latest based on the date in the file name, and not the date of the file creation or modified.

You will need to set the BACKUP_PATH variable to the location of your backup folder with no trailing slash, and BACKUP_EXT with the extension used for your chosen backup format.

If you save this script in the /bin then you can call the script from the terminal without having to move to the scripts directory. Create the file and paste the below script into it.

Make the script executable using chmod.

Use the below command, and substitute [VMID] with the container VMID to restore, to run the script.

See my other script on restoring multiple OpenVZ containers in Proxmox.


  • 0

Restore all Proxmox OpenVZ Containers From The command Line

Get Social!

proxmox logo gradI use Proxmox to host a development environment using OpenVZ containers. I take frequent backups of all OpenVZ containers in the event I need to roll back any development work.

The below script restores all OpenVZ containers which are available in the backup folder, but not available in the Proxmox GUI. Using this script, you can remove the containers in Proxmox which you would like to restore and run the script to restore the latest backup.

The script iterates through all of your backup files and only restores the latest based on the date in the file name.

You will need to set the BACKUP_PATH variable to the location of your backup folder with no trailing slash, and BACKUP_EXT with the extension used for your chosen backup format.

If you save this script in the /bin then you can call the script from the terminal without having to move to the scripts directory. Create the file and paste the below script into it.

Make the script executable using chmod.

Use the below command to run the script and restore all containers which are missing from backup.