Author Archives: James Coyle

Restore a single Proxmox OpenVZ Container From The command Line

Get Social!

proxmox logo gradI mostly use Proxmox from the command line, or terminal, and I have created a few scripts to perform common and repetitive tasks.

The below script will restore a single OpenVZ container to the latest backup file available in the dump directory. The scripts takes a parameter for the container VMID to restore from backup. If the container exists, it will be stopped and removed before restoring the latest backup file available in the backup directory.

The script iterates through all of your backup files and only restores the latest based on the date in the file name, and not the date of the file creation or modified.

You will need to set the BACKUP_PATH variable to the location of your backup folder with no trailing slash, and BACKUP_EXT with the extension used for your chosen backup format.

If you save this script in the /bin then you can call the script from the terminal without having to move to the scripts directory. Create the file and paste the below script into it.

vi /bin/restore_one
#!/bin/bash
#
# Filename : restore_one
# Description : Restores a single OpenVZ Proxmox container to the latest backup file
#               available in the dump folder.
# Author : James Coyle
#
# Version:
# -Date -Author -Description
# 01-11-2013 James Coyle Initial
#
#

BACKUP_PATH=/var/lib/vz/dump
BACKUP_EXT=tar.lzo

# Do not change
SEARCH_PATH=$BACKUP_PATH/vzdump-openvz-$1-*.$BACKUP_EXT

function display-useage
{
  echo "Useage $0 [vmid to restore]"
  echo "Example: $0 999"

}

# Check dir exists
if [ ! -d $BACKUP_PATH ]; then
  echo "The directory $BACKUP_PATH does not exist"
  exit 99
fi

# Check if argument is present
if [ -z "$1" ]
then
  echo "Argument not present."
  display-useage
  exit 99
fi

# Check if vmid is available, on or off
VMON=$(vzlist | grep -P "[ ]+$1[ ]+")
VMOFF=$(vzlist --stopped | grep -P "[ ]+$1[ ]+")

if [ -n "$VMON" ]; then
  echo "Requesting stop of container."
  vzctl stop $1
  echo "Requesting deletion of container."
  vzctl delete $1
elif [ -n "$VMOFF" ]; then
  echo "Container is stopped."
  echo "Requesting deletion of container."
  vzctl delete $1
else
  echo "Container is not live."
fi

# Get unique VMIDs
for F in $SEARCH_PATH
do
  FILENAME=${F##*/}
  FILE_DATE=${FILENAME:18:19}
  FILE_DATE=${FILE_DATE//[_\-]/}

  if [ -z "$BACKUP_FILE" ]; then
    BACKUP_FILE=$F
  fi

  TEST_FILENAME=${BACKUP_FILE##*/}
  TEST_FILE_DATE=${TEST_FILENAME:18:19}
  TEST_FILE_DATE=${TEST_FILE_DATE//[_\-]/}
  if [ "$FILE_DATE" -gt "$TEST_FILE_DATE" ]; then
    BACKUP_FILE=$F
  fi
done

if [ -n $BACKUP_FILE ]; then
  # Restore VM
  echo "Restoring $1 with $BACKUP_FILE..."
  vzrestore $BACKUP_FILE $1
else
  echo "No backup file for VMID $1 exists."
fi

Make the script executable using chmod.

chmod +x /bin/restore_one

Use the below command, and substitute [VMID] with the container VMID to restore, to run the script.

restore_one [VMID]

See my other script on restoring multiple OpenVZ containers in Proxmox.


Restore all Proxmox OpenVZ Containers From The command Line

Get Social!

proxmox logo gradI use Proxmox to host a development environment using OpenVZ containers. I take frequent backups of all OpenVZ containers in the event I need to roll back any development work.

The below script restores all OpenVZ containers which are available in the backup folder, but not available in the Proxmox GUI. Using this script, you can remove the containers in Proxmox which you would like to restore and run the script to restore the latest backup.

The script iterates through all of your backup files and only restores the latest based on the date in the file name.

You will need to set the BACKUP_PATH variable to the location of your backup folder with no trailing slash, and BACKUP_EXT with the extension used for your chosen backup format.

If you save this script in the /bin then you can call the script from the terminal without having to move to the scripts directory. Create the file and paste the below script into it.

vi /bin/restore_all
#!/bin/bash
#
# Filename : restore_all
# Description : Restores all missing OpenVZ containers in Proxmox to the latest version available in the dump folder.
# Author : James Coyle
#
# Version:
# -Date       -Author      -Description
# 01-11-2013  James Coyle  Initial
#
#

BACKUP_PATH=/var/lib/vz/dump
BACKUP_EXT=tar.lzo

# Do not change
SEARCH_PATH=$BACKUP_PATH/vzdump-openvz-*.$BACKUP_EXT

# Check dir exists
if [ ! -d $BACKUP_PATH ]; then
  echo "The directory $BACKUP_PATH does not exist"
  exit 99
fi

# Get unique VMIDs
for F in $SEARCH_PATH
do
  FILENAME=${F##*/}
  FILE_DATE=${FILENAME:18:19}
  FILE_DATE=${FILE_DATE//[_\-]/}
  VMID=${FILENAME:14:3}
  if [ -n $FILE_VIMS[$VMID] ]; then
    FILE_VIMS[$VMID]=$F
  fi

  TEST_FILENAME=${FILE_VIMS[$VMID]##*/}
  TEST_FILE_DATE=${TEST_FILENAME:18:19}
  TEST_FILE_DATE=${TEST_FILE_DATE//[_\-]/}
  if [ "$FILE_DATE" -gt "$TEST_FILE_DATE" ]; then
    FILE_VIMS[$VMID]=F
  fi
done

# Restore VM
for I in ${!FILE_VIMS[*]}
do
  echo "Restoring $I with ${FILE_VIMS[$I]}..."
  vzrestore ${FILE_VIMS[$I]} $I
done

Make the script executable using chmod.

chmod +x /bin/restore_all

Use the below command to run the script and restore all containers which are missing from backup.

restore_all

Advanced GlusterFS Log Rotation

Category : How-to

Get Social!

gluster-orange-ant

If installing GlusterFS on Debian using the launchpad repository then a log rotate entry will be automatically set up. This entry handles all the logs created by the GlusterFS application by rotating them daily and keeping 7 days of old log files.

Other packages or install methods may  vary and log and configuration paths may be different.

On Debian and Ubuntu the log files are kept in the following location:

/var/log/glusterfs/

And the logrotate configuration files is found here:

/etc/logrotate.d/glusterfs-common

The default configuration is to rotate the logs each day, compress old logs and keep them for 7 days. Here is the default configuration file:

/var/log/glusterfs/*.log {
 daily
 rotate 7
 delaycompress
 compress
 notifempty
 missingok
 postrotate
 [ ! -f /var/run/glusterd.pid ] || kill -HUP `cat /var/run/glusterd.pid`
 endscript
}

You can edit this file directly to make any changes you may need in your GlusterFS environment. See my cheat sheet for details on the logrotate commands.


logrotate Cheat Sheet

Get Social!

logrotate Most Linux applications create log files that need to be managed by either archiving or deleting old log files. This process is called log file rotation. The most common log rotation utility for Linux is conveniently called logrotate. logrotate is configured using the main configuration file, or the logrotate configuration folder. The configuration file usually holds the global configuration and the pointer to the configuration folder. In common Linux distributions such as Ubuntu and Debian, the logrotate configuration file can be found:

/etc/logrotate.conf

And the configuration folder can be found:

/etc/logrotate.d/

The configuration folder is usually where you would add new log file configurations which are unique for a particular log file, or set of log files. For example, if you were to add a new log rotation action for the Ubuntu package manager apt, you may use something like below:

vi /etc/logrotate.d/apt
/var/log/apt/term.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}
/var/log/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

logrotate runs each day by default, and is invoked using the daily cron job. The below subject detail the common components of creating a logrotate configuration file.

Create a new empty template

To create a new logrotate configuration file, you need to create a new file in /etc/logrotate.d/. You will then need to add a reference to the log files you wish to rotate. This can be directly to a single file or use pattern matching to match a group of log files. The below example matches all log files in /var/log/myapp/ which have a .log extension.

/var/log/myapp/*.log {

}

You will need to add further commands to this template before it becomes useful. Further commands will be added inside the and } tags.

Rotate based on log file size

Use the size keyword to rotate the log file when it exceeded a given file size. The below example rotates a file when it reaches 10 KB.

/var/log/myapp/*.log {
  size 10k
}

Rotate based on time (Month, Week or Day)

You can rotate logs using the monthlyweekly or daily keyword to create a new log based on duration. The keywords explain them selves, and they can be used in conjunction with the size keyword to rotate on which ever criteria is met first.

/var/log/myapp/*.log {
  size 10k
  weekly
}

Limit how many log files are kept after rotation by number

The rotate keyword allows you to specify how many old, rotated, log files are kept before logrotate deletes them. The rotate keyword requires an integer to specify how many old log files are kept.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
}

The above example will keep 8 old log files.

Limit how many files are kept after rotation by date

You can specify how long to keep rotated files using the maxage keyword. Any rotated log files which are older than maxage will be deleted. The below example will keep rotated log files for 56 days.

/var/log/myapp/*.log {
  size 10k
  weekly
  maxage 56
}

Compress rotated log files

Log files which have been rotated can be compressed to save disk space. Gzip is used by default.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  compress
}

You can change the default gzip compression to another format by specifying the compresscmd command and a different executable to use. The below example changes the compression format to bzip2 for better file compression.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  compress
  compresscmd /bin/bzip2
}

Ignore missing log files

If a log file does not exist when logrotate is running then an error will be thrown. You can use the keyword missingok to avoid this scenario and instruct logrotate to ignore the log file if it does not exist.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  missingok
}

Continue writing to the same log file after rotation

Usually when a log file is rotated the log file is moved to a new location. Some applications may throw an error, and others may continue to write to the relocated file. The copytruncate keyword copies all the log in the file to a new file and then truncates the original file. This keeps the original log file in place and also allows rotation to continue.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  copytruncate
}

 

Let me know in the comments if you think anything is missing.


Simple GlusterFS log rotation

Category : How-to

Get Social!

gluster-orange-ant

You’ll be glad to know that GlusterFS has built in log rotation! This means that you can use a simple gluster command to rotate the log for a specific volume. This is very helpful when troubleshooting where it can be easiest to truncate logs before regenerating the issue.

For enduring log rotation I recommend using logrotate which I will cover in a future blog post.

Logs are rotated per volume, so you will need to know the volume name before issuing the command to rotate the log. Use the below command to list all the volumes available on your server:

gluster volume info all | grep "Volume Name"

Use the gluster command, replacing [VOL_NAME] with the volume name of the log file you would like to rotate.

gluster volume log rotate [VOL_NAME]

In the below example, the logs for volume datastore will be rotated.

gluster volume log rotate datastore

Below is the result on the file system displayed with the ls command.

ls -l /var/log/glusterfs/bricks/
total 28
-rw------- 1 root root 119 Nov 1 19:06 mnt-gfs_block.log
-rw------- 1 root root 9085 Nov 1 13:46 mnt-gfs_block.log.1383313581
-rw------- 1 root root 236 Nov 1 13:48 mnt-gfs_block.log.1383313684
-rw------- 1 root root 236 Nov 1 13:49 mnt-gfs_block.log.1383313742
-rw------- 1 root root 236 Nov 1 19:06 mnt-gfs_block.log.1383332808

And that’s it! Your log file will be moved and have a time stamp appended, and a new log will be started for the volume.

See my other post on using logrotate for more advanced log rotation.


chkconfig Cheat Sheet

Get Social!

Linux penguinControlling startup services in Linux distributions such as Red Hat Enterprise Linux (RHEL), CentOS and Oracle Enterprise Linux (OEL) can be done using a Gnome GUI or a command line utility. The command line utility is called chkconfig and can list existing, add new or remove services from the operating systems startup list.

As Linux operating systems have multiple states, or runlevels, you need to make sure you add any new services to the correct runlevels. For example, you would not want to start a web service application before starting networking. See my post on runlevels.

What services are available for startup?

Use the –list switch to see your existing services and when they should be running.

chkconfig --list

An example output is below. This shows all of the machine runlevels and what the state of the service will be.

chkconfig --list

Note: This output shows SysV services only and does not include native systemd services. SysV configuration data might be overridden by native systemd configuration.

modules_dep 0:off 1:off 2:on 3:on 4:on 5:on 6:off
netconsole 0:off 1:off 2:off 3:off 4:off 5:off 6:off
network 0:off 1:off 2:off 3:on 4:off 5:off 6:off

In this example, there are three services available. netconsole is not available at any runlevels and is therefore never started automatically, network is available only in runlevel 3 and modules_dep is available in runlevel 2 and runlevel 3.

You could also use the example below to detail the runlevels of a single service only.

chkconfig network --list

Add a new service with chkconfig

Adding a new service is  easily done with the below command. The below example shows the service network being enabled to start at the next machine boot.

chkconfig network on

Use the –level switch to enable the service at specific runlevels. Use the below example to enable the service at runlevel 3.

chkconfig network on --level 3

Remove a service with chkconfig

Removing an existing service is done with the below command. The below example shows the service network being disabled from automatic start.

chkconfig network off

Use the –level switch to remove the service from specific runlevels. Use the below example to disable the service at runlevel 3.

chkconfig network off --level 3

Start a service

Starting a service is done using the command service followed by the service name and the command to start the service.

service network start

Stop a service

Use the stop keyword with service to stop a service.

service network stop

Check the status of a service

Each service has a status, usually running or not running. Some services, such as network, may have a different output and output more information on the service.

service network status

 

Let me know in the comments if you think anything is missing.


Visit our advertisers

Quick Poll

Do you use GlusterFS in your workplace?

Visit our advertisers