Bash Command to Check Container Disk Space in Proxmox/ OpenVZ

Bash Command to Check Container Disk Space in Proxmox/ OpenVZ

Get Social!

proxmox logo gradKeeping an eye on all of your containers disk space can be time consuming if done one by one. Luckily, the vzlist command gives us access to many of the data and counters associated with each container.

Running vzlist on it’s own will give a list of the currently running containers and a few extra columns of information such as IP address and hostname.

vzlist
CTID NPROC STATUS  IP_ADDR     HOSTNAME
100  27    running 10.10.10.10 test.jamescoyle.net

You might be thinking that the above command doesn’t show anything about containers which are currently turned off, or about disk space. You’d be right!

Adding the –all switch will include all containers, regardless of their running state.

To add the disk space information we add the –output switch followed by the column names we want to display. For disk space, the column name is helpfully named diskspace so the command will look like this:

vzlist --all --output diskspace
DSPACE
1184648

We now see the disk space of all of our containers, but with a problem. We have no idea (although in this example we do because we only have one container) which container the disk space output is for. We need to add a few more columns to the –output switch such as ctid and hostname.  (see below for a complete list of output columns)

vzlist --all --output ctid,hostname,diskspace
CTID HOSTNAME             DSPACE
100  test.jamescoyle.net  1184660

The last trick here is to add a sort using the –sort switch and the column name

vzlist --all --output ctid,hostname,diskspace --sort diskspace
CTID HOSTNAME             DSPACE
100  test.jamescoyle.net  1184660

Let’s put all that together, plus a spot of awk magic to get a nice disk full percentage that we can work with:

vzlist --all --output ctid,hostname,diskspace,diskspace.s,diskspace.h --sort diskspace | awk '{if (NR>1) {printf("%-4s %-30s %-10s %-10s %-10s %d\n", $1, $2, $3, $4, $5, $3/$4*100)} else printf("%-4s %-30s %-10s %-10s %-10s %s\n", $1, $2, $3, $4, $5, "PERC_USED")}'

Output:

CTID HOSTNAME            DSPACE  DSPACE.S DSPACE.H PERC_USED
100  test.jamescoyle.net 1184684 5242880  5767168  22

Complete list of vzlist column headings

You can get a complete list of vzlist column headings with the following command:

vzlist -L
ctid            CTID
veid            CTID
vpsid           CTID
private         PRIVATE
root            ROOT
mount_opts      MOUNT_OPTS
origin_sample   ORIGIN_SAMPLE
hostname        HOSTNAME
name            NAME
smart_name      SMARTNAME
description     DESCRIPTION
ostemplate      OSTEMPLATE
ip              IP_ADDR
nameserver      NAMESERVER
searchdomain    SEARCHDOMAIN
status          STATUS
kmemsize        KMEMSIZE
kmemsize.m      KMEMSIZE.M
kmemsize.b      KMEMSIZE.B
kmemsize.l      KMEMSIZE.L
kmemsize.f      KMEMSIZE.F
lockedpages     LOCKEDP
lockedpages.m   LOCKEDP.M
lockedpages.b   LOCKEDP.B
lockedpages.l   LOCKEDP.L
lockedpages.f   LOCKEDP.F
privvmpages     PRIVVMP
privvmpages.m   PRIVVMP.M
privvmpages.b   PRIVVMP.B
privvmpages.l   PRIVVMP.L
privvmpages.f   PRIVVMP.F
shmpages        SHMP
shmpages.m      SHMP.M
shmpages.b      SHMP.B
shmpages.l      SHMP.L
shmpages.f      SHMP.F
numproc         NPROC
numproc.m       NPROC.M
numproc.b       NPROC.B
numproc.l       NPROC.L
numproc.f       NPROC.F
physpages       PHYSP
physpages.m     PHYSP.M
physpages.b     PHYSP.B
physpages.l     PHYSP.L
physpages.f     PHYSP.F
vmguarpages     VMGUARP
vmguarpages.m   VMGUARP.M
vmguarpages.b   VMGUARP.B
vmguarpages.l   VMGUARP.L
vmguarpages.f   VMGUARP.F
oomguarpages    OOMGUARP
oomguarpages.m  OOMGUARP.M
oomguarpages.b  OOMGUARP.B
oomguarpages.l  OOMGUARP.L
oomguarpages.f  OOMGUARP.F
numtcpsock      NTCPSOCK
numtcpsock.m    NTCPSOCK.M
numtcpsock.b    NTCPSOCK.B
numtcpsock.l    NTCPSOCK.L
numtcpsock.f    NTCPSOCK.F
numflock        NFLOCK
numflock.m      NFLOCK.M
numflock.b      NFLOCK.B
numflock.l      NFLOCK.L
numflock.f      NFLOCK.F
numpty          NPTY
numpty.m        NPTY.M
numpty.b        NPTY.B
numpty.l        NPTY.L
numpty.f        NPTY.F
numsiginfo      NSIGINFO
numsiginfo.m    NSIGINFO.M
numsiginfo.b    NSIGINFO.B
numsiginfo.l    NSIGINFO.L
numsiginfo.f    NSIGINFO.F
tcpsndbuf       TCPSNDB
tcpsndbuf.m     TCPSNDB.M
tcpsndbuf.b     TCPSNDB.B
tcpsndbuf.l     TCPSNDB.L
tcpsndbuf.f     TCPSNDB.F
tcprcvbuf       TCPRCVB
tcprcvbuf.m     TCPRCVB.M
tcprcvbuf.b     TCPRCVB.B
tcprcvbuf.l     TCPRCVB.L
tcprcvbuf.f     TCPRCVB.F
othersockbuf    OTHSOCKB
othersockbuf.m  OTHSOCKB.M
othersockbuf.b  OTHSOCKB.B
othersockbuf.l  OTHSOCKB.L
othersockbuf.f  OTHSOCKB.F
dgramrcvbuf     DGRAMRB
dgramrcvbuf.m   DGRAMRB.M
dgramrcvbuf.b   DGRAMRB.B
dgramrcvbuf.l   DGRAMRB.L
dgramrcvbuf.f   DGRAMRB.F
numothersock    NOTHSOCK
numothersock.m  NOTHSOCK.M
numothersock.b  NOTHSOCK.B
numothersock.l  NOTHSOCK.L
numothersock.f  NOTHSOCK.F
dcachesize      DCACHESZ
dcachesize.m    DCACHESZ.M
dcachesize.b    DCACHESZ.B
dcachesize.l    DCACHESZ.L
dcachesize.f    DCACHESZ.F
numfile         NFILE
numfile.m       NFILE.M
numfile.b       NFILE.B
numfile.l       NFILE.L
numfile.f       NFILE.F
numiptent       NIPTENT
numiptent.m     NIPTENT.M
numiptent.b     NIPTENT.B
numiptent.l     NIPTENT.L
numiptent.f     NIPTENT.F
swappages       SWAPP
swappages.m     SWAPP.M
swappages.b     SWAPP.B
swappages.l     SWAPP.L
swappages.f     SWAPP.F
diskspace       DSPACE
diskspace.s     DSPACE.S
diskspace.h     DSPACE.H
diskinodes      DINODES
diskinodes.s    DINODES.S
diskinodes.h    DINODES.H
laverage        LAVERAGE
uptime          UPTIME
cpulimit        CPULIM
cpuunits        CPUUNI
cpus            CPUS
ioprio          IOP
onboot          ONBOOT
bootorder       BOOTORDER
layout          LAYOUT
features        FEATURES
vswap           VSWAP
disabled        DISABL

 


OpenVZ disk space issues

Get Social!

proxmox logo gradI started to receive an error in one of my Proxmox OpenVZ containers which indicated that the quota had been exceeded for disk space. Immediately I logged into the container and performed a df -h to view the disk usage. To my surprise only 50% of the disk space had been used.

Error in the logs:

kernel: VZ QUOTA: file softlimit exceeded for id=103

OpenVZ containers have two storage related limits;

  • Disk space – this is the amount of space which can be used by the container, usually measured in gigabytes.
  • inodes – this loosely translates to the number of files which can be stored on the filesystem.

Below is an example of the disk based limits configuration.

DISKSPACE="10485760:11534336"
DISKINODES="2000000:2200000"

The problem with my container was that although I wasn’t using all the allocated disk space, I was using all of the allocated inodes as I had a java application running which created 1000s of new small files every hour. CCTV recording software such as Zoneminder would likely have the same problem as it creates many small image files when recoding movement from a camera.

You can test how many inodes are in use by running df -i on the container.  An example below of a very low inode usage:

Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/simfs 10000000 116024 9883976 2% /

Increase disk space and inode quota

Proxmox does not allow you to change the inodes limits directly from the GUI. You will need to edit the configuration file for the container directly on the OS or use the vz command. The config files are stored in /etc/vz/conf by default and named VMID.conf. Edit the required config file for your container and increase the DISKSPACE and DISKINODES values as required.

vi /etc/vz/conf/103.conf
DISKSPACE="10485760:11534336"
DISKINODES="4000000:4400000"

OpenVZ has the concept of HARD and SOFT limits. A soft limit will trigger a warning but allow the applications to continue to use disk space/ inodes however when the hard limit is hit new requests for storage will be denied. A hard limit is like a physical disk being full.

DISKSPACE="SOFT:HARD"
DISKINODES="SOFT:HARD"

You can also use the word UNLIMITED to remove any virtual limits on storage however you will still be constrained by the underlying physical storage limits.

You will need to restart your container for the settings to take effect.

To avoid having to restart your container, you can use the vzctl command to apply the new limits immediately, even to a running container.

vzctl set 103 --diskspace 45G:49G --save
vzctl set 103 --diskinodes 4000000:4400000 --save

Visit our advertisers

Quick Poll

Do you use GlusterFS in your workplace?

Visit our advertisers