Chris Schuld’s Blog

Disk space can be easily controlled via OpenVZ but I have yet to find anyone to actually explain what the heck to “really” do when you need to add more! Everything I have found about OpenVZ just explains the parameters and never shows you how to do it easily. When I need to adjust disk space on an VPS it is usually when I have someone beating up my ear on the phone or my IM so I needed a fast way to expand the disk without worrying about the details.

There are three parameters in OpenVZ which are directly related to disk usage. They are disk_quota, diskspace and diskinodes. NOTE: there are a lot of other parameters that control and effect the disk but this tutorial will only cover the basics!

The parameter disk_quota is a YES or NO value which disables the file system quotas; if you are not worried about the quotas set it to NO and stop reading. Otherwise; leave it set at YES and continue.

The parameter diskspace is the count of 1K blocks available to the VPS in a soft and hard limit. The hard limit is a stop point similar to filling up a physical disk – when you are out, you are out. The soft limit is when the bean counters get angry and the quotatime timer starts. On a basic installation and VPS setup you will have a 1048576 1K blocks as a soft limit and 1153024 1K blocks as a hard limit. The numbers are not crazy as they are derived from base2. Thus, 1048576 1K blocks is 1GB of disk space. Add an additional 10.2MB to the disk space and you arrive at the 1048576 1K blocks. These are the basic numbers for the basic template that ships with OpenVZ.

The parameter diskinodes is the total number of files, directories and links you can have in the container. Think of them as Post-it® notes and each file, directory and link gets a single note. The default basic number is 200,000 for a soft limit for 1GB of disk space and 220,000 for the hard limit. Normally *nix systems will set aside enough inodes for one inode per 4K disk space block. In the default template for OpenVZ they are setting aside enough inodes for 5.2K blocks. Which I’ll write off as either (a) a magic number or (b) a unique calculation I am not familiar with. Thus, because the 4K block inode count for 1GB of disk space should be 262,144 inodes we’ll use the default template values for our calculations and simply multiply times the number of GB requested.

So…

Now…

The question is how do you adjust them quickly and easily. In this example we are going to work with units of GBs. If you need more granularity you will need to divide it back out to MBs but Gigabytes works great for our needs:

First, we need to define the soft and hard limits, next we apply the updated diskspace numbers and finally set the inode numbers correctly based on the ratio we know from the default template:

Here are the commands (and note below for a quick and easy Perl script):

cid=1324
gb=5
vzctl set ${cid} --diskspace $((1048576 * ${gb})):$((1153434 * ${gb})) --save
vzctl set ${cid} --diskinodes $((200000 * ${gb})):$((220000 * ${gb})) --save

#!/usr/bin/perl
 
# display the commands to update an OpenVZ VPS with new disk space requirements
# 2009/11/15 - Chris Schuld (chris@chrisschuld.com)
 
use strict;
 
print "Enter VPS CID: "; my $_CID = <STDIN>; chomp($_CID);
print "Enter SOFT Diskspace Limit (ex 10GB):"; my $_SOFT = <STDIN>; chomp($_SOFT); $_SOFT =~ s/[^0-9]//g;
print "Enter HARD Diskspace Limit (ex 11GB):"; my $_HARD = <STDIN>; chomp($_HARD); $_HARD =~ s/[^0-9]//g;
my $_INODE_SOFT = ( 200000 * $_SOFT );
my $_INODE_HARD = ( 220000 * $_HARD );
print "Run these commands:\n";
print "vzctl set $_CID --diskspace ".$_SOFT."G:".$_HARD."G --save\n";
print "vzctl set $_CID --diskinodes $_INODE_SOFT:$_INODE_HARD --save\n";

There are a few items required for installing vzdump for OpenVZ on CentOS.

First, you’ll need an MTA – I suggest making sure you have postfix installed; if you have postfix installed the initial RPM requirement for “MTA” will be handled for you. Next, you’ll need cstream. This installation is slightly more tricky because (as far as I know) there is no real way to gain this from yum unless you use the DAG Wieers repo. Also, depending on what you have already installed you will likely need the Simple Locking file I/O library for Perl.

Here is how you get vzdump on a clean version of CentOS (via the hostnode):

rpm -ivh "ftp://ftp.pbone.net/mirror/ftp.freshrpms.net/pub/freshrpms/pub/dag/redhat/el5/en/x86_64/RPMS.dag/cstream-2.7.4-3.el5.rf.x86_64.rpm"
wget http://dag.wieers.com/rpm/packages/perl-LockFile-Simple/perl-LockFile-Simple-0.206-1.el5.rf.noarch.rpm
rpm -ivh perl-LockFile-Simple-0.206-1.el5.rf.noarch.rpm
/bin/rm perl-LockFile-Simple-0.206-1.el5.rf.noarch.rpm
rpm -ivh "http://www.proxmox.com/cms_proxmox/cms/upload/vzdump/vzdump-1.2-4.noarch.rpm"

Since version 1.2-4 of vzdump the location of the modules is not “automatic” and have found it necessary to export the location of the PVE libraries that vzdump requires via this command:

export PERL5LIB=/usr/share/perl5/

All said and done there has to be a better way to do this… anyone… anyone??

All of our servers are currently based off of the Phoenix, Arizona, USA Timezone. This script allows this conversion from each HN (Host Node):

 
#!/bin/bash
for f in `ls /vz/private`
do
  vzctl exec $f rm -f /etc/localtime 2>/dev/null
  vzctl exec $f ln -s /usr/share/zoneinfo/America/Phoenix /etc/localtime
done

Here are commands to help adjust the RAM for an OpenVZ VPS:

64MB Guaranteed, 128MB Burstable

cid=1000
vzctl set ${cid} --vmguarpages $((64 * 64)) --save
vzctl set ${cid} --privvmpages $((64 * 128)) --save

256MB Guaranteed, 512MB Burstable

cid=1000
vzctl set ${cid} --vmguarpages $((256 * 256)) --save
vzctl set ${cid} --privvmpages $((256 * 512)) --save

512MB Guaranteed, 1024MB Burstable

cid=1000
vzctl set ${cid} --vmguarpages $((512 * 512)) --save
vzctl set ${cid} --privvmpages $((512 * 1024)) --save

1024MB Guaranteed, 2048MB Burstable

cid=1000
vzctl set ${cid} --vmguarpages $((1024 * 1024)) --save
vzctl set ${cid} --privvmpages $((1024 * 2048)) --save

I still use a lot of NFS connections on my equipment and when I create OpenVZ VPS systems I need them to have access to NFS. Here are the steps I use:

From the Host Node (HN):

modprobe nfs
vzctl set 101 --features "nfs:on" --save

From the VPS:

yum -y install nfs-utils nfs-utils-lib
chkconfig --levels 345 portmap on
/etc/init.d/portmap start

Here is my quick-and dirty way to build DNS servers using OpenVZ, CentOS and Bind/Named. This assumes you are creating a new server via OpenVZ. Although Bind is easy to admin with the configuration files recently I have found it is easier to simply admin the zones with Webmin. This setup will create the VPS, install Bind and install Webmin.

Create the VPS:

cid=1161
cd /vz/template/cache/
wget http://download.openvz.org/template/precreated/centos-5-x86_64.tar.gz
vzctl create ${cid} --ostemplate centos-5-x86_64 --config vps.basic
vzctl set ${cid} --hostname [HOSTNAMEHERE] --save
vzctl set ${cid} --ipadd [IP] --save
vzctl set ${cid} --nameserver [IP] --save
vzctl start ${cid}
vzctl exec ${cid} passwd
vzctl enter ${cid}

From inside the VPS I install bind and webmin

yum -y install bind bind-chroot bind-libs bind-utils caching-nameserver
cd /root
wget http://prdownloads.sourceforge.net/webadmin/webmin-1.480-1.noarch.rpm
rpm -Uvh webmin-1.480-1.noarch.rpm

Now I simply visit webmin’s panel and tap in any new zones (or copy over our zones from another box).

OpenVZ and virtual serving makes this time-consuming task of bringing new boxes up a simple task!

There are a few tutorials out there on installing OpenVZ in CentOS 5.3. Here are the steps I use to install OpenVZ on a brand new installation of CentOS 5.3:

Note: this tutorial / walkthrough is for use 64 bit only

1. Update the box via yum
2. Install the OpenVZ Repository and grab the GPG key
3. Install OpenVZ
4. Install OpenVZ Controller and Quota system
5. Configure the local system for the OpenVZ kernel
6. Update OpenVZ’s ARP Settings
7. Disable SELINUX
8. Reboot

yum -y update
cd /etc/yum.repos.d
wget http://download.openvz.org/openvz.repo
rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ
yum -y install ovzkernel.x86_64
yum -y install vzctl.x86_64 vzquota.x86_64

Now; we need to configure the sysctl.conf file for OpenVZ

perl -pi -e 's/net\.ipv4\.ip_forward = 0/net\.ipv4\.ip_forward = 1/' /etc/sysctl.conf
perl -pi -e 's/kernel\.sysrq = 0/kernel\.sysrq = 1/' /etc/sysctl.conf
echo -e "\n\nnet.ipv4.conf.default.proxy_arp = 0\nnet.ipv4.conf.all.rp_filter = 1\nnet.ipv4.conf.default.send_redirects = 1\nnet.ipv4.conf.all.send_redirects = 0\nnet.ipv4.icmp_echo_ignore_broadcasts=1\nnet.ipv4.conf.default.forwarding=1\nkernel.ve_allow_kthreads=1\n" >> /etc/sysctl.conf
perl -pi -e 's/NEIGHBOUR_DEVS=detect/NEIGHBOUR_DEVS=all/' /etc/vz/vz.conf

Disable SELINUX

vi /etc/sysconfig/selinux

Reboot the machine

reboot

Here are the steps I continue to use over and over to build utilities/project servers in OpenVZ. Utilities servers host web apps, mailing applications, etc for my company. We usually build them as self-contained little appliance-like servers. My vision and goal is simple == better every day!

First, we create the virtual machine:

cid=1164
cd /vz/template/cache/
wget http://download.openvz.org/template/precreated/centos-5-x86_64.tar.gz
vzctl create ${cid} --ostemplate centos-5-x86_64 --config vps.basic
vzctl set ${cid} --hostname [HOSTNAMEHERE] --save
vzctl set ${cid} --ipadd [IP] --save
vzctl set ${cid} --nameserver [IP] --save
vzctl start ${cid}
vzctl exec ${cid} passwd
vzctl enter ${cid}

Now that we are in the virtual machine and away from the HN we add the REMI repo and apply the updates directly on the box.

wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
wget http://rpms.famillecollet.com/enterprise/remi-release-5.rpm
rpm -Uvh remi-release-5*.rpm epel-release-5*.rpm
/bin/rm epel-release-5-3.noarch.rpm remi-release-5.rpm
 
yum --enablerepo remi -y update
yum --enablerepo remi -y install httpd php php-devel php-pear php-gd php-xsl php-mbstring php-mcrypt php-mysql mysql

Add Postfix and switch it on via system-switch-mail

yum --enablerepo remi -y install postfix system-switch-mail
system-switch-mail

Now… customer configurations and setups…

There are a lot of ways to backup an OpenVPS without powering them down. I have two critical VPS systems both operating phone/PBX apps (asterisk) which I need to backup and I cannot get them to backup correctly while powered up (driving me nuts). So weekly (early on Sunday mornings) I backup them up by quickly powering them off; backing them up and powering them back up.

Side Note: Yes, yes, begin the hate mail: I power down the boxes; which creates all of the 65sec of down time. If anyone is calling my office at 2AM on Sunday morning and can’t leave a message please email me and I’ll adjust our backup schedule!

The script also pushes the backup images to a mount at /nfs/backup (a backup NAS system we have in place)

Here is how I do it:

#!/bin/sh
 
if mount|grep -q ' nfs ' && df -T | grep -w nfs | grep -w "\/nas\/backup" | grep -q -wv "100%";then
 
        # date in YYYYMMDD format
        today=`date +%Y%m%0e`;
 
        echo -e "Stopping VZ 105"
        /usr/sbin/vzctl stop 105
 
        echo -e "Dumping VZ 105"
        /usr/bin/vzdump --suspend 105
 
        echo -e "Starting VZ 105"
        /usr/sbin/vzctl start 105
 
        echo -e "Compressing Output"
        /bin/gzip -9 /vz/dump/vzdump-105.tar
 
        echo -e "Backing up - moving file to NAS"
        /bin/mv /vz/dump/vzdump-105.tar.gz /nas/backup/__hostname__here__/vzdump-105-$today.tar.gz
 
else
        echo Error: the NFS mount for the backup NAS does not appear to be correct
fi

If you happen to have a Asterisk solution for voice on your network occasionally it is nice to be able to send faxes from your network without the need of a PSTN line at your endpoint. At my company we do this by running a fax server on an OpenVZ VPS on the same subnet as our Asterisk PBX. Here is how we build our fax server:

First, we build the OpenVZ VPS with CentOS and add the /dev/ptmx device (your VPS will likely already have this on it):

vzctl create 1057 --ostemplate centos-5-i386-default --config vps.basic
vzctl set 1057 --hostname fax.aztecsoftware.net --save
vzctl set 1057 --ipadd 10.0.0.57 --save
vzctl set 1057 --nameserver 10.0.0.15 --save
mknod --mode 666 /vz/private/1057/dev/ptmx c 5 2

(Note: there is a good chance the 98 UNIX device /dev/ptmx is going to already be there from your template — you can disregard any errors you get during the “forced” creation of that device)

Next, we start the VPS and enter the VE by running the following:

vzctl start 1057
vzctl enter 1057

Next, we start the update and install process via yum and a few RPMs I built for Ghostscript:

yum -y update
yum -y install rpm-build make libtiff-devel zlib-devel gcc gcc-c++ pam-devel openldap-devel freeglut libjpeg-devel libICE libSM libXt cairo urw-fonts
 
rpm -Uvh http://chrisschuld.com/centos52/jasper-libs-1.900.1-8.i386.rpm
rpm -Uvh http://chrisschuld.com/centos52/jasper-1.900.1-8.i386.rpm
rpm -Uvh http://chrisschuld.com/centos52/jasper-utils-1.900.1-8.i386.rpm
rpm -Uvh http://chrisschuld.com/centos52/jasper-devel-1.900.1-8.i386.rpm
rpm -Uvh http://chrisschuld.com/centos52/ghostscript-fonts-8.11-1ht.noarch.rpm
rpm -Uvh http://chrisschuld.com/centos52/ghostscript-8.63-1.i386.rpm
rpm -Uvh http://chrisschuld.com/centos52/ghostscript-devel-8.63-1.i386.rpm
 
cd /root
wget http://internap.dl.sourceforge.net/sourceforge/hylafax/hylafax-5.2.7-1.src.rpm
wget http://internap.dl.sourceforge.net/sourceforge/hylafax/hylafax.spec
 
rpm -i hylafax-5.2.7-1.src.rpm
rpmbuild -bb hylafax.spec
rpm -i /usr/src/redhat/RPMS/i386/hylafax-5.2.7-1.i386.rpm
chkconfig --levels 345 hylafax on

Now, because we are on Asterisk we’ll use an IAXModem for our communication device; if you do have access to a “real” PSTN by all means use a fax-capable modem and don’t use the IAXModem because it is not a good 100% solution. However, here is our IAXModem method:

cd /root
wget http://voxel.dl.sourceforge.net/sourceforge/iaxmodem/iaxmodem-1.1.1.tar.gz
tar xzvf iaxmodem-1.1.1.tar.gz
cd iaxmodem-1.1.1
./configure
make
cp iaxmodem /usr/local/sbin/
mkdir /etc/iaxmodem /var/log/iaxmodem
touch /var/log/iaxmodem/iaxmodem
pushd .; cd /etc/iaxmodem
wget http://chrisschuld.com/wp-content/uploads/2008/11/ttyIAX0
popd
pushd .; cd /etc/init.d/
wget http://chrisschuld.com/wp-content/uploads/2008/11/iaxmodem
popd
cp config.ttyIAX /var/spool/hylafax/etc/config.ttyIAX0
 
### Add our local configuration differences
vi /etc/iaxmodem/ttyIAX0
 
### Add our local configuration differences
vi /var/spool/hylafax/etc/config.ttyIAX0
 
chmod +x /etc/init.d/iaxmodem
chkconfig --add iaxmodem
/usr/local/sbin/iaxmodem
echo "iax0:2345:respawn:/usr/sbin/faxgetty ttyIAX0" >> /etc/inittab

Now, we REBOOT to make sure everything comes up:

reboot

After the VE re-appears we run our fax setup program:

faxsetup

That should do 90% of the work for you — no you just have to configure the server for your needs!