openvz

Bridge Networking On OpenVZ Containers Using VETH Devices

1. Host will be open on LAN and guests on WAN. Additionally, there is a bridged LAN for guests.
2. We will be using bridge networking for protecting the Host Network and saving IP addresses, also giving flexibility with the guest network setup.
3. Configure LAN Eth1 port to 192.168.1.2
4. WAN Eth0 port is not assigned any IP address.
6. Install the required bridge-utils package via:

yum install bridge-utils

* Network:

# cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=<host.domain.tld>
GATEWAY=192.168.1.1
GATEWAYDEV="eth1"

* WAN Bridge device br0:

# cat /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
BROADCAST=xxx.xxx.xxx.xxx />NETMASK=255.255.255.24
NETWORK=xxx.xxx.xxx.xxx
ONBOOT=yes
GATEWAY=xxx.xxx.xxx.xxx
TYPE=Bridge

* WAN eth0 device:

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:30:48:65:12:b4
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br0

* LAN Bridge device br1:

# cat /etc/sysconfig/network-scripts/ifcfg-br1
DEVICE=br1
ONBOOT=yes
TYPE=Bridge
IPADDR=192.168.1.2
NETMASK=255.255.255.0
GATEWAY=192.168.1.1

* LAN eth1 device:

# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
HWADDR=00:30:48:65:12:b5
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br1

Create users in OpenVZ Containers

Helper script to create users on all OpenVZ VEs simultaneously:

#!/bin/bash
# create_ve_users.sh
# Usage: ./create_ve_users.sh <username> <password> <uid> <group1,group2>

USERNAME=$1
PASSWORD=$2
USERID=$3
GROUP=$4
EXPECTED_ARGS=4

OUT_FILE=.create_users_$$ />
if [ $# -ne $EXPECTED_ARGS ]
then
  echo "Usage: `basename $0` <username> <password> <uid> <group1,group2>" />  exit 65
fi

VE_LIST=$(/usr/sbin/vzlist -H -o veid)

for VE in ${VE_LIST}
do
    &nbsp;   vzctl exec $VE \
    &nbsp;    &nbsp;    &nbsp; "echo useradd -u $USERID -G $GROUP -p \''`openssl passwd -1 $PASSWORD`'\' $USERNAME > /tmp/${OUT_FILE} ; \
    &nbsp;    &nbsp;    &nbsp; sh /tmp/${OUT_FILE} ; \
    &nbsp;    &nbsp;    &nbsp; rm -f /tmp/${OUT_FILE}"
done

exit 0

Update OpenVZ containers

If you manage several OpenVZ containers, here is a simple bash script to keep the OpenVZ containers upto date.

#!/bin/bash
# vzyum_updates.sh
# updates VEs

VE_LIST=$(/usr/sbin/vzlist -H -o veid | grep -v Warning)

for VE in ${VE_LIST}
do
  /usr/bin/vzyum $VE update
done

exit 0

OpenVZ CentOS template cache creation with vzpkgcache

Following the instruction over at OpenVZ Wiki, I've had no problems with the installation and creation of templates prior to CentOS-5.2 on x86_64 systems. However, with the latest set of updates to CentOS-5.2, the vzpkgcache seems to have been broken as sysklogd is no longer being installed by default. Below is how I got it to work:

Note: edit /vz/template/centos/5/x86_64/config/minimal.list and append .x86_64 to all except for the addons packages. Then run:

# vzpkgcache -f centos-5-x86_64-minimal

which gives the below error at the end:

sed: can't read /etc/init.d/syslog: No such file or directory
ERROR: Script install-post failed

The solution was to edit "/vz/template/centos/5/x86_64/config/install-post" and set syslog and syslog.conf sed replacements with exit status of "0".

# Disable klogd
$VZCTL exec2 $VEID \
    &nbsp;   "sed -i -e 's/daemon\\ klogd/passed\\ klogd\\ skipped/' \
    &nbsp;    &nbsp;    &nbsp; -e 's/killproc\\ klogd/passed\\ klogd\\ skipped/' \
    &nbsp;    &nbsp;    &nbsp;    &nbsp;    /etc/init.d/syslog; exit 0"
...
# Disable fsync() in syslog
$VZCTL exec2 $VEID \
    &nbsp;   "sed -i -e 's@\\\([[:space:]]\\\)\\\(/var/log/\\\)@\\\1-\\\2@'; \
    &nbsp;    &nbsp;    &nbsp; /etc/syslog.conf; exit 0"

APF on CentOS-5 OpenVZ Containers

  1. Increase the NUMIPTENT values in VE conf file to 1000 on the host:
    NUMIPTENT="1000:1000";
  2. Edit "/etc/sysconfig/vz" on the host:
    IPTABLES="ipt_REJECT ipt_tos ipt_TOS ipt_LOG ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_owner ipt_length ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state iptable_nat ip_nat_ftp ipt_recent"
  3. Make sure the above modules are loaded on the host, so it would help if you have APF on the host as well.
  4. Restart the VE
    # vzctl restart <VEID>
  5. It is normal to get ipt_recent error as below:
    Warning: Unknown iptable module: ipt_recent, skipped

    You can verify the modules loaded via:

    # vzctl exec <VEID> grep ipt_recent /proc/net/ip_tables_matches

    Note: ipt_recent is required for passive ftp to work, else... will need to specify passive ftp ports in ftp conf file and open those ports via apf as well.

  6. Here is a typical apf config on a VE with CentOS-5 running ISPConfig.

    DEVEL_MODE="0"
    IFACE_IN="venet0"
    IFACE_OUT="venet0"
    IFACE_TRUSTED=""r />SET_MONOKERN="1"r />IG_TCP_CPORTS="21,22,25,53,80,81,110,143,443" />IG_UDP_CPORTS="53"
    EGF="1"
    EG_TCP_CPORTS="21,25,80,443,43"
    EG_UDP_CPORTS="20,21,53"

openvz user and quota issues

Recently, I've had a client who's had issues with uploading files and general functioning of the ISPConfig hosting control panel.

So the first thing I checked out was "/proc/user_beancounters", and everything seemed normal there.

Quick check with `vzquota` turned out that the inodes were maxed out.

Blocks and Inodes can also be checked/displayed within the container via `df -h` and `df -i`.

Additionally, since he had a lot of users, he was maxed out on the users limits too. Noticed that with `repquota -a` which pulled up a huge number of users.

Increasing the appropriate limits with vzctl on diskspace, diskinodes and quotaugidlimit resolved all issues.

UserBeanCounters resources, vmguarpages and kmemsize explained

If you're hosted on a VPS, the below would explain if you are getting the resources that you paid for:

As mentioned in the resources (/proc/user_beancounters):

vmguarpages	0	30,000	2,147,483,647	4KB pages	\
      Memory allocation guarantee

This is the guaranteed RAM you get which works out to be:

30000 x 4 / 1024 = 117.1875 MB

Accordingly kmemsize is set to:

kmemsize	7,167,393	12,288,832	13,517,715	bytes	\
     Size of unswappable memory, allocated by the operating system kernel

Minimum kmemsize should be 10% of the vmguarpages, which is correct for the current setup:

12288832/1024/1024 = 11.7 MB == 10% of 117 (vmgaurpges)
Comment