Bridge Networking On OpenVZ Containers Using VETH Devices

Wed, 08/13/2008 - 23:52 — sandip

1. Host will be open on LAN and guests on WAN. Additionally, there is a bridged LAN for guests.
2. We will be using bridge networking for protecting the Host Network and saving IP addresses, also giving flexibility with the guest network setup.
3. Configure LAN Eth1 port to 192.168.1.2
4. WAN Eth0 port is not assigned any IP address.
6. Install the required bridge-utils package via:

yum install bridge-utils

* Network:

# cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=<host.domain.tld>
GATEWAY=192.168.1.1
GATEWAYDEV="eth1"

* WAN Bridge device br0:

# cat /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
BROADCAST=xxx.xxx.xxx.xxx />NETMASK=255.255.255.24
NETWORK=xxx.xxx.xxx.xxx
ONBOOT=yes
GATEWAY=xxx.xxx.xxx.xxx
TYPE=Bridge

* WAN eth0 device:

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:30:48:65:12:b4
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br0

* LAN Bridge device br1:

# cat /etc/sysconfig/network-scripts/ifcfg-br1
DEVICE=br1
ONBOOT=yes
TYPE=Bridge
IPADDR=192.168.1.2
NETMASK=255.255.255.0
GATEWAY=192.168.1.1

* LAN eth1 device:

# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
HWADDR=00:30:48:65:12:b5
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br1

Persistent VE Network

Setup persistent veth with the below steps and custom script:

Edit the container's configuration:

Add the network customization section to "/etc/vz/conf/{VEID}.conf"

# Network customization section
CONFIG_CUSTOMIZED="yes&quot;
VETH_IP_ADDRESS="<WAN IP/MASK>;<LAN IP/MASK>"
BRIDGEDEV="br0;br1";
NETIF="<ETH0;ETH1>;"

Add the veth devices:

vzctl set {VEID} --netif_add eth0 --save
vzctl set {VEID} --netif_add eth1 --save

Create a custom network configuration script

Place the custom script as ”/usr/local/sbin/vznetcfg.custom” and make it executable, which will be called when the container is started:

#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIFS=`echo $NETIF | sed 's/;/\n/g'`
for NETIFX in $NETIFS
do

  NETIF_OPTIONS=`echo $NETIFX | sed 's/,/\n/g'`
  for str in $NETIF_OPTIONS; do \
    &nbsp;     # getting 'ifname' parameter value
    &nbsp;     if [[ "$str" =~ "^ifname=" ]]; then
    &nbsp;    &nbsp;    &nbsp;   # remove the parameter name from the string (along with '=')
    &nbsp;    &nbsp;    &nbsp;   VEIFNAME=${str#*=};
    &nbsp;     fi
    &nbsp;     # getting 'host_ifname' parameter value
    &nbsp;     if [[ "$str" =~ "^host_ifname=" ]]; then
    &nbsp;    &nbsp;    &nbsp;   # remove the parameter name from the string (along with '=')
    &nbsp;    &nbsp;    &nbsp;   VZHOSTIF=${str#*=};
    &nbsp;     fi
  done

  # Get the bridge names
  BRIDGEX=${BRIDGEDEV%%;*}
  BRIDGEL=${BRIDGEDEV#*;}
  BRIDGEDEV=$BRIDGEL;

  VETH_IP_ADDRX=${VETH_IP_ADDRESS%%;*}
  VETH_IP_ADDRL=${VETH_IP_ADDRESS#*;}
  VETH_IP_ADDRESS=$VETH_IP_ADDRL;

  if [ ! -n "$VETH_IP_ADDRX" ]; then
     echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
     exit 1
  fi


  if [ ! -n "$VZHOSTIF" ]; then
     echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
     exit 1
  fi

  if [ ! -n "$VEIFNAME" ]; then
     echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
     exit 1
  fi

  echo "Initializing interface $VZHOSTIF for VE$VEID."
  $ifconfig $VZHOSTIF 0

  VEROUTEDEV=$VZHOSTIF

  if [ -n "$BRIDGEX" ]; then
     echo "Adding interface $VZHOSTIF to the bridge $BRIDGEX."
     VEROUTEDEV=$BRIDGEX
     $brctl addif $BRIDGEX $VZHOSTIF
  fi

  # Up the interface $VEIFNAME link in VE$VEID
  $vzctl exec $VEID $ip link set $VEIFNAME up

  for IP in $VETH_IP_ADDRX; do
     echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
     $vzctl exec $VEID $ip address add $IP dev $VEIFNAME

     # removing the netmask
     IP_STRIP=${IP%%/*};

     echo "Adding a route from VE0 to VE$VEID."
     $ip route add $IP_STRIP dev $VEROUTEDEV
  done

  if [ -n "$VE0_IP" ]; then
     echo "Adding a route from VE$VEID to VE0."
     $vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
  fi

  if [ -n "$VE_DEFAULT_GATEWAY"; ]; then
     echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
     $vzctl exec $VEID \
    &nbsp;     $ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
  fi
done

# Set the default mac adresses:
$ifconfig br0 hw ether $($ifconfig eth0 | awk '{print $5; exit}')
$ifconfig br1 hw ether $($ifconfig eth1 | awk '{print $5; exit}')

# Bring venet0 down
$ifconfig venet0 down

exit 0

Make the script to run on container start:

Create the file "/etc/vz/vznet.conf" with the following contents:

EXTERNAL_SCRIPT="/usr/local/sbin/vznetcfg.custom"

Remove routes to container with veth-bridge from bridge

Create "/etc/sysconfig/vz/vps.umount":

#!/bin/bash
# /etc/sysconfig/vz/$VEID.umount or /etc/sysconfig/vz/vps.umountr /># a script to remove routes to container with veth-bridge from bridge 

CTCONFIGFILE=/etc/vz/conf/$VEID.conf
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $CTCONFIGFILE

if [ ! -n "$VETH_IP_ADDRESS" ]; then
   exit 0
fi

if [ ! -n "$BRIDGEDEV" ]; then
   exit 0
fi

for IP in $VETH_IP_ADDRESS; do
   # removing the netmask
   IP_STRIP=${IP%%/*};
   
   echo "Remove a route from CT0 to CT$VEID using $IP_STRIP."
   $ip route del $IP_STRIP dev $BRIDGEDEV
done

# Set the default mac adresses:
echo "Setting default mac addressed for br0 and br1"
$ifconfig br0 hw ether $($ifconfig eth0 | awk '{print $5; exit}')
$ifconfig br1 hw ether $($ifconfig eth1 | awk '{print $5; exit}')

exit 0

Note:
Seperate table route rules can be added within VEs via:

ip rule add from 192.168.1.2 table 6
ip route add default dev eth1 via 192.168.1.1 table 6

Bookmark/Search this post with

sandip's blog
Login or register to post comments

Comments

Tue, 09/30/2008 - 14:15 — sandip Listing rules and routing tables

List rules and get the table ID:

ip rule list

List routing tables corresponding to the table ID:

ip route list table {table_id}

LinuxWebLog.com