[lxc-devel] [PATCH 1/6] Move lxcbr0 setup logic into lxc.net script
Serge Hallyn
serge.hallyn at ubuntu.com
Mon Aug 11 15:37:10 UTC 2014
Quoting Michael H. Warfield (mhw at WittsEnd.com):
> On Thu, 2014-07-31 at 08:53 +0200, Martin Pitt wrote:
> > Factor this out of the lxc-net.conf upstart job, so that it can be used by
> > init.d scripts and systemd units, too.
>
> Crap. Never fails. I was in Europe when this came out.
>
> Looking at lxc.net, I would say it's going to break some existing setups
> (notably mine) where lxcbr0 is already setup. Yes, we can set
> LXC_BRIDGE to no but we should also include some autodetect logic such
> that, if lxcbr0 already exists, this doesn't commit random acts of
> terrorism.
That logic should already be there. If /sys/class/net/lxcbr0 already
exists, then start will do nothing; if /run/lxc/network_up does not
exist then stop will do nothing.
> For example, my lxcbr0 on Fedora 20 is a bridge bridge (I happen to have
> LOTS of IPv4 address space so eth0 or whatever is bridged to the bridge
> in static networking), not a nat bridge. I'm also not real sure how
> this use of iptables is going to play with firewalld on Fedora or CentOS
> 7 either (or maybe Oracle 7 if they're using firewalld). I have to
> examine that (and I'm not a big fan of firewalld).
>
> This also potentially impacts the default lxc.confg for Fedora, CentOS,
> Oracle, and possibly others, that had been depending on libvirt for
> setting up virbr0 as a natted bridge. That's in configure with this:
>
> redhat|centos|fedora|oracle|oracleserver)
> distroconf=default.conf.libvirt
>
> So we either have to change those defaults and change all preexisting
> systems and containers, or this has a high probability of doing the
> wrong thing even on new setups. At the very least, if "lxc.network.link
> = " is not "lxcbr0" it won't work properly for new containers using that
> default. It will set up an unnecessary bridge and firewall rules where
> containers are using virbr0. At worst, it'll break existing setups
> where lxcbr0 is already set up in static networking in a conflicting
> manner.
>
> I'm already looking at the "make rpm" breakage. I'll look at this as
> well. At the very least, there has to be a "do no harm" check early in
> on the lxc.net script and exit in start() if lxcbr0 already exists.
>
> I'd like to hear from Dwight and the Oracle side since he did most of
> the sysvinit stuff for Oracle (most particularly the bridge wait code
> based on the lxc.conf default) and now we both have to deal with the
> systemd side of things for Oracle, Fedora, and CentOS (and possibly
> Suse).
>
> Regards,
> Mike
>
> > Part of https://launchpad.net/bugs/1312532
> > ---
> > config/init/upstart/lxc-net.conf | 88 +----------------------------------
> > src/lxc/Makefile.am | 1 +
> > src/lxc/lxc.net | 99 ++++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 102 insertions(+), 86 deletions(-)
> > create mode 100755 src/lxc/lxc.net
> >
> > diff --git a/config/init/upstart/lxc-net.conf b/config/init/upstart/lxc-net.conf
> > index 279cd1e..38f6ea3 100644
> > --- a/config/init/upstart/lxc-net.conf
> > +++ b/config/init/upstart/lxc-net.conf
> > @@ -4,89 +4,5 @@ author "Serge Hallyn <serge.hallyn at canonical.com>"
> > start on starting lxc
> > stop on stopped lxc
> >
> > -env USE_LXC_BRIDGE="true"
> > -env LXC_BRIDGE="lxcbr0"
> > -env LXC_ADDR="10.0.3.1"
> > -env LXC_NETMASK="255.255.255.0"
> > -env LXC_NETWORK="10.0.3.0/24"
> > -env LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
> > -env LXC_DHCP_MAX="253"
> > -env LXC_DHCP_CONFILE=""
> > -env varrun="/run/lxc"
> > -env LXC_DOMAIN=""
> > -
> > -pre-start script
> > - [ -f /etc/default/lxc ] && . /etc/default/lxc
> > -
> > - [ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }
> > -
> > - use_iptables_lock="-w"
> > - iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > - cleanup() {
> > - # dnsmasq failed to start, clean up the bridge
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > - iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > - iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > - iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > - iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > - ifconfig ${LXC_BRIDGE} down || true
> > - brctl delbr ${LXC_BRIDGE} || true
> > - }
> > -
> > - if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > - if [ ! -f ${varrun}/network_up ]; then
> > - # bridge exists, but we didn't start it
> > - stop;
> > - fi
> > - exit 0;
> > - fi
> > -
> > - # set up the lxc network
> > - brctl addbr ${LXC_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
> > - echo 1 > /proc/sys/net/ipv4/ip_forward
> > - mkdir -p ${varrun}
> > - ifconfig ${LXC_BRIDGE} ${LXC_ADDR} netmask ${LXC_NETMASK} up
> > - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > - iptables $use_iptables_lock -I FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > - iptables $use_iptables_lock -I FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > - iptables $use_iptables_lock -t nat -A POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE
> > - iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > -
> > - LXC_DOMAIN_ARG=""
> > - if [ -n "$LXC_DOMAIN" ]; then
> > - LXC_DOMAIN_ARG="-s $LXC_DOMAIN -S /$LXC_DOMAIN/"
> > - fi
> > - dnsmasq $LXC_DOMAIN_ARG -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${LXC_DHCP_CONFILE} --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${LXC_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${LXC_BRIDGE}.leases --dhcp-authoritative || cleanup
> > - touch ${varrun}/network_up
> > -end script
> > -
> > -post-stop script
> > - [ -f /etc/default/lxc ] && . /etc/default/lxc
> > - [ -f "${varrun}/network_up" ] || exit 0;
> > - # if $LXC_BRIDGE has attached interfaces, don't shut it down
> > - ls /sys/class/net/${LXC_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;
> > -
> > - if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > - use_iptables_lock="-w"
> > - iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > - ifconfig ${LXC_BRIDGE} down
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > - iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > - iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > - iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > - iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > - pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
> > - rm -f ${varrun}/dnsmasq.pid
> > - brctl delbr ${LXC_BRIDGE}
> > - fi
> > - rm -f ${varrun}/network_up
> > -end script
> > +pre-start exec /usr/share/lxc/lxc.net start
> > +post-stop exec /usr/share/lxc/lxc.net stop
> > diff --git a/src/lxc/Makefile.am b/src/lxc/Makefile.am
> > index cdc6833..ee74e3c 100644
> > --- a/src/lxc/Makefile.am
> > +++ b/src/lxc/Makefile.am
> > @@ -255,6 +255,7 @@ endif
> > install-exec-local: install-soPROGRAMS
> > mkdir -p $(DESTDIR)$(datadir)/lxc
> > install -c -m 644 lxc.functions $(DESTDIR)$(datadir)/lxc
> > + install -c -m 755 lxc.net $(DESTDIR)$(datadir)/lxc
> > mv $(DESTDIR)$(libdir)/liblxc.so $(DESTDIR)$(libdir)/liblxc.so.$(VERSION)
> > cd $(DESTDIR)$(libdir); \
> > ln -sf liblxc.so.$(VERSION) liblxc.so.$(firstword $(subst ., ,$(VERSION))); \
> > diff --git a/src/lxc/lxc.net b/src/lxc/lxc.net
> > new file mode 100755
> > index 0000000..5ea4f1d
> > --- /dev/null
> > +++ b/src/lxc/lxc.net
> > @@ -0,0 +1,99 @@
> > +#!/bin/sh
> > +set -eu
> > +
> > +USE_LXC_BRIDGE="true"
> > +LXC_BRIDGE="lxcbr0"
> > +LXC_ADDR="10.0.3.1"
> > +LXC_NETMASK="255.255.255.0"
> > +LXC_NETWORK="10.0.3.0/24"
> > +LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
> > +LXC_DHCP_MAX="253"
> > +LXC_DHCP_CONFILE=""
> > +varrun="/run/lxc"
> > +LXC_DOMAIN=""
> > +
> > +start() {
> > + [ -f /etc/default/lxc ] && . /etc/default/lxc
> > +
> > + [ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }
> > +
> > + use_iptables_lock="-w"
> > + iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > + cleanup() {
> > + # dnsmasq failed to start, clean up the bridge
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > + iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > + iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > + iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > + iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > + ifconfig ${LXC_BRIDGE} down || true
> > + brctl delbr ${LXC_BRIDGE} || true
> > + }
> > +
> > + if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > + if [ ! -f ${varrun}/network_up ]; then
> > + # bridge exists, but we didn't start it
> > + stop;
> > + fi
> > + exit 0;
> > + fi
> > +
> > + # set up the lxc network
> > + brctl addbr ${LXC_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
> > + echo 1 > /proc/sys/net/ipv4/ip_forward
> > + mkdir -p ${varrun}
> > + ifconfig ${LXC_BRIDGE} ${LXC_ADDR} netmask ${LXC_NETMASK} up
> > + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > + iptables $use_iptables_lock -I FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > + iptables $use_iptables_lock -I FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > + iptables $use_iptables_lock -t nat -A POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE
> > + iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > +
> > + LXC_DOMAIN_ARG=""
> > + if [ -n "$LXC_DOMAIN" ]; then
> > + LXC_DOMAIN_ARG="-s $LXC_DOMAIN -S /$LXC_DOMAIN/"
> > + fi
> > + dnsmasq $LXC_DOMAIN_ARG -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${LXC_DHCP_CONFILE} --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${LXC_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${LXC_BRIDGE}.leases --dhcp-authoritative || cleanup
> > + touch ${varrun}/network_up
> > +}
> > +
> > +stop() {
> > + [ -f /etc/default/lxc ] && . /etc/default/lxc
> > + [ -f "${varrun}/network_up" ] || exit 0;
> > + # if $LXC_BRIDGE has attached interfaces, don't shut it down
> > + ls /sys/class/net/${LXC_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;
> > +
> > + if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > + use_iptables_lock="-w"
> > + iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > + ifconfig ${LXC_BRIDGE} down
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > + iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > + iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > + iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > + iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > + pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
> > + rm -f ${varrun}/dnsmasq.pid
> > + brctl delbr ${LXC_BRIDGE}
> > + fi
> > + rm -f ${varrun}/network_up
> > +}
> > +
> > +if [ "$1" = start ]; then
> > + start
> > +elif [ "$1" = stop ]; then
> > + stop
> > +else
> > + echo "Usage: $0 start|stop" >&2
> > + exit 1
> > +fi
> > +
> > --
> > 2.0.1
> >
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >
>
> --
> Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> NIC whois: MHW9 | An optimist believes we live in the best of all
> PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
More information about the lxc-devel
mailing list