[lxc-devel] [PATCH 1/6] Move lxcbr0 setup logic into lxc.net script

Serge Hallyn serge.hallyn at ubuntu.com
Mon Aug 11 15:37:10 UTC 2014


Quoting Michael H. Warfield (mhw at WittsEnd.com):
> On Thu, 2014-07-31 at 08:53 +0200, Martin Pitt wrote:
> > Factor this out of the lxc-net.conf upstart job, so that it can be used by
> > init.d scripts and systemd units, too.
> 
> Crap.  Never fails.  I was in Europe when this came out.
> 
> Looking at lxc.net, I would say it's going to break some existing setups
> (notably mine) where lxcbr0 is already setup.  Yes, we can set
> LXC_BRIDGE to no but we should also include some autodetect logic such
> that, if lxcbr0 already exists, this doesn't commit random acts of
> terrorism.

That logic should already be there.  If /sys/class/net/lxcbr0 already
exists, then start will do nothing;  if /run/lxc/network_up does not
exist then stop will do nothing.

> For example, my lxcbr0 on Fedora 20 is a bridge bridge (I happen to have
> LOTS of IPv4 address space so eth0 or whatever is bridged to the bridge
> in static networking), not a nat bridge.  I'm also not real sure how
> this use of iptables is going to play with firewalld on Fedora or CentOS
> 7 either (or maybe Oracle 7 if they're using firewalld).  I have to
> examine that (and I'm not a big fan of firewalld).
> 
> This also potentially impacts the default lxc.confg for Fedora, CentOS,
> Oracle, and possibly others, that had been depending on libvirt for
> setting up virbr0 as a natted bridge.  That's in configure with this:
> 
> redhat|centos|fedora|oracle|oracleserver)
> 	 distroconf=default.conf.libvirt
> 
> So we either have to change those defaults and change all preexisting
> systems and containers, or this has a high probability of doing the
> wrong thing even on new setups.  At the very least, if "lxc.network.link
> = " is not "lxcbr0" it won't work properly for new containers using that
> default.  It will set up an unnecessary bridge and firewall rules where
> containers are using virbr0.  At worst, it'll break existing setups
> where lxcbr0 is already set up in static networking in a conflicting
> manner.
> 
> I'm already looking at the "make rpm" breakage.  I'll look at this as
> well.  At the very least, there has to be a "do no harm" check early in
> on the lxc.net script and exit in start() if lxcbr0 already exists.
> 
> I'd like to hear from Dwight and the Oracle side since he did most of
> the sysvinit stuff for Oracle (most particularly the bridge wait code
> based on the lxc.conf default) and now we both have to deal with the
> systemd side of things for Oracle, Fedora, and CentOS (and possibly
> Suse).
> 
> Regards,
> Mike
> 
> > Part of https://launchpad.net/bugs/1312532
> > ---
> >  config/init/upstart/lxc-net.conf | 88 +----------------------------------
> >  src/lxc/Makefile.am              |  1 +
> >  src/lxc/lxc.net                  | 99 ++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 102 insertions(+), 86 deletions(-)
> >  create mode 100755 src/lxc/lxc.net
> > 
> > diff --git a/config/init/upstart/lxc-net.conf b/config/init/upstart/lxc-net.conf
> > index 279cd1e..38f6ea3 100644
> > --- a/config/init/upstart/lxc-net.conf
> > +++ b/config/init/upstart/lxc-net.conf
> > @@ -4,89 +4,5 @@ author "Serge Hallyn <serge.hallyn at canonical.com>"
> >  start on starting lxc
> >  stop on stopped lxc
> >  
> > -env USE_LXC_BRIDGE="true"
> > -env LXC_BRIDGE="lxcbr0"
> > -env LXC_ADDR="10.0.3.1"
> > -env LXC_NETMASK="255.255.255.0"
> > -env LXC_NETWORK="10.0.3.0/24"
> > -env LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
> > -env LXC_DHCP_MAX="253"
> > -env LXC_DHCP_CONFILE=""
> > -env varrun="/run/lxc"
> > -env LXC_DOMAIN=""
> > -
> > -pre-start script
> > -	[ -f /etc/default/lxc ] && . /etc/default/lxc
> > -
> > -	[ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }
> > -
> > -	use_iptables_lock="-w"
> > -	iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > -	cleanup() {
> > -		# dnsmasq failed to start, clean up the bridge
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > -		iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > -		iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > -		iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > -		iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > -		ifconfig ${LXC_BRIDGE} down || true
> > -		brctl delbr ${LXC_BRIDGE} || true
> > -	}
> > -
> > -	if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > -		if [ ! -f ${varrun}/network_up ]; then
> > -			# bridge exists, but we didn't start it
> > -			stop;
> > -		fi
> > -		exit 0;
> > -	fi
> > -
> > -	# set up the lxc network
> > -	brctl addbr ${LXC_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
> > -	echo 1 > /proc/sys/net/ipv4/ip_forward
> > -	mkdir -p ${varrun}
> > -	ifconfig ${LXC_BRIDGE} ${LXC_ADDR} netmask ${LXC_NETMASK} up
> > -	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > -	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > -	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > -	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > -	iptables $use_iptables_lock -I FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > -	iptables $use_iptables_lock -I FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > -	iptables $use_iptables_lock -t nat -A POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE
> > -	iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > -
> > -	LXC_DOMAIN_ARG=""
> > -	if [ -n "$LXC_DOMAIN" ]; then
> > -		LXC_DOMAIN_ARG="-s $LXC_DOMAIN -S /$LXC_DOMAIN/"
> > -	fi
> > -	dnsmasq $LXC_DOMAIN_ARG -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${LXC_DHCP_CONFILE} --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${LXC_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${LXC_BRIDGE}.leases --dhcp-authoritative || cleanup
> > -	touch ${varrun}/network_up
> > -end script
> > -
> > -post-stop script
> > -	[ -f /etc/default/lxc ] && . /etc/default/lxc
> > -	[ -f "${varrun}/network_up" ] || exit 0;
> > -	# if $LXC_BRIDGE has attached interfaces, don't shut it down
> > -	ls /sys/class/net/${LXC_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;
> > -
> > -	if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > -		use_iptables_lock="-w"
> > -		iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > -		ifconfig ${LXC_BRIDGE} down
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > -		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > -		iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > -		iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > -		iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > -		iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > -		pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
> > -		rm -f ${varrun}/dnsmasq.pid
> > -		brctl delbr ${LXC_BRIDGE}
> > -	fi
> > -	rm -f ${varrun}/network_up
> > -end script
> > +pre-start exec /usr/share/lxc/lxc.net start
> > +post-stop exec /usr/share/lxc/lxc.net stop
> > diff --git a/src/lxc/Makefile.am b/src/lxc/Makefile.am
> > index cdc6833..ee74e3c 100644
> > --- a/src/lxc/Makefile.am
> > +++ b/src/lxc/Makefile.am
> > @@ -255,6 +255,7 @@ endif
> >  install-exec-local: install-soPROGRAMS
> >  	mkdir -p $(DESTDIR)$(datadir)/lxc
> >  	install -c -m 644 lxc.functions $(DESTDIR)$(datadir)/lxc
> > +	install -c -m 755 lxc.net $(DESTDIR)$(datadir)/lxc
> >  	mv $(DESTDIR)$(libdir)/liblxc.so $(DESTDIR)$(libdir)/liblxc.so.$(VERSION)
> >  	cd $(DESTDIR)$(libdir); \
> >  	ln -sf liblxc.so.$(VERSION) liblxc.so.$(firstword $(subst ., ,$(VERSION))); \
> > diff --git a/src/lxc/lxc.net b/src/lxc/lxc.net
> > new file mode 100755
> > index 0000000..5ea4f1d
> > --- /dev/null
> > +++ b/src/lxc/lxc.net
> > @@ -0,0 +1,99 @@
> > +#!/bin/sh
> > +set -eu
> > +
> > +USE_LXC_BRIDGE="true"
> > +LXC_BRIDGE="lxcbr0"
> > +LXC_ADDR="10.0.3.1"
> > +LXC_NETMASK="255.255.255.0"
> > +LXC_NETWORK="10.0.3.0/24"
> > +LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
> > +LXC_DHCP_MAX="253"
> > +LXC_DHCP_CONFILE=""
> > +varrun="/run/lxc"
> > +LXC_DOMAIN=""
> > +
> > +start() {
> > +	[ -f /etc/default/lxc ] && . /etc/default/lxc
> > +
> > +	[ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }
> > +
> > +	use_iptables_lock="-w"
> > +	iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > +	cleanup() {
> > +		# dnsmasq failed to start, clean up the bridge
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > +		iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > +		iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > +		iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > +		iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > +		ifconfig ${LXC_BRIDGE} down || true
> > +		brctl delbr ${LXC_BRIDGE} || true
> > +	}
> > +
> > +	if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > +		if [ ! -f ${varrun}/network_up ]; then
> > +			# bridge exists, but we didn't start it
> > +			stop;
> > +		fi
> > +		exit 0;
> > +	fi
> > +
> > +	# set up the lxc network
> > +	brctl addbr ${LXC_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
> > +	echo 1 > /proc/sys/net/ipv4/ip_forward
> > +	mkdir -p ${varrun}
> > +	ifconfig ${LXC_BRIDGE} ${LXC_ADDR} netmask ${LXC_NETMASK} up
> > +	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > +	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > +	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > +	iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > +	iptables $use_iptables_lock -I FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > +	iptables $use_iptables_lock -I FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > +	iptables $use_iptables_lock -t nat -A POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE
> > +	iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > +
> > +	LXC_DOMAIN_ARG=""
> > +	if [ -n "$LXC_DOMAIN" ]; then
> > +		LXC_DOMAIN_ARG="-s $LXC_DOMAIN -S /$LXC_DOMAIN/"
> > +	fi
> > +	dnsmasq $LXC_DOMAIN_ARG -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${LXC_DHCP_CONFILE} --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${LXC_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${LXC_BRIDGE}.leases --dhcp-authoritative || cleanup
> > +	touch ${varrun}/network_up
> > +}
> > +
> > +stop() {
> > +	[ -f /etc/default/lxc ] && . /etc/default/lxc
> > +	[ -f "${varrun}/network_up" ] || exit 0;
> > +	# if $LXC_BRIDGE has attached interfaces, don't shut it down
> > +	ls /sys/class/net/${LXC_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;
> > +
> > +	if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> > +		use_iptables_lock="-w"
> > +		iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> > +		ifconfig ${LXC_BRIDGE} down
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> > +		iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> > +		iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> > +		iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> > +		iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> > +		iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> > +		pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
> > +		rm -f ${varrun}/dnsmasq.pid
> > +		brctl delbr ${LXC_BRIDGE}
> > +	fi
> > +	rm -f ${varrun}/network_up
> > +}
> > +
> > +if [ "$1" = start ]; then
> > +	start
> > +elif [ "$1" = stop ]; then
> > +	stop
> > +else
> > +	echo "Usage: $0 start|stop" >&2
> > +	exit 1
> > +fi
> > +
> > -- 
> > 2.0.1
> > 
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
> > 
> 
> -- 
> Michael H. Warfield (AI4NB) | (770) 978-7061 |  mhw at WittsEnd.com
>    /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
>    NIC whois: MHW9          | An optimist believes we live in the best of all
>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
> 



> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel



More information about the lxc-devel mailing list