[lxc-devel] [PATCH 1/6] Move lxcbr0 setup logic into lxc.net script
Michael H. Warfield
mhw at WittsEnd.com
Sat Aug 9 14:28:51 UTC 2014
On Thu, 2014-07-31 at 08:53 +0200, Martin Pitt wrote:
> Factor this out of the lxc-net.conf upstart job, so that it can be used by
> init.d scripts and systemd units, too.
Crap. Never fails. I was in Europe when this came out.
Looking at lxc.net, I would say it's going to break some existing setups
(notably mine) where lxcbr0 is already setup. Yes, we can set
LXC_BRIDGE to no but we should also include some autodetect logic such
that, if lxcbr0 already exists, this doesn't commit random acts of
terrorism.
For example, my lxcbr0 on Fedora 20 is a bridge bridge (I happen to have
LOTS of IPv4 address space so eth0 or whatever is bridged to the bridge
in static networking), not a nat bridge. I'm also not real sure how
this use of iptables is going to play with firewalld on Fedora or CentOS
7 either (or maybe Oracle 7 if they're using firewalld). I have to
examine that (and I'm not a big fan of firewalld).
This also potentially impacts the default lxc.confg for Fedora, CentOS,
Oracle, and possibly others, that had been depending on libvirt for
setting up virbr0 as a natted bridge. That's in configure with this:
redhat|centos|fedora|oracle|oracleserver)
distroconf=default.conf.libvirt
So we either have to change those defaults and change all preexisting
systems and containers, or this has a high probability of doing the
wrong thing even on new setups. At the very least, if "lxc.network.link
= " is not "lxcbr0" it won't work properly for new containers using that
default. It will set up an unnecessary bridge and firewall rules where
containers are using virbr0. At worst, it'll break existing setups
where lxcbr0 is already set up in static networking in a conflicting
manner.
I'm already looking at the "make rpm" breakage. I'll look at this as
well. At the very least, there has to be a "do no harm" check early in
on the lxc.net script and exit in start() if lxcbr0 already exists.
I'd like to hear from Dwight and the Oracle side since he did most of
the sysvinit stuff for Oracle (most particularly the bridge wait code
based on the lxc.conf default) and now we both have to deal with the
systemd side of things for Oracle, Fedora, and CentOS (and possibly
Suse).
Regards,
Mike
> Part of https://launchpad.net/bugs/1312532
> ---
> config/init/upstart/lxc-net.conf | 88 +----------------------------------
> src/lxc/Makefile.am | 1 +
> src/lxc/lxc.net | 99 ++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 102 insertions(+), 86 deletions(-)
> create mode 100755 src/lxc/lxc.net
>
> diff --git a/config/init/upstart/lxc-net.conf b/config/init/upstart/lxc-net.conf
> index 279cd1e..38f6ea3 100644
> --- a/config/init/upstart/lxc-net.conf
> +++ b/config/init/upstart/lxc-net.conf
> @@ -4,89 +4,5 @@ author "Serge Hallyn <serge.hallyn at canonical.com>"
> start on starting lxc
> stop on stopped lxc
>
> -env USE_LXC_BRIDGE="true"
> -env LXC_BRIDGE="lxcbr0"
> -env LXC_ADDR="10.0.3.1"
> -env LXC_NETMASK="255.255.255.0"
> -env LXC_NETWORK="10.0.3.0/24"
> -env LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
> -env LXC_DHCP_MAX="253"
> -env LXC_DHCP_CONFILE=""
> -env varrun="/run/lxc"
> -env LXC_DOMAIN=""
> -
> -pre-start script
> - [ -f /etc/default/lxc ] && . /etc/default/lxc
> -
> - [ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }
> -
> - use_iptables_lock="-w"
> - iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> - cleanup() {
> - # dnsmasq failed to start, clean up the bridge
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> - iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> - iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> - iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> - iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> - ifconfig ${LXC_BRIDGE} down || true
> - brctl delbr ${LXC_BRIDGE} || true
> - }
> -
> - if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> - if [ ! -f ${varrun}/network_up ]; then
> - # bridge exists, but we didn't start it
> - stop;
> - fi
> - exit 0;
> - fi
> -
> - # set up the lxc network
> - brctl addbr ${LXC_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
> - echo 1 > /proc/sys/net/ipv4/ip_forward
> - mkdir -p ${varrun}
> - ifconfig ${LXC_BRIDGE} ${LXC_ADDR} netmask ${LXC_NETMASK} up
> - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> - iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> - iptables $use_iptables_lock -I FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> - iptables $use_iptables_lock -I FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> - iptables $use_iptables_lock -t nat -A POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE
> - iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> -
> - LXC_DOMAIN_ARG=""
> - if [ -n "$LXC_DOMAIN" ]; then
> - LXC_DOMAIN_ARG="-s $LXC_DOMAIN -S /$LXC_DOMAIN/"
> - fi
> - dnsmasq $LXC_DOMAIN_ARG -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${LXC_DHCP_CONFILE} --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${LXC_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${LXC_BRIDGE}.leases --dhcp-authoritative || cleanup
> - touch ${varrun}/network_up
> -end script
> -
> -post-stop script
> - [ -f /etc/default/lxc ] && . /etc/default/lxc
> - [ -f "${varrun}/network_up" ] || exit 0;
> - # if $LXC_BRIDGE has attached interfaces, don't shut it down
> - ls /sys/class/net/${LXC_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;
> -
> - if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> - use_iptables_lock="-w"
> - iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> - ifconfig ${LXC_BRIDGE} down
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> - iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> - iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> - iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> - iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> - iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> - pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
> - rm -f ${varrun}/dnsmasq.pid
> - brctl delbr ${LXC_BRIDGE}
> - fi
> - rm -f ${varrun}/network_up
> -end script
> +pre-start exec /usr/share/lxc/lxc.net start
> +post-stop exec /usr/share/lxc/lxc.net stop
> diff --git a/src/lxc/Makefile.am b/src/lxc/Makefile.am
> index cdc6833..ee74e3c 100644
> --- a/src/lxc/Makefile.am
> +++ b/src/lxc/Makefile.am
> @@ -255,6 +255,7 @@ endif
> install-exec-local: install-soPROGRAMS
> mkdir -p $(DESTDIR)$(datadir)/lxc
> install -c -m 644 lxc.functions $(DESTDIR)$(datadir)/lxc
> + install -c -m 755 lxc.net $(DESTDIR)$(datadir)/lxc
> mv $(DESTDIR)$(libdir)/liblxc.so $(DESTDIR)$(libdir)/liblxc.so.$(VERSION)
> cd $(DESTDIR)$(libdir); \
> ln -sf liblxc.so.$(VERSION) liblxc.so.$(firstword $(subst ., ,$(VERSION))); \
> diff --git a/src/lxc/lxc.net b/src/lxc/lxc.net
> new file mode 100755
> index 0000000..5ea4f1d
> --- /dev/null
> +++ b/src/lxc/lxc.net
> @@ -0,0 +1,99 @@
> +#!/bin/sh
> +set -eu
> +
> +USE_LXC_BRIDGE="true"
> +LXC_BRIDGE="lxcbr0"
> +LXC_ADDR="10.0.3.1"
> +LXC_NETMASK="255.255.255.0"
> +LXC_NETWORK="10.0.3.0/24"
> +LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
> +LXC_DHCP_MAX="253"
> +LXC_DHCP_CONFILE=""
> +varrun="/run/lxc"
> +LXC_DOMAIN=""
> +
> +start() {
> + [ -f /etc/default/lxc ] && . /etc/default/lxc
> +
> + [ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }
> +
> + use_iptables_lock="-w"
> + iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> + cleanup() {
> + # dnsmasq failed to start, clean up the bridge
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> + iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> + iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> + iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> + iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> + ifconfig ${LXC_BRIDGE} down || true
> + brctl delbr ${LXC_BRIDGE} || true
> + }
> +
> + if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> + if [ ! -f ${varrun}/network_up ]; then
> + # bridge exists, but we didn't start it
> + stop;
> + fi
> + exit 0;
> + fi
> +
> + # set up the lxc network
> + brctl addbr ${LXC_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
> + echo 1 > /proc/sys/net/ipv4/ip_forward
> + mkdir -p ${varrun}
> + ifconfig ${LXC_BRIDGE} ${LXC_ADDR} netmask ${LXC_NETMASK} up
> + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> + iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> + iptables $use_iptables_lock -I FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> + iptables $use_iptables_lock -I FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> + iptables $use_iptables_lock -t nat -A POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE
> + iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> +
> + LXC_DOMAIN_ARG=""
> + if [ -n "$LXC_DOMAIN" ]; then
> + LXC_DOMAIN_ARG="-s $LXC_DOMAIN -S /$LXC_DOMAIN/"
> + fi
> + dnsmasq $LXC_DOMAIN_ARG -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${LXC_DHCP_CONFILE} --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${LXC_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${LXC_BRIDGE}.leases --dhcp-authoritative || cleanup
> + touch ${varrun}/network_up
> +}
> +
> +stop() {
> + [ -f /etc/default/lxc ] && . /etc/default/lxc
> + [ -f "${varrun}/network_up" ] || exit 0;
> + # if $LXC_BRIDGE has attached interfaces, don't shut it down
> + ls /sys/class/net/${LXC_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;
> +
> + if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
> + use_iptables_lock="-w"
> + iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
> + ifconfig ${LXC_BRIDGE} down
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
> + iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
> + iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
> + iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
> + iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
> + iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> + pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
> + rm -f ${varrun}/dnsmasq.pid
> + brctl delbr ${LXC_BRIDGE}
> + fi
> + rm -f ${varrun}/network_up
> +}
> +
> +if [ "$1" = start ]; then
> + start
> +elif [ "$1" = stop ]; then
> + stop
> +else
> + echo "Usage: $0 start|stop" >&2
> + exit 1
> +fi
> +
> --
> 2.0.1
>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
--
Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140809/eefd3e93/attachment.sig>
More information about the lxc-devel
mailing list