[lxc-users] Hotplug new network interfaces not working
Michael H. Warfield
mhw at WittsEnd.com
Wed May 14 01:22:42 UTC 2014
On Tue, 2014-05-13 at 20:45 -0400, CDR wrote:
> I copied your file, copy-paste, and the file ended in
> /usr/share/lxc/templates/lxc-fedora20
... NO!!!
> Then chmod +x
> then lxc-create -t fedora20 -n masterfe
> Am I wrong?
Unbelievable wrong. You should have never done that.
Whatever you've done, undo it.
You need to start, from scratch, with a true base deployment. If you've
compiled your own manually, uninstall it. Wipe it OUT! On Fedora 20, I
would pull down the tarball for 1.0.3 at:
https://linuxcontainers.org/downloads/lxc-1.0.3.tar.gz
Then rebuild it into an rpm using "rpmbuild -ta lxc-1.0.3.tar.gz".
Then install the appropriate rpm's using "yum localupdate ..." for the
correct rpms (I do lxc, lxc-libs, lxc-lua, and lxc-devel).
Then run "lxc-create -t fedora -n masterfe"
NOTE: Not "-t fedora20"! Use the real template!
Don't try and take short cuts and don't copy files around thinking they
will just work. Hint: They won't.
Quit trying to "roll your own". We're working very hard to make this
work and when you "roll your own" and do things like this, you make a
lot of work for us figuring out what the hell you're doing (MOST
especially when you are not telling us what your doing till we pry it
out of you) and making in impossible to support you.
First your script to create your "roll your own" to create your
template. Then you copy a .in file to a final template file without
knowing it needed to be autoconf'ed. You're expending a great deal of
effort to create your own problems.
You're trying too hard. You're doing things you don't need to do and
efforts you don't need to make and making assumptions that are, quite
frankly, wrong.
Build the packages properly, install them properly, and create the
containers properly. You'll save yourself and us a lot of grief.
Regards,
Mike
> On Tue, May 13, 2014 at 8:39 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> > On Tue, 2014-05-13 at 20:30 -0400, CDR wrote:
> >> I am trying to create a new container inside a Fedora 20 box, fully updated.
> >> It blows up immediately.
> >> I can give you access if you wish. Maybe we can make a better template.
> >> Note: I am root.
> >> uname -r
> >> 3.14.3-200.fc20.x86_64
> >>
> >>
> >> lxc-create -t fedora20 -n masterfe
> >> Host CPE ID from /etc/os-release: cpe:/o:fedoraproject:fedora:20
> >> Checking cache download in @LOCALSTATEDIR@/cache/lxc/fedora/x86_64/20/rootfs ...
> >
> > Woa woa woa woa!
> >
> > Something is SERIOUSLY wrong here.
> >
> > You should NEVER see "@LOCALSTATEDIR@/cache/lxc/..." That's something
> > from the lxc.fedora.in file that gets processed by autoconf to create
> > the correct paths. That should NEVER been seen in a running template
> > script.
> >
> > What did you do? did you copy lxc.fedora.in to lxc.fedora somewhere?
> > How did you create your LXC installation? You've got some serious
> > problems in your deployed installation if you are seeing any sort of
> > message that says "@....@".
> >
> > Anything from here down is considered invalid and disregarded.
> >
> >> Downloading fedora minimal ...
> >> Fetching rpm name from
> >> http://mirror.pnl.gov/fedora/linux/releases/20/Everything/x86_64/os//Packages/f...
> >> % Total % Received % Xferd Average Speed Time Time Time Current
> >> Dload Upload Total Spent Left Speed
> >> 100 288 100 288 0 0 892 0 --:--:-- --:--:-- --:--:-- 897
> >> 0 0 0 215k 0 0 240k 0 --:--:-- --:--:-- --:--:-- 2340k
> >> Fetching fedora release rpm from
> >> http://mirror.pnl.gov/fedora/linux/releases/20/Everything/x86_64/os//Packages/f/fedora-release-20-1.noarch.rpm......
> >> % Total % Received % Xferd Average Speed Time Time Time Current
> >> Dload Upload Total Spent Left Speed
> >> 100 34036 100 34036 0 0 107k 0 --:--:-- --:--:-- --:--:-- 107k
> >> Bootstrap Environment testing...
> >>
> >> OS fedora is whitelisted. Installation Bootstrap Environment not required.
> >>
> >> rpm: arguments to --root (-r) must begin with a /
> >> sed: can't read
> >> /@LOCALSTATEDIR@/cache/lxc/fedora/x86_64/20/partial/etc/yum.repos.d/*:
> >> No such file or directory
> >> CRITICAL:yum.cli:--installroot must be an absolute path:
> >> @LOCALSTATEDIR@/cache/lxc/fedora/x86_64/20/partial
> >> Failed to download the rootfs, aborting.
> >> Failed to download 'fedora base'
> >> failed to install fedora
> >> lxc_container: container creation template for masterfe failed
> >> lxc_container: Error creating container masterfe
> >>
> >>
> >> On Tue, May 13, 2014 at 8:15 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> >> > On Tue, 2014-05-13 at 20:03 -0400, CDR wrote:
> >> >> I decided to generate a new container and reinstall all my apps.
> >> >> A lot of work, but you successfully demolished all my work so far, for
> >> >> which I am thankful.
> >> >
> >> > Well, we don't mean to demolish others efforts but we have put a lot of
> >> > work into these templates so others don't need to learn the the lessons
> >> > we've learned and can avoid the sins we've committed. Sorry if this has
> >> > been more difficult than it needed to be.
> >> >
> >> > My deepest regards,
> >> > Mike
> >> >
> >> >> Philip
> >> >
> >> >> On Tue, May 13, 2014 at 7:57 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> >> >> > On Tue, 2014-05-13 at 18:13 -0400, CDR wrote:
> >> >> >> Dear Mike
> >> >> >> You are right, I only see one line.
> >> >> >
> >> >> >> ls rootfs/etc/systemd/system/getty.target.wants -l
> >> >> >> total 0
> >> >> >> lrwxrwxrwx 1 root root 38 Apr 30 11:16 getty at tty1.service ->
> >> >> >> /usr/lib/systemd/system/getty at .service
> >> >> >
> >> >> > Ok...
> >> >> >
> >> >> >> You wouldn't have the script to fix this around?
> >> >> >
> >> >> > I think Fajar pointed you to it. It's in the template.
> >> >> >
> >> >> >> I created my Fedora 20 containers from within a Fedora box, like this:
> >> >> >
> >> >> >> #!/bin/sh
> >> >> >> source /etc/profile
> >> >> >> container=nat-1
> >> >> >> mkdir -p /var/lib/lxc/${container}/rootfs
> >> >> >> yum -y --releasever=20 --nogpg --installroot=/var/lib/lxc/$container/rootfs \
> >> >> >> install systemd autofs passwd yum fedora-release vim-minimal
> >> >> >> openssh-server openssh-clients gcc autogen automake\
> >> >> >> subversion procps-ng initscripts net-tools ethtool nano dhcp dhclient
> >> >> >> lsof bind-utils psmisc bash-completion policycoreutils\
> >> >> >> libvirt libcap-devel lxc deltarpm bridge-utils strace git rpm-build
> >> >> >> docbook2X graphviz man netstat-nat
> >> >> >> cp /usr/src//ifup-local /var/lib/lxc/$container/rootfs/sbin/
> >> >> >> cp /etc/sysconfig/network /var/lib/lxc/$container/rootfs/etc/sysconfig
> >> >> >> cp /usr/src/ifcfg-eth0
> >> >> >> /var/lib/lxc/$container/rootfs/etc/sysconfig/network-scripts/ifcfg-eth0
> >> >> >> cp /usr/src/ifcfg-eth1
> >> >> >> /var/lib/lxc/$container/rootfs/etc/sysconfig/network-scripts/ifcfg-eth1
> >> >> >> echo "pts/0" >> /var/lib/lxc/$container/rootfs/etc/securetty
> >> >> >> cp /usr/src/config /var/lib/lxc/$container/
> >> >> >> cp /usr/src/lxc*.rpm /var/lib/lxc/$container/rootfs/usr/src
> >> >> >> touch /var/lib/lxc/$container/rootfs/etc/resolv.conf
> >> >> >> chroot /var/lib/lxc/$container/rootfs /bin/passwd root
> >> >> >
> >> >> > Oh, that explains a LOT and tells me you will have much bigger problems.
> >> >> > These templates are designed to not only copy the distros into root file
> >> >> > systems but to fine tune some of the peculiarities of running in a
> >> >> > container, which can be (definitely IS) dependent on the startup init
> >> >> > process. You should read through some of these template scripts and
> >> >> > understand, very thoroughly, how and why (I DO try and comment my
> >> >> > templates about WHY I'm doing something that's non-intuitive) they are
> >> >> > doing what they're doing before you attempt to create a script like
> >> >> > this.
> >> >> >
> >> >> >> Yours
> >> >> >
> >> >> >> Federico
> >> >> >
> >> >> > Regards,
> >> >> > Mike
> >> >> >
> >> >> >> On Tue, May 13, 2014 at 1:42 PM, CDR <venefax at gmail.com> wrote:
> >> >> >> > Let me digest all this.
> >> >> >> > You must be right, because Fedora 20 containers are the only ones that
> >> >> >> > use systemd, in my box
> >> >> >> >
> >> >> >> > many thanks
> >> >> >> >
> >> >> >> > Federico
> >> >> >> >
> >> >> >> > On Tue, May 13, 2014 at 1:24 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> >> >> >> >> On Tue, 2014-05-13 at 13:00 -0400, CDR wrote:
> >> >> >> >>> I am forced to use libvirt-lxc only because nobody can help me solve
> >> >> >> >>> this issue. If somebody can figure this out, then I will only use lXC.
> >> >> >> >>> In a Fedora20 container, if I start it with the "-d" flag, then it
> >> >> >> >>> never let's me enter the container via the console, it times out on me
> >> >> >> >>> when I use
> >> >> >> >>> lxc-console -n mycontainer.
> >> >> >> >>
> >> >> >> >> I guess the problem is, here, that this doesn't make sense (to me). I
> >> >> >> >> work and develop under Fedora (17,18,19,20,rawhide). I have containers
> >> >> >> >> of virtually every guest distro, even Gentoo and SUSE and a few others
> >> >> >> >> (NST, Kali) which are not supported. I've got Fedora20 containers of
> >> >> >> >> both x86_64 and i686 archs. I'm not seeing this problem. I've started
> >> >> >> >> them without the -d as well as with the -d and currently start them
> >> >> >> >> using lxc-autostart and I'm just not seeing this problem.
> >> >> >> >>
> >> >> >> >> The problem is in trying to decide why your setup is (or your containers
> >> >> >> >> are) different or what you are doing different so we might address it
> >> >> >> >> (either in code or documentation).
> >> >> >> >>
> >> >> >> >> These are Fedora20 containers you are trying to start? Under what rev
> >> >> >> >> where they created (not what you're running now)? I know I had to add
> >> >> >> >> some code to the Fedora template to create the console devices...
> >> >> >> >>
> >> >> >> >> Look here:
> >> >> >> >>
> >> >> >> >> /var/lib/lxc/{container}/rootfs/etc/systemd/system/getty.target.wants
> >> >> >> >>
> >> >> >> >> Now look for a series of symlinks like this:
> >> >> >> >>
> >> >> >> >> [root at hydra getty.target.wants]# pwd
> >> >> >> >> /var/lib/lxc/Fedora20/rootfs/etc/systemd/system/getty.target.wants
> >> >> >> >> [root at hydra getty.target.wants]# ls -l
> >> >> >> >> total 0
> >> >> >> >> lrwxrwxrwx. 1 root root 17 Mar 7 19:02 getty at tty1.service -> ../getty at .service
> >> >> >> >> lrwxrwxrwx. 1 root root 17 Mar 7 19:02 getty at tty2.service -> ../getty at .service
> >> >> >> >> lrwxrwxrwx. 1 root root 17 Mar 7 19:02 getty at tty3.service -> ../getty at .service
> >> >> >> >> lrwxrwxrwx. 1 root root 17 Mar 7 19:02 getty at tty4.service -> ../getty at .service
> >> >> >> >>
> >> >> >> >> If you don't have them, that's your problem. The lxc-fedora template
> >> >> >> >> now creates these automatically. This is systemd stuff. The
> >> >> >> >> pre-systemd Fedora template did not do this. If you want more than
> >> >> >> >> four, you're going to have to create them yourself and add the changes
> >> >> >> >> to the config file for the vty's.
> >> >> >> >>
> >> >> >> >> If you don't have them, you probably DON'T have the properly munged
> >> >> >> >> getty at .service file in the parent directory, so you can't just copy the
> >> >> >> >> systemd default or blindly create them
> >> >> >> >>
> >> >> >> >> This change has to be made!
> >> >> >> >>
> >> >> >> >> [root at hydra rootfs]# diff lib/systemd/system/getty at .service etc/systemd/system/getty at .service
> >> >> >> >> 24c24
> >> >> >> >> < ConditionPathExists=/dev/tty0
> >> >> >> >> ---
> >> >> >> >>> # ConditionPathExists=/dev/tty0
> >> >> >> >>
> >> >> >> >> If you don't have that... That's your problem.
> >> >> >> >>
> >> >> >> >>> This happens only in Fedora 20 containers. But that does not happen
> >> >> >> >>> under libvirt. I have 3 different OS' containers, and the other two
> >> >> >> >>> work fine under pure LXC.
> >> >> >> >>
> >> >> >> >> It's not just under Fedora 20 containers but will impact any containers
> >> >> >> >> running systemd in a container, which has proven to be a black art for
> >> >> >> >> me.
> >> >> >> >>
> >> >> >> >> Regards,
> >> >> >> >> Mike
> >> >> >> >>
> >> >> >> >>> I think that the bug is in LXC. If I ask Libvirt, they will respond
> >> >> >> >>> "it does work, doesn't it?
> >> >> >> >>> Any idea?
> >> >> >> >>> Philip
> >> >> >> >>>
> >> >> >> >>> On Tue, May 13, 2014 at 12:35 PM, Stéphane Graber <stgraber at ubuntu.com> wrote:
> >> >> >> >>> > On Tue, May 13, 2014 at 12:29:07PM -0400, Michael H. Warfield wrote:
> >> >> >> >>> >> On Tue, 2014-05-13 at 12:09 -0400, CDR wrote:
> >> >> >> >>> >> > Dear Friends
> >> >> >> >>> >> > I have a Fedora 20 LXC (libirt) container in production and I cannot reboot it.
> >> >> >> >>> >> > So I used "virsh edit mycontainer" and added several
> >> >> >> >>> >> >
> >> >> >> >>> >> > <interface type='direct'>
> >> >> >> >>> >> > <mac address='00:5A:0C:18:C9:E9'/>
> >> >> >> >>> >> > <source dev='eth1' mode='bridge'/>
> >> >> >> >>> >> > </interface>
> >> >> >> >>> >>
> >> >> >> >>> >> Ok... But that's libvirt LXC, not LXC-Tools LXC.
> >> >> >> >>> >>
> >> >> >> >>> >> > The problem is that after it gets saved, the new interfaces never show
> >> >> >> >>> >> > up in ip link, and I have no idea how to make Fedora under LXC to
> >> >> >> >>> >> > check for the new hardware.
> >> >> >> >>> >>
> >> >> >> >>> >> > Is this a limitation of LXC in general?
> >> >> >> >>> >>
> >> >> >> >>> >> The term "LXC" in this case is ambiguous. Are you talking about libvirt
> >> >> >> >>> >> lxc or this project. They are not the same.
> >> >> >> >>> >>
> >> >> >> >>> >> > I bet there is a workaround.
> >> >> >> >>> >>
> >> >> >> >>> >> Only if you're skilled at creating hotplug and udev rules. It has to be
> >> >> >> >>> >> done under the host and the host has to transfer that device into the
> >> >> >> >>> >> container. It's not something that's really controllable from the
> >> >> >> >>> >> container per se. I can run devices and such in and out of LXC
> >> >> >> >>> >> containers (NOT libvirt) with some scripting and some rules in the host,
> >> >> >> >>> >> but I doubt that would help you (I take advantage of some of the
> >> >> >> >>> >> devtmpfs stuff I wrote for this project). Libvirt may have a way to
> >> >> >> >>> >> work around that but you'll have to consult with them.
> >> >> >> >>> >>
> >> >> >> >>> >> I think Stéphane also had some utility for moving devices (interfaces,
> >> >> >> >>> >> I'm not so sure) but, again, that's for this project, not libvirt.
> >> >> >> >>> >
> >> >> >> >>> > Correct, lxc-device will let you do that with our LXC.
> >> >> >> >>> >
> >> >> >> >>> >>
> >> >> >> >>> >> > Yours
> >> >> >> >>> >> > Philip
> >> >> >> >>> >>
> >> >> >> >>> >>
> >> >> >> >>> >> --
> >> >> >> >>> >> Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> >> >> >> >>> >> /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> >> >> >> >>> >> NIC whois: MHW9 | An optimist believes we live in the best of all
> >> >> >> >>> >> PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >> >> >> >>> >>
> >> >> >> >>> >
> >> >> >> >>> >
> >> >> >> >>> >
> >> >> >> >>> >> _______________________________________________
> >> >> >> >>> >> lxc-users mailing list
> >> >> >> >>> >> lxc-users at lists.linuxcontainers.org
> >> >> >> >>> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >> >> >>> >
> >> >> >> >>> >
> >> >> >> >>> > --
> >> >> >> >>> > Stéphane Graber
> >> >> >> >>> > Ubuntu developer
> >> >> >> >>> > http://www.ubuntu.com
> >> >> >> >>> >
> >> >> >> >>> > _______________________________________________
> >> >> >> >>> > lxc-users mailing list
> >> >> >> >>> > lxc-users at lists.linuxcontainers.org
> >> >> >> >>> > http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >> >> >>> _______________________________________________
> >> >> >> >>> lxc-users mailing list
> >> >> >> >>> lxc-users at lists.linuxcontainers.org
> >> >> >> >>> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> >> >> >> >> /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> >> >> >> >> NIC whois: MHW9 | An optimist believes we live in the best of all
> >> >> >> >> PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> _______________________________________________
> >> >> >> >> lxc-users mailing list
> >> >> >> >> lxc-users at lists.linuxcontainers.org
> >> >> >> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >> >> _______________________________________________
> >> >> >> lxc-users mailing list
> >> >> >> lxc-users at lists.linuxcontainers.org
> >> >> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >> >
> >> >> > --
> >> >> > Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> >> >> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> >> >> > NIC whois: MHW9 | An optimist believes we live in the best of all
> >> >> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >> >> >
> >> >> >
> >> >> > _______________________________________________
> >> >> > lxc-users mailing list
> >> >> > lxc-users at lists.linuxcontainers.org
> >> >> > http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >> _______________________________________________
> >> >> lxc-users mailing list
> >> >> lxc-users at lists.linuxcontainers.org
> >> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >
> >> > --
> >> > Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> >> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> >> > NIC whois: MHW9 | An optimist believes we live in the best of all
> >> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >> >
> >> >
> >> > _______________________________________________
> >> > lxc-users mailing list
> >> > lxc-users at lists.linuxcontainers.org
> >> > http://lists.linuxcontainers.org/listinfo/lxc-users
> >> _______________________________________________
> >> lxc-users mailing list
> >> lxc-users at lists.linuxcontainers.org
> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >
> > --
> > Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> > NIC whois: MHW9 | An optimist believes we live in the best of all
> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >
> >
> > _______________________________________________
> > lxc-users mailing list
> > lxc-users at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-users
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
--
Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20140513/fe35eb76/attachment.sig>
More information about the lxc-users
mailing list