[Lxc-users] what's the difference in lxc-attach

[Ramez Hanna]

On Sun, Jul 17, 2011 at 2:25 AM, Michael H. Warfield <mhw at wittsend.com>wrote:

> On Sat, 2011-07-16 at 23:59 +0300, Ramez Hanna wrote:
> > On Sat, Jul 16, 2011 at 8:27 PM, Michael H. Warfield <mhw at wittsend.com
> >wrote:
> >
> > > On Fri, 2011-07-15 at 20:17 +0300, Ramez Hanna wrote:
> > >
> > > << Big Snip >>
> > >
> > > > > > thanks a lot for the detailed answer
> > > > > > by the way have you been succesfull in starting a f15 container
> on
> > > your
> > > > > f15?
> > >
> > > I now have an F15 container working.
> > >
> > > > > > I have been debuggin for 2 hours now
> > > > > > when i start f15 container it screws my host by interfering with
> my
> > > > > hosts's
> > > > > > systemd which somehow doesn't make sense
> > > > > > and when i use systemd-nspawn i get a bunch of errors and the
> system
> > > > > doesn't
> > > > > > finish starting
> > > > > > here is a paste of systemd log from systemd-nspawn session
> > > > > > http://pastie.org/2218625
> > > > >
> > > > > I haven't tried it yet.  Will see what I can do.
> > > > >
> > > > > Couple of quick questions.
> > > > >
> > > > > 1) You say it screws your host if you don't uses nospawn.  What
> > > happens?
> > >
> > > > host console is not useable, random issues around missing characters
> when
> > > i
> > > > type
> > > > unable to login on other terminals because i cannot type
> > > > and i see so many systemd logs on the console
> > >
> > > I have a very strong suspicion that systemd is not going to be
> > > compatible with running in a container because it wants to set up and
> > > managed cgroups in the container which it can not do.
> > >
> > > When I try to start it with systemd, the first process doesn't even
> seem
> > > to come up (number of tasks is 0) and then the host can not remove the
> > > container even after I've done an lxc-stop on it.  But that's when I'm
> > > logged in and running lxc-start from an ssh terminal Window.  If I
> start
> > > it from a real ttyX console then I get all sorts of startup messages
> > > from the container and the consoles are hosed up like the console in
> the
> > > container has gotten crosswise with the console in the host.  Things
> try
> > > to initialize but all sorts of things time out and eventually I have to
> > > reset the host with an Magic SysRq sequence.
> > >
> > > Gave up on systemd.
> > >
> > > > > 2) Have you disabled the sys_admin cap by dropping it in that
> > > container?
> > > > > I find that causes me all sorts of grief.
> > > > >
> > > > i will try that
> > >
> > > Don't.  It wouldn't do any good and causes lots of other problems (for
> > > me at least).
> > >
> > > > > 3) Was this a fresh template build or did you upgrade an F14
> machine to
> > > > > F15 (I was going to use "yum --releasever=15 distro-sync" in one of
> my
> > > > > running F14 containers).
> > >
> > > > yes fresh install
> > >
> > > Here's what I've done and now gotten an F15 container to work.
> > >
> > > I started out with an F14 container and upgraded it to F15 using the
> > > "yum --releasever=15 distro-sync" method.  I was able to reproduce your
> > > problems above and thought there may be some conflicts over cgroups so
> I
> > > decided to disable systemd.
> > >
> > > If it's not present (it wasn't for me) install upstart into the
> > > container from the host using "yum --installroot={your VM root}
> > > upstart".
> > >
> > > Next cd to {your VM root}/sbin and rm init (which is symlinked
> > > to ../bin/systemd) and symlink it to upstart (which is in sbin).
> > >
> > > This got me almost there.  The machine was starting but I was having
> > > your funky console problem and I realized (largely because I'm working
> > > on other related problems) that it was the ptmx device causing this.
>  It
> > > was mapping incorrectly.
> > >
> > > So, cd to {your VM root}/dev and rm ptmx if it's a hard device and not
> a
> > > symlink.  Then symlink pts/ptmx to ptmx.  If you started with some sort
> > > of template, this may already be done and you may not run into this
> > > problem at all.
> > >
> > > Now you should be able to fire your F15 container up.
> > >
> > > Also find the lines in /etc/init.d/halt that remount file systems ro or
> > > you'll screw your /dev/pts fs in the host when you shut that container
> > > down or reboot it (and, no, newinstance is not helping with that
> > > problem).
> > >
> > > Regards,
> > > Mike
> > > --
> > > Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
> > >   /\/\|=mhw=|\/\/          | (678) 463-0932 |
> > > http://www.wittsend.com/mhw/
> > >   NIC whois: MHW9          | An optimist believes we live in the best
> of
> > > all
> > >  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of
> it!
> > >
>
> > it is very clear to me that systemd is interfering with the host's
> systemd
> > your solution of running f15 is not much different than running a f14
> > container (as systemd is the major diff)
> > systemd-nspawn can start systemd inside a "light weight" container
> > i think the problem is related to the fact that when lxc starts teh
> cgroup
> > is on the root of the tree
> > while it should have been under the user's tree
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> I'm not so sure I understand what you mean by that last line.  What
> "user's tree" are you referring to?
>
in f15 systemd whenever a user starts a process it looks like this
├ user
│ ├ root
│ │ └ 86
│ │   ├ 24814 -bash
│ │   ├ 24848 top
│ │   └ 31324 login -- root
│ └ rhanna
│   ├ 56
│   │ ├  1002 pam: gdm-password
│   │ ├  1047 /usr/bin/enlightenment
│   │ ├  1058 dbus-launch --sh-syntax --exit-with-session
│   │ ├  1059 /bin/dbus-daemon --fork --print-pid 5 --print-address 7
--sess...

so i would expect lxc to create it's cgroup under the user (root in this
case) instead
while it currebtly shows it like this
boss is the name of the container
├ 24811 [kworker/1:0]
├ boss
│ ├ 8914 init [3]
│ ├ 9135 /usr/sbin/cron
│ ├ 9146 /usr/sbin/sshd

now I am not trying to use systemd-nspawn to replace lxc or anything, I am
just using it to debug if i had problems in my container rootfs
and well if nspawn doesn't screw up my host then it is doing something
better



> > maybe serge can say somethiing about this
>
> Maybe, maybe not.
>
> The cgroup mounts are where systemd is putting them, not where lxc is
> putting them.  As it is, lxc is not "starting" the cgroup anywhere, it's
> just using them where they are found.  And systemd-nspawn has nothing to
> do with lxc.
>
> Seems to me that where you're trying an F15 guest on an F15 host with
> cgroups mounted in the host by systemd where systemd wants them and then
>
well the container's systemd should not "see" the host's cgroup tree but
rather a cgroup relative to it's rootfs
but again i don't think this is the main problem
i think that the container's systemd is triggering and "runlevel" change at
the host as well, which means that something is not isolated correctly,
AFAIK systemd uses /run/systemd/

using systemd-nspawn to fire up the container, you've completely cut lxc
> out of the loop here?  What does that have to do with lxc at this point?


> I just read up on systemd-nspawn (which I had never tried before) and it
> sounds like a really primitive lxc competitor comparable to lxc-start
> only without the configuration stuff and helper utilities.
>
> Tried it out myself firing up the same Faces container only under
> systemd-nspawn instead of lxc-start and execing /bin/systemd instead
> of /sbin/init (upstart) and yeah it face-planted with the same errors
> you encountered.  It also wouldn't start the same container
> running /sbin/init but I suspect that was more because of missing
> mounted file systems like sys and proc that lxc-start is so nicely
> handling for us.
>
> I would say that problems with systemd-nspawn are systemd-nspawn's and
> not ours here.  After looking at it and trying it out, I don't think
> I'll try it again.  Even the man pages on it mentions "systemd-nspawn -
> Spawn a namespace container for debugging, testing and building".
> Doesn't sound like a production tool to me.
>
> Regards,
> Mike
> --
> Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
>   /\/\|=mhw=|\/\/          | (678) 463-0932 |
> http://www.wittsend.com/mhw/
>   NIC whois: MHW9          | An optimist believes we live in the best of
> all
>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20110717/e71cb905/attachment.html>