[Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
Michael H. Warfield
mhw at WittsEnd.com
Tue Oct 30 20:59:14 UTC 2012
On Mon, 2012-10-29 at 10:18 +0100, Serge Hallyn wrote:
> Quoting Michael H. Warfield (mhw at WittsEnd.com):
> ...
> > Yeah, I don't think I "need to play a game like this" anymore. I'd have
> > to go back through some old old E-Mails to see why I did that before. I
> > seem to recall we were playing with all sorts of bind mount options for
> > some PRIVATE thing or another. It may not be necessary at all any
> > longer. IAC, it's minor to switch it back. I seem to recall switching
> > back and forth using bind mounts several times back when that got done
> > that way.
> >
> > I may play with the pre-mount hook just for giggles and see how that
> > might work as well. Any idea why I was experiencing the problem with
> > the mount hook when trying to populate /dev? I know it wouldn't have
> The only idea I have is that perhaps your root is MS_SHARED by default?
> Can you post the script you were using and the container config?
Another point on the curve... The documentation says that "pre-mount"
takes place before the mount occurs and "mount takes place after the
mount occurs. Only problem is that "pre-mount" is not being recognized.
lxc-start 1351627853.032 ERROR lxc_confile - unknown key lxc.hook.pre-mount
This is the same binaries from git that recognize lxc.hook.mount so I'm
assuming the doco and the code don't match at this point.
Even without my original bind mount, if I have a mount hook that does
something in a newly mounted tmpfs directory, it doesn't show up in that
directory it shows up in the parent directory as if it ran before the
mounts took place. I could put the mount in the hook.mount script and
then do it but it's seriously acting like the pre-mount hook isn't even
there (parameter unknown) and the mount hook is running before the
mounts are complete.
Simple exerts from some test scripts doing, really, nothing but testing
sequencing...
Ok... Lets try this. I won't post entire configs but... For machine
Alcove (my testbed)...
--
lxc.tty = 6
lxc.pts = 64
lxc.rootfs = /srv/lxc/private/Alcove
lxc.mount.entry=none /srv/lxc/private/Alcove/dev.tmp tmpfs defaults 0 0
lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0
lxc.autodev = 1
# lxc.hook.pre-mount = /var/lib/lxc/Alcove/pre-mount
lxc.hook.mount = /var/lib/lxc/Alcove/mount
lxc.mount.entry=shmfs /var/lib/lxc/private/Alcove/dev/shm tmpfs mode=0644 0 0
--
Now /var/lib/lxc/Alcove/mount:
--
#!/bin/sh -
touch /srv/lxc/private/Alcove/dev.tmp/mounted
--
In that directory on the host fs I have this:
[root at forest mhw]# touch /srv/lxc/private/Alcove/dev.tmp/no-mounted
[root at forest mhw]# ls /srv/lxc/private/Alcove/dev.tmp/
no-mounted
Now, when I start the container, the tmpfs should get mounted
on /dev.tmp in the container (relative to the container rootfs) and
should have the single file "mounted" in it while the parent file system
back on the host should have the single file "not-mounted" in it.
Let's see... lxc-start -n Alcove...
In the container...
[mhw at alcove ~]$ mount | grep tmpfs
none on /dev type tmpfs (rw,relatime,seclabel,size=100k)
none on /dev.tmp type tmpfs (rw,relatime,seclabel)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)
tmpfs on /media type tmpfs (rw,nosuid,nodev,noexec,relatime,seclabel,mode=755)
Looks like the mount took place. I have tmpfs on /dev.tmp
[mhw at alcove ~]$ ls /dev.tmp/
[mhw at alcove ~]$
Opps... Where did the file end up?
Let's look on the host...
[mhw at forest ~]$ ls /srv/lxc/private/Alcove/dev.tmp/
mounted no-mounted
Arg... Wrong answer. It ended up in the parent file system before
tmpfs was mounted. But, the documentation says hook.mount runs after
the mounts have completed.
There's something wrong here or I am badly mistaken in my
understanding... (Probably the later, I'll admit.)
Regards,
Mike
> > worked because of the /dev/pts mount but I have more heartburn in that
> > it looks like it ran too early and the mount on /dev had not even taken
> > place at that time.
> >
> > > > I believe I can see why... You're doing the autodev populate prior to
> > > > any of the mounts being performed, so that "private" root file system is
> > > > not bound to the directory at that time.
> > > >
> > > > Drop that bind mount for the rootfs and this then worked like a charm:
> > > >
> > > > --
> > > > lxc.rootfs = /srv/lxc/private/Alcove
> > > > lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0
> > > >
> > > > lxc.autodev = 1
> > > > --
> > > >
> > > > I think that rootfs directory bind was an effort to more fully match the
> > > > OpenVZ behavior but also trying to deal with some of the read-only
> > > > problems were where having in the past with shutdowns. If it won't
> > > > work, it won't work and I won't miss it.
> > > >
> > > > I did see some errors setting up that dev...
> > > >
> > > > --
> > > > [root at forest mhw]# lxc-start -n Alcove
> > > > lxc-start: No such file or directory - failed to mount '/dev/pts/59'->'/usr/lib64/lxc/rootfs/dev/tty1'
> > > > lxc-start: No such file or directory - failed to mount '/dev/pts/60'->'/usr/lib64/lxc/rootfs/dev/tty2'
> > > > lxc-start: No such file or directory - failed to mount '/dev/pts/61'->'/usr/lib64/lxc/rootfs/dev/tty3'
> > > > lxc-start: No such file or directory - failed to mount '/dev/pts/62'->'/usr/lib64/lxc/rootfs/dev/tty4'
> > > > lxc-start: No such file or directory - failed to mount '/dev/pts/63'->'/usr/lib64/lxc/rootfs/dev/tty5'
> > > > lxc-start: No such file or directory - failed to mount '/dev/pts/64'->'/usr/lib64/lxc/rootfs/dev/tty6'
> > > > systemd 44 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP; fedora)
> > > >
> > > > Welcome to Fedora 17 (Beefy Miracle)!
> > > >
> > > > --
> > > >
> > > > Not sure what that's all about but, since systemd isn't going to start
> > > > getty's on the tty? interfaces anyways, it probably doesn't make much
> > > > difference.
> >
> > > Oh, I see. Yeah, in the !lxc.ttydir case, when we created our own /dev
> > > we should create the tty files. I need to fix that.
> >
> > Cool. Once again... Looks like we got some real progress here with
> > this one. I've still got more testing to do, undoing some of my changes
> > in the container itself and making sure it all still works.
> >
> > Also looks like I can stop and restart one of these containers now
> > without the hung cgroup directory. I suspected it was some stray
> > devices behind that. This is good.
> >
> > > Of course in your case since systemd isn't going to start getty's on
> > > them, you should not have the lxc.tty = 6 in your container config,
> > > which it looks like you still do?
> >
> > Yeah. I was taking it one step at a time. I wish they WOULD fire up
> > some getty's on those tty's since that basically makes lxc-console kinda
>
> As I recall, you can specify gettys to start on any tty by creating a
> magical symlink, something like
>
> $ETC/getty.target.wants/getty\@tty{1,2,3,4,5,6}.service
>
> > useless and the one on /dev/console is kinda useless in disconnected
> > mode with the console redirected to a file. Maybe we need some what for
> > lxc-console to be able to grab that? I'll have to look deeper at
> > systemd and figure out if it can be parameterized or conditionalized in
> > some way. ITMT, I probably will just turn them off.
> >
> > Many thanks!
> >
> > Regards,
> > Mike
> > --
> > Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> > NIC whois: MHW9 | An optimist believes we live in the best of all
> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
>
>
>
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20121030/617a07e5/attachment.pgp>
More information about the lxc-users
mailing list