[Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053

Mon Nov 5 15:04:40 UTC 2012

Quoting Michael H. Warfield (mhw at WittsEnd.com):
> On Thu, 2012-11-01 at 20:15 -0400, Michael H. Warfield wrote:
> > On Thu, 2012-11-01 at 19:17 -0400, Michael H. Warfield wrote:
> > > On Thu, 2012-11-01 at 23:28 +0100, Serge Hallyn wrote:
> > > > Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > > > > On Thu, 2012-11-01 at 22:44 +0100, Serge Hallyn wrote:
> > > > > > Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > > > > > > On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote:
> > > > > > > > On 11/01/2012 09:08 PM, Michael H. Warfield wrote:
> > > > > > > > > I know, I KNOW this is an 11th hour request.  Can we please get Serge's
> > > > > > > > > autodev stuff into this release?  Please?
> > > > > > > 
> > > > > > > > release early, release often?
> > > > > > > 
> > > > > > > > just release current git as 0.8.0 now, and the one with the autofoo as 
> > > > > > > > 0.8.1 soon after that?
> > > > > > > 
> > > > > > > That would be ideal but we've been sitting at 0.8.0rc2 for something
> > > > > > > like 3-1/2 months now.  I know Daniel (the other Daniel, the Daniel) has
> > > > > > > been incredibly busy.  I have no objection to getting this out the door
> > > > > > > as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another
> > > > > > > several months is not good.  Can we get this fast bump?  We'll be
> > > > > > > staring Fedora 18 in the face by then.  The working versions of Fedora
> > > > > > > are no longer in support and we've got more distros adopting systemd.
> > > > > 
> > > > > > I think this will end up slated for 0.9.0 (which we're hoping will be
> > > > > > soon), but in any case I went ahead and created a branch at
> > > > > > git://github.com/hallyn/lxc called upstream.nov1.2012.autodev, with
> > > > > > an autodev patch on top of Daniel's latest push.
> > > > > 
> > > > > > I quickly tried my hand at fixing the error you had with /dev/ttyN.  I
> > > > > > haven't tested that bit.  I will not be able to be online at all from
> > > > > > now until weekend or monday, so if it needs more tweaks please feel
> > > > > > free to 'just fix it'.
> > > > > 
> > > > > Problem.  Works for the systemd containers but not for my older
> > > > > containers.  I get this...
> > > > > 
> > > > > [root at forest Plover]# cat 2012-10-30-18:17:46.log
> > > > >       lxc-start 1351635466.998 ERROR    lxc_conf - Operation not permitted - error 1 creating /usr/lib64/lxc/rootfs/dev/tty6
> > > > > 
> > > > >       lxc-start 1351635466.999 ERROR    lxc_conf - failed to setup the ttys for 'Plover'
> > > > >       lxc-start 1351635466.999 ERROR    lxc_start - failed to setup the container
> > > > >       lxc-start 1351635466.999 ERROR    lxc_sync - invalid sequence number 1. expected 2
> > > > >       lxc-start 1351635466.999 ERROR    lxc_start - failed to spawn 'Plover'
> > > > > 
> > > > > Alcove (the systemd container) was the first one started so it may be an
> > > > > ordinal thing or it may be a systemd thing.  But it's a problem.
> > > > \
> > > > Hm, perhaps the container doesn't have mknod?
> > 
> > > They all should have, but I will investigate.  Those devices would have
> > > existed in the static file system with /dev.  Could it be a problem with
> > > the device already existing in the /dev directory?
> > 
> > Ok...  Now this is just bloody weird.  I do not understand this.
> > 
> > Yes the containers come up.  But...
> > 
> > Here's what shows up in the detached container's log...
> > 
> > [root at forest Audience]# cat 2012-10-30-18:52:41.log
> >       lxc-start 1351637562.011 ERROR    lxc_conf - Operation not permitted - error creating /usr/lib64/lxc/rootfs/dev/tty6
> 
> > Now wait a minute...  What about 1, 2, 3, 4, and 5???  They succeeded
> > but 6 failed?  How does that make any sense.  In the container...
> 
> > crw-rw-rw- 1 root root   5,   0 Apr 13  2006 tty
> > crw--w---- 1 root tty  136,  16 Oct 30  2012 tty1
> > crw--w---- 1 root tty  136,  17 Oct 30  2012 tty2
> > crw--w---- 1 root tty  136,  18 Oct 30  2012 tty3
> > crw--w---- 1 root tty  136,  19 Oct 30  2012 tty4
> > crw--w---- 1 root tty  136,  20 Oct 30  2012 tty5
> > crw--w---- 1 root tty  136,  21 Oct 30  2012 tty6
> 
> > Ok...  That's probably from a couple of days ago.  But no error messages
> > for the others and they are not "freshly made" either...  That was a
> > CentOS 5 container.
> 
> > Trying it with another Fedora container but removed the tty? entries.
> > No errors.  Hmmm...  Wait...  Another problem...  Container "Plover"...
> 
> > [mhw at plover ~]$ who
> > mhw      pts/9        2012-10-30 19:47 (forest.ip6.wittsend.com)
> > [mhw at plover ~]$ sudo -s
> > sudo: sorry, you must have a tty to run sudo
> 
> > What?
> 
> Ok...  The problem with the container "plover" appears to have been an
> error on my part.  I think I cleaned out too many tty devices.
> 
> Carefully only removed tty? devices from the static /dev in that
> container and started it up...
> 
> [root at forest Plover]# cat 2012-10-31-06:41:37.log
>       lxc-start 1351680097.900 ERROR    lxc_conf - Operation not permitted - error creating /usr/lib64/lxc/rootfs/dev/tty6
> 
> Still got the error creating /dev/tty6 but not 1-5.
> 
> But wait.  In the container itself...
> 
> [mhw at canyon mhw]$ ssh plover.ip6.wittsend.com
> Last login: Tue Oct 30 18:40:12 2012 from canyon.ip6.wittsend.com
> [mhw at plover ~]$ sudo -s
> [root at plover mhw]# ls -l /dev/tty?
> crw-------. 1 root root 136, 36 Oct 31 06:41 /dev/tty1
> crw-------. 1 root root 136, 37 Oct 31 06:41 /dev/tty2
> crw-------. 1 root root 136, 38 Oct 31 06:41 /dev/tty3
> crw-------. 1 root root 136, 39 Oct 31 06:41 /dev/tty4
> crw-------. 1 root root 136, 40 Oct 31 06:41 /dev/tty5
> crw-------. 1 root root 136, 41 Oct 31 06:41 /dev/tty6
> 
> So it gave an error but did actual work.

What error is it giving?  Does it list the errno?

> I've got this narrowed down...  If "devtty" is defined you do one thing
> and, if not, you drop into the autodev logic.  Is that correct?  If I

devtty can be used with autodev = 1, and was working fine for me in
ubuntu containers using autodev = 1.  I just need to see if I can
better consolidate those codepaths.

> use devtty, then the INFO dump says "0 ttys created and, in fact, none
> are, so I guess I don't know how to use that parameter.

If you say

	lxc.devtty = joe

then lxc will create a directory /dev/joe in the container, create
/dev/joe/console and /dev/joe/tty{1-N}, bind mount the host ptys
to those, and then symlink those into /dev.  The purpose of this
sad hackery is to allow package upgrades which want to do

	rm /dev/console
	MAKEDEV /dev/console

to succeed.  If lxc has bind-mounted the lxc consoles onto /dev/console
directly, then 'rm /dev/console' fails because you can't delete the
mount target.

(This of course results in bad console until a restart with lxc, but
let's the container upgrade succeed.  Device namespaces, where are you?)

-serge