[Lxc-users] Confusing behaviour using LXC container with systemd

[John]

On 12/03/13 22:25, Serge Hallyn wrote:
> Quoting John (lxc at jelmail.com):
>> Hello, I've been trying to convert some containers to systemd. I have
>> implemented a systemd container using lxc.autodev with lxc.hook.autodev
>> to create additional devices and I can now start containers that run
>> systemd internally. I am, however, experiencing some problems and am at
>> a loss as to how I should troubleshoot.
>>
>> My autodev hook contains the below:
>>
>> #!/bin/bash
>> # LXC Autodev hook.
>> cd ${LXC_ROOTFS_MOUNT}
>> mknod -m 600 .//dev/tty0 c 4 0
>> mknod -m 666 .//dev/ptmx c 5 2
>>
>> The first problem is that starting a container affects devices on the
>> host. I don't know how widespread the problem is but I noticed it
> Presumably this is just because your guest is running 'udevadm trigger'.
> We disable that in ubuntu using apparmor, but if you allow it then the
> guest triggers the host into resetting hardware.  Sound card goes back
> to default level, keyboard gets reset, ...
Hmmm, I'll look at that but it's going to be a bit of a find. I am not 
doing that explicitly as far as I know and I don't know where to look to 
see if it is happening. I can do some searching with grep ...
> ...
>
>> The second problem might be related... If I start a systemd container
>> from a gui terminal window, the window starts pumping out characters
>> (normally "=" symbols but this changes if I type on the keyboard, though
>> not to the character typed). If I move the focus to another terminal
>> window the character output moves with the focus. This happens if I
>> start the container with  "lxc-start" or with "systemctl". It does not
>> happen if I start the container on a text virtual terminal (Alt+F1). (in
>> case it's relevant, the gui is itself inside a container that is still
>> init-based).
> Sounds like systemd is playing with /dev in the container.  You say
> you're using autodev hooks, but (a) what is the actual filesystem in
> the container's /dev?  If it's devtmpfs then it's shared with your
> host, and your container is actually corrupting your host's /dev and
> (b) otherwise systemd may simply be re-creating things like /dev/console
> and /dev/tty after you've set them up in your autodev hook.
my mounts inside container

[root at boron ~]# mount
none on /dev type tmpfs (rw,relatime,size=100k)
devpts on /dev/console type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
devpts on /dev/tty1 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts 
(rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup 
(rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
tmpfs on /tmp type tmpfs (rw)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
[root at boron ~]#

I presume all of the mounts on /dev/are created by autodev.

>
> Oh!  or systemd is mounting the host's /dev/pts into the container.
Not sure how to tell if they're the same but they both look identical 
(same character device files, same timestamps, etc). I just did a chown 
on one device on the host and the change was visible on the guest so I 
guess they are the same... is this a config mistake that I have made or 
is systemd doing it?

Some additional info... my container config:

# Use autodev to be compatible with systemd
lxc.autodev = 1
lxc.hook.autodev = /etc/lxc/autodev

lxc.utsname = boron
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.name = eth0
lxc.network.mtu = 1500

lxc.cap.drop = sys_module

lxc.cgroup.devices.deny = a
lxc.cgroup.devices.allow = c 1:3 rwm # /dev/null
lxc.cgroup.devices.allow = c 1:5 rwm # /dev/zero
lxc.cgroup.devices.allow = c 1:7 rwm # /dev/full
lxc.cgroup.devices.allow = c 1:8 rwm # /dev/urandom
lxc.cgroup.devices.allow = c 1:9 rwm # /dev/random

lxc.tty = 1                   #   allow this many ttys
lxc.cgroup.devices.allow = c 4:0 rwm # /dev/tty0
lxc.cgroup.devices.allow = c 5:0 rwm # /dev/tty
lxc.cgroup.devices.allow = c 5:1 rwm # /dev/console
lxc.cgroup.devices.allow = c 5:2 rwm   # /dev/ptmx
lxc.cgroup.devices.allow = c 136:* rwm # /dev/pts/*

lxc.rootfs = /srv/lxc/boron.x86_64


> -serge
>