[lxc-users] Converting from libvirt lxc

[Peter Steele]

This message is a bit long and I apologize for that, although the bulk 
is cut-and-paste output. I'm migrating our container project from 
libvirt-lxc under CentOS 7.1 to LXC and I'm seeing some errors in 
/var/log/messages that I don't see in libvirt-lxc. The LXC containers I 
am creating are based on the same custom CentOS image that I've been 
using with libvirt-lxc. My assumption is that this image should be able 
to be used without any significant changes as long as I have the 
appropriate config file defined for this image when an LXC container is 
installed.

The lxc-create command I'm using looks generally like this:

# lxc-create -f /hf/cs/vm-03/config -t /bin/true -n vm-03 
--dir=/hf/cs/vm-03/rootfs

where the config file has the following options defined:

     lxc.tty = 4
     lxc.pts = 1024
     lxc.kmsg = 0
     lxc.utsname = vm-03
     lxc.network.type = veth
     lxc.network.flags = up
     lxc.network.link = br0
     lxc.network.veth.pair = vm-03
     lxc.network.hwaddr = fe:d6:e8:f2:aa:e6
     lxc.rootfs = /hf/cs/vm-03/rootfs

When a container boots, I'm seeing the set of errors below:

     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdb, 
10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sdb1, 10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sda, 
10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sdb2, 10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sdb4, 10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sdb3, 10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sda4, 10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sda3, 10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sda2, 10) failed: No such file or directory
     Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sda1, 10) failed: No such file or directory
     Nov 30 09:28:49 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdc, 
10) failed: No such file or directory
     Nov 30 09:28:49 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sdc2, 10) failed: No such file or directory
     Nov 30 09:28:49 vm-03 systemd-udevd: inotify_add_watch(7, 
/dev/sdc1, 10) failed: No such file or directory
     ...
     Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on 
/dev/snd/hwC0D0: No such file or directory
     Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on 
/dev/snd/controlC0: No such file or directory
     Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on 
/dev/snd/pcmC0D0c: No such file or directory
     Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on 
/dev/snd/pcmC0D0p: No such file or directory
     Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on 
/dev/dri/card0: No such file or directory

The host's drives have not been made available in the containers, and 
that's intentional. These errors are all being created by the udev 
service of course, and that's the ultimate cause. When I create a 
container under libvirt-lxc though, the udev service is not enabled and 
I therefore do not see these errors. Containers created with LXC using 
the same CentOS image have the udev suite of services enabled, and even 
if I explicitly disable them using

# systemctl disable systemd-udevd-kernel.socket
# systemctl disable systemd-udevd-control.socket
# systemctl disable systemd-udevd.service
# systemctl disable systemd-udev-trigger.service

when I restart the container the services are enabled and I still see 
these errors. My guess is I'm missing something in the config file for 
my LXC containers but I'm not sure what's needed. This appears to be 
further indicated by the set of sys services that are running in my 
libvirt-lxc containers:

# systemctl|grep sys-
sys-fs-fuse-connections.mount      loaded active mounted FUSE Control 
File System
sys-kernel-config.mount            loaded active mounted Configuration 
File System
sys-kernel-debug.mount             loaded active mounted Debug File System

compared to what I see in my equivalent LXC container:

# systemctl|grep sys-
sys-devices-pci0000:00-0000:00:04.0-sound-card0.device loaded active 
plugged   QEMU Virtual Machine
sys-devices-pci0000:00-0000:00:05.7-usb1-1\x2d1-1\x2d1:1.0-host8-target8:0:0-8:0:0:0-block-sdc-sdc1.device 
loaded active plugged   QEMU_HARDDISK
sys-devices-pci0000:00-0000:00:05.7-usb1-1\x2d1-1\x2d1:1.0-host8-target8:0:0-8:0:0:0-block-sdc-sdc2.device 
loaded active plugged   QEMU_HARDDISK
sys-devices-pci0000:00-0000:00:05.7-usb1-1\x2d1-1\x2d1:1.0-host8-target8:0:0-8:0:0:0-block-sdc.device 
loaded active plugged   QEMU_HARDDISK
sys-devices-pci0000:00-0000:00:06.0-ata4-host3-target3:0:0-3:0:0:0-block-sda-sda1.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata4/host3/target3:0:0/3:0:0:0/block/sda/sda1
sys-devices-pci0000:00-0000:00:06.0-ata4-host3-target3:0:0-3:0:0:0-block-sda-sda2.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata4/host3/target3:0:0/3:0:0:0/block/sda/sda2
sys-devices-pci0000:00-0000:00:06.0-ata4-host3-target3:0:0-3:0:0:0-block-sda-sda3.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata4/host3/target3:0:0/3:0:0:0/block/sda/sda3
sys-devices-pci0000:00-0000:00:06.0-ata4-host3-target3:0:0-3:0:0:0-block-sda-sda4.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata4/host3/target3:0:0/3:0:0:0/block/sda/sda4
sys-devices-pci0000:00-0000:00:06.0-ata4-host3-target3:0:0-3:0:0:0-block-sda.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata4/host3/target3:0:0/3:0:0:0/block/sda
sys-devices-pci0000:00-0000:00:06.0-ata5-host4-target4:0:0-4:0:0:0-block-sdb-sdb1.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata5/host4/target4:0:0/4:0:0:0/block/sdb/sdb1
sys-devices-pci0000:00-0000:00:06.0-ata5-host4-target4:0:0-4:0:0:0-block-sdb-sdb2.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata5/host4/target4:0:0/4:0:0:0/block/sdb/sdb2
sys-devices-pci0000:00-0000:00:06.0-ata5-host4-target4:0:0-4:0:0:0-block-sdb-sdb3.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata5/host4/target4:0:0/4:0:0:0/block/sdb/sdb3
sys-devices-pci0000:00-0000:00:06.0-ata5-host4-target4:0:0-4:0:0:0-block-sdb-sdb4.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata5/host4/target4:0:0/4:0:0:0/block/sdb/sdb4
sys-devices-pci0000:00-0000:00:06.0-ata5-host4-target4:0:0-4:0:0:0-block-sdb.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:06.0/ata5/host4/target4:0:0/4:0:0:0/block/sdb
sys-devices-pci0000:00-0000:00:07.0-virtio0-virtio\x2dports-vport0p1.device 
loaded active plugged 
/sys/devices/pci0000:00/0000:00:07.0/virtio0/virtio-ports/vport0p1
sys-devices-platform-serial8250-tty-ttyS1.device loaded active plugged   
/sys/devices/platform/serial8250/tty/ttyS1
sys-devices-platform-serial8250-tty-ttyS2.device loaded active plugged   
/sys/devices/platform/serial8250/tty/ttyS2
sys-devices-platform-serial8250-tty-ttyS3.device loaded active plugged   
/sys/devices/platform/serial8250/tty/ttyS3
sys-devices-pnp0-00:04-tty-ttyS0.device loaded active plugged   
/sys/devices/pnp0/00:04/tty/ttyS0
sys-devices-virtual-block-md0.device loaded active plugged   
/sys/devices/virtual/block/md0
sys-devices-virtual-block-md1.device loaded active plugged   
/sys/devices/virtual/block/md1
sys-devices-virtual-net-eth0.device loaded active plugged   
/sys/devices/virtual/net/eth0
sys-module-configfs.device loaded active plugged   /sys/module/configfs
sys-module-fuse.device loaded active plugged   /sys/module/fuse
sys-subsystem-net-devices-eth0.device loaded active plugged   
/sys/subsystem/net/devices/eth0
proc-sys-fs-binfmt_misc.mount loaded active mounted   Arbitrary 
Executable File Formats File System
sys-fs-fuse-connections.mount loaded active mounted   FUSE Control File 
System
sys-kernel-config.mount loaded active mounted   Configuration File System
sys-kernel-debug.mount loaded active mounted   Debug File System

Is the udev service needed in LXC and if so, how do I keep it from 
complaining?