<div dir="ltr"><div class="gmail_default" style="font-size:small">I could not find on Google any mention of Red Hat killing LXC on Libvirt. Care to elaborate?<br><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 2, 2015 at 10:30 AM, Daniel P. Berrange <span dir="ltr"><<a href="mailto:berrange@redhat.com" target="_blank">berrange@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On Mon, Nov 30, 2015 at 02:43:39PM -0800, Peter Steele wrote:<br>
> This message is a bit long and I apologize for that, although the bulk is<br>
> cut-and-paste output. I'm migrating our container project from libvirt-lxc<br>
> under CentOS 7.1 to LXC and I'm seeing some errors in /var/log/messages that<br>
> I don't see in libvirt-lxc. The LXC containers I am creating are based on<br>
> the same custom CentOS image that I've been using with libvirt-lxc. My<br>
> assumption is that this image should be able to be used without any<br>
> significant changes as long as I have the appropriate config file defined<br>
> for this image when an LXC container is installed.<br>
><br>
> The lxc-create command I'm using looks generally like this:<br>
><br>
> # lxc-create -f /hf/cs/vm-03/config -t /bin/true -n vm-03<br>
> --dir=/hf/cs/vm-03/rootfs<br>
><br>
> where the config file has the following options defined:<br>
><br>
> lxc.tty = 4<br>
> lxc.pts = 1024<br>
> lxc.kmsg = 0<br>
> lxc.utsname = vm-03<br>
> lxc.network.type = veth<br>
> lxc.network.flags = up<br>
> lxc.network.link = br0<br>
> lxc.network.veth.pair = vm-03<br>
> lxc.network.hwaddr = fe:d6:e8:f2:aa:e6<br>
> lxc.rootfs = /hf/cs/vm-03/rootfs<br>
><br>
> When a container boots, I'm seeing the set of errors below:<br>
><br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdb, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdb1, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sda, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdb2, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdb4, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdb3, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sda4, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sda3, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sda2, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:48 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sda1, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:49 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdc, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:49 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdc2, 10)<br>
> failed: No such file or directory<br>
> Nov 30 09:28:49 vm-03 systemd-udevd: inotify_add_watch(7, /dev/sdc1, 10)<br>
> failed: No such file or directory<br>
> ...<br>
> Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on<br>
> /dev/snd/hwC0D0: No such file or directory<br>
> Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on<br>
> /dev/snd/controlC0: No such file or directory<br>
> Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on<br>
> /dev/snd/pcmC0D0c: No such file or directory<br>
> Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on<br>
> /dev/snd/pcmC0D0p: No such file or directory<br>
> Nov 30 09:28:56 vm-03 systemd-udevd: Failed to apply ACL on<br>
> /dev/dri/card0: No such file or directory<br>
><br>
> The host's drives have not been made available in the containers, and that's<br>
> intentional. These errors are all being created by the udev service of<br>
> course, and that's the ultimate cause. When I create a container under<br>
> libvirt-lxc though, the udev service is not enabled and I therefore do not<br>
> see these errors. Containers created with LXC using the same CentOS image<br>
> have the udev suite of services enabled, and even if I explicitly disable<br>
> them using<br>
><br>
> # systemctl disable systemd-udevd-kernel.socket<br>
> # systemctl disable systemd-udevd-control.socket<br>
> # systemctl disable systemd-udevd.service<br>
> # systemctl disable systemd-udev-trigger.service<br>
><br>
> when I restart the container the services are enabled and I still see these<br>
> errors. My guess is I'm missing something in the config file for my LXC<br>
> containers but I'm not sure what's needed. This appears to be further<br>
> indicated by the set of sys services that are running in my libvirt-lxc<br>
> containers:<br>
<br>
</div></div>The systemd-udevd.service file has<br>
<br>
ConditionPathIsReadWrite=/sys<br>
<br>
And libvirt LXC sets /sys as read-only, so if you have /sys as writable<br>
that could explain the difference in behaviour.<br>
<br>
The other notable thing libvirt does is drop CAP_SYS_MKNOD. Previously<br>
systemd would look at that capability when starting some things like udev,<br>
but it looks like these days it triggers off /sys read-only status.<br>
<span class=""><br>
<br>
> Is the udev service needed in LXC and if so, how do I keep it from<br>
> complaining?<br>
<br>
</span>No, you really don't want udev enabled or running inside containers at all.<br>
<br>
Regards,<br>
Daniel<br>
<span class="HOEnZb"><font color="#888888">--<br>
|: <a href="http://berrange.com" rel="noreferrer" target="_blank">http://berrange.com</a> -o- <a href="http://www.flickr.com/photos/dberrange/" rel="noreferrer" target="_blank">http://www.flickr.com/photos/dberrange/</a> :|<br>
|: <a href="http://libvirt.org" rel="noreferrer" target="_blank">http://libvirt.org</a> -o- <a href="http://virt-manager.org" rel="noreferrer" target="_blank">http://virt-manager.org</a> :|<br>
|: <a href="http://autobuild.org" rel="noreferrer" target="_blank">http://autobuild.org</a> -o- <a href="http://search.cpan.org/~danberr/" rel="noreferrer" target="_blank">http://search.cpan.org/~danberr/</a> :|<br>
|: <a href="http://entangle-photo.org" rel="noreferrer" target="_blank">http://entangle-photo.org</a> -o- <a href="http://live.gnome.org/gtk-vnc" rel="noreferrer" target="_blank">http://live.gnome.org/gtk-vnc</a> :|<br>
</font></span><div class="HOEnZb"><div class="h5">_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a></div></div></blockquote></div><br></div>