[lxc-users] Using LXC and VMs together runs into a cgroup issue

[Daniel P. Berrange]

On Sat, Jan 17, 2015 at 03:58:18PM +0000, Anjali Kulkarni wrote:
> Hi,
> 
> I have been using LXC container and VM started on a host machine simultaneously for a while. However, recently on using another machine, I see that either one can be started, but not both.
> If the /cgroups is mounted, then the LXC starts, but the VM(started via libvirt) does not, giving the error:
> 
> virsh start VM1
> error: Failed to start domain VM1
> error: Controller 'memory' is not wanted, but 'cpu' is co-mounted: Invalid argument

Have you edited the /etc/qemu/qemu.conf to customize the set
of cgroup controllers libvirt will use ?  This error message
is saying that libvirt has been told not to use the memory
controller, but this is impossible since you have mounted
the memory controller at the same place as the cpu controller

> If I umount /cgroups, and restart libvirt, then the VM1 VM is created via libvirt, but LXC start fails because it fails to locate cgroups.:
> 
> lxc-start -n lxc1
> lxc-start: Could not find writable mount point for cgroup hierarchy 2 while trying to create cgroup.
> lxc-start: failed creating cgroups
> 
> I am not sure if the bug is in libvirt or anywhere else, can someone pls recommend how to fix or workaround this issue?

Generally you want to avoid mounting the cgroup controllers
all on a single mount point. Instead give each controller a
separate mount. The kernel devs recommended is to use a mount
location of /sys/fs/cgroup/<controller> these days. On a
systemd based host you'd see a setup that looks like this:

$ cat /proc/mounts | grep cgroup
tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0

where only cpu+cpuacct and net_ctl+net_prio are co-mounted at the
same place


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|