[lxc-users] Download and install trusty-64 unprivileged container cannot start anymore, requires system reboot to work - used to work

[Martijn Unknown]

Hello everyone,


A couple of months ago I created some provisioning scripts to install lxc on the host, and then add a separate user to create and run unprivileged containers.


Host os: ubuntu server 14.0.4.3 LTS


As root user:


Installs lxc, adds the user for unpriv. containers and does some settings (nothing special I left it out for now as nothing changed there).


Then it launches the script below with the container user:

sshpass -p "$CNT_PASSWORD" ssh -o StrictHostKeyChecking=no "$CNT_USER at localhost" "$SCRIPTS_DIR_FOR_INSTALL/setup_empty_container.sh" "$NAME" 0 0 0



The script called, as CNT_USER does:


lxc-create -t download -n "$NAME" -- -d ubuntu -r trusty -a amd64

lxc-start -n "$NAME" -d


This worked fine. The container downloaded, installed and then could be started just fine. Now the newly container fails to start. Rebooting the host, and the container can be started again.


Nothing changed to the install scripts - except today I patched the host OS to the latest patches (just ordinary update & upgrade). The install still works, but the container fails to start (part of the debug where it goes wrong):


    lxc-start 20160605141904.134 INFO     lxc_monitor - monitor.c:lxc_monitor_sock_name:178 - using monitor sock name lxc/01a94ea14f07b6c1//home/cntusr/.local/share/lxc
      lxc-start 20160605141904.136 INFO     lxc_start - start.c:lxc_init:488 - 'cnt-base' is initialized
      lxc-start 20160605141904.136 DEBUG    lxc_start - start.c:__lxc_start:1326 - Not dropping cap_sys_boot or watching utmp
      lxc-start 20160605141904.136 INFO     lxc_start - start.c:resolve_clone_flags:1013 - Cloning a new user namespace
      lxc-start 20160605141904.136 INFO     lxc_cgroup - cgroup.c:cgroup_init:68 - cgroup driver cgroupfs initing for cnt-base
      lxc-start 20160605141904.136 ERROR    lxc_cgfs - cgfs.c:lxc_cgroupfs_create:1007 - Permission denied - Could not create cgroup '/lxc' in '/sys/fs/cgroup/freezer'.
      lxc-start 20160605141904.136 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/hugetlb/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_prio/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_cls/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/freezer/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/devices/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/memory/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuacct/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset/
      lxc-start 20160605141904.137 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/systemd/user/1002.user/8.session
      lxc-start 20160605141904.137 ERROR    lxc_start - start.c:lxc_spawn:1100 - failed creating cgroups
      lxc-start 20160605141904.137 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'cnt-base'
      lxc-start 20160605141904.137 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'cnt-base', config section 'lxc'
      lxc-start 20160605141904.640 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
      lxc-start 20160605141909.645 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 20160605141909.645 ERROR    lxc_start_ui - lxc_start.c:main:346 - To get more details, run the container in foreground mode.
      lxc-start 20160605141909.645 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.



So this has probably to do with the user initially created and it's cgroups but I am not sure why this fails now. It does successfully create the container, but cannot start it.


After a reboot of the host it works again, and the container created can be started just fine with the container user.


Does someone have an idea what is wrong as of a sudden?


Thanks,

Martijn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20160605/dfbbf5ca/attachment.html>