[lxc-users] using cgroups

Thu Jun 30 00:43:48 UTC 2016

 On Thursday, 30 June 2016, 10:36, Serge E. Hallyn <serge at hallyn.com> wrote:

 Quoting Rob Edgerton (redgerhoo at yahoo.com.au):
> hi,I have the same problem (cgroups not working as expected) on a clean Xenial build (lxc PPA NOT installed, LXD not installed)In my case I have some Ubuntu Trusty containers I really need to use on Xenial, but they won't start because I use cgroups.If I change the existing containers to remove the "lxc.cgroup" clauses from config they start, but not otherwise.Similarly, I created a new Xenial container for testing. It works, until I add "lxc.cgroups" clauses at which point it also fails to start. at virt-host:~$ lxc-start -n trusty_unp_ibvpn -F -l debug -o lxc.log
> lxc-start: cgfsng.c: cgfsng_setup_limits: 1662 No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
> lxc-start: start.c: lxc_spawn: 1180 failed to setup the cgroup limits for 'trusty_unp_ibvpn'
> lxc-start: start.c: __lxc_start: 1353 failed to spawn 'trusty_unp_ibvpn'
> lxc-start: lxc_start.c: main: 344 The container failed to start.
> lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority  options.
> 
> Logfile Contents=============
>       lxc-start 20160628155820.562 INFO     lxc_start_ui - lxc_start.c:main:264 - using rcfile /mnt/lxc_images/containers/trusty_unp_ibvpn/config
>       lxc-start 20160628155820.562 WARN     lxc_confile - confile.c:config_pivotdir:1879 - lxc.pivotdir is ignored.  It will soon become an error.
>       lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 100000 range 65536
>       lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 100000 range 65536
>       lxc-start 20160628155820.564 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts
> 
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for reject_force_umount action 0
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts
> 
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .[all].
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .kexec_load errno 1.
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for kexec_load action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for kexec_load action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .open_by_handle_at errno 1.
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for open_by_handle_at action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for open_by_handle_at action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .init_module errno 1.
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for init_module action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for init_module action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .finit_module errno 1.
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for finit_module action 327681
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .delete_module errno 1.
>       lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for delete_module action 327681
>       lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for delete_module action 327681
>       lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:456 - Merging in the compat seccomp ctx into the main one
>       lxc-start 20160628155820.565 DEBUG    lxc_start - start.c:setup_signal_fd:289 - sigchild handler set
>       lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:431 - opening /dev/tty for console peer
>       lxc-start 20160628155820.565 INFO     lxc_caps - caps.c:lxc_caps_up:101 - Last supported cap was 36
>       lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:437 - using '/dev/tty' as console
>       lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_sigwinch_init:145 - 3234 got SIGWINCH fd 9
>       lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_winsz:72 - set winsz dstfd:6 cols:212 rows:73
>       lxc-start 20160628155820.611 INFO     lxc_start - start.c:lxc_init:488 - 'trusty_unp_ibvpn' is initialized
>       lxc-start 20160628155820.611 DEBUG    lxc_start - start.c:__lxc_start:1326 - Not dropping cap_sys_boot or watching utmp
>       lxc-start 20160628155820.611 INFO     lxc_start - start.c:resolve_clone_flags:1013 - Cloning a new user namespace
>       lxc-start 20160628155820.611 INFO     lxc_cgroup - cgroup.c:cgroup_init:68 - cgroup driver cgroupfs-ng initing for trusty_unp_ibvpn
>       lxc-start 20160628155820.614 DEBUG    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1667 - cgroup 'devices.allow' set to 'c 10:200 rwm'
>       lxc-start 20160628155820.614 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1662 - No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn

ENOENT - that's unexpected...

>       lxc-start 20160628155820.615 ERROR    lxc_start - start.c:lxc_spawn:1180 - failed to setup the cgroup limits for 'trusty_unp_ibvpn'
>       lxc-start 20160628155820.615 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty_unp_ibvpn'
>       lxc-start 20160628155820.659 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty_unp_ibvpn', config section 'lxc'
>       lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
>       lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
>    
> Repeating the commands you were discussing with Mike
> 
> cgmanager is already the newest version (0.39-2ubuntu5).
> @virt-host:~$cgm --version
> 0.29

Can you show 'dpkg -l | grep cgmanager' ?

as well as cat /etc/*release

> @virt-host:~$ls /proc/self/cgroup
> /proc/self/cgroup
> 
> @virt-host:~$ls /proc/self/mountinfo
> /proc/self/mountinfo

Hi,
For /proc/self/cgroup and /proc/self/mountinfo, we actually need to see
the contents.  Can you show 'cat /proc/self/cgroup' and
'cat /proc/self/mountinfo'?

-serge
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

hi Serge,thanks for your response
$ dpkg -l | grep cgmanager
ii  cgmanager                                       0.39-2ubuntu5                              amd64        Central cgroup manager daemon
ii  libcgmanager0:amd64                             0.39-2ubuntu5                              amd64        Central cgroup manager daemon (client library)
$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/1
8:pids:/user.slice/user-1000.slice
7:perf_event:/
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:devices:/user.slice
3:memory:/user/redger/1
2:cpuset:/
1:name=systemd:/user.slice/user-1000.slice/session-1.scope

$ cat /proc/self/mountinfo
19 25 0:18 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
20 25 0:4 / /proc rw,nosuid,nodev,noexec,relatime shared:12 - proc proc rw
21 25 0:6 / /dev rw,nosuid,relatime shared:2 - devtmpfs udev rw,size=8026104k,nr_inodes=2006526,mode=755
22 21 0:14 / /dev/pts rw,nosuid,noexec,relatime shared:3 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
23 25 0:19 / /run rw,nosuid,noexec,relatime shared:5 - tmpfs tmpfs rw,size=1615856k,mode=755
25 0 8:41 / / rw,relatime shared:1 - ext4 /dev/sdc9 rw,errors=remount-ro,data=ordered
26 19 0:12 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime shared:8 - securityfs securityfs rw
27 21 0:21 / /dev/shm rw,nosuid,nodev shared:4 - tmpfs tmpfs rw
28 23 0:22 / /run/lock rw,nosuid,nodev,noexec,relatime shared:6 - tmpfs tmpfs rw,size=5120k
29 19 0:23 / /sys/fs/cgroup rw shared:9 - tmpfs tmpfs rw,mode=755
30 29 0:24 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
31 19 0:25 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime shared:11 - pstore pstore rw
32 29 0:26 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,cpuset,clone_children
33 29 0:27 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,memory
34 29 0:28 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,devices
35 29 0:29 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,net_cls,net_prio
36 29 0:30 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:17 - cgroup cgroup rw,cpu,cpuacct
37 29 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:18 - cgroup cgroup rw,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event
38 29 0:32 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:19 - cgroup cgroup rw,pids,release_agent=/run/cgmanager/agents/cgm-release-agent.pids
39 29 0:33 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:20 - cgroup cgroup rw,freezer
40 29 0:34 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:21 - cgroup cgroup rw,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb
41 29 0:35 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:22 - cgroup cgroup rw,blkio
42 20 0:36 / /proc/sys/fs/binfmt_misc rw,relatime shared:23 - autofs systemd-1 rw,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct
43 19 0:7 / /sys/kernel/debug rw,relatime shared:24 - debugfs debugfs rw
44 21 0:37 / /dev/hugepages rw,relatime shared:25 - hugetlbfs hugetlbfs rw
45 23 0:38 / /run/rpc_pipefs rw,relatime shared:26 - rpc_pipefs sunrpc rw
46 21 0:17 / /dev/mqueue rw,relatime shared:27 - mqueue mqueue rw
47 20 0:39 / /proc/fs/nfsd rw,relatime shared:28 - nfsd nfsd rw
48 19 0:40 / /sys/fs/fuse/connections rw,relatime shared:29 - fusectl fusectl rw
49 25 8:34 / /mnt/snd480_boot_01 rw,relatime shared:30 - ext4 /dev/sdc2 rw,data=ordered
50 25 8:35 / /mnt/snd480_root_01 rw,relatime shared:31 - ext4 /dev/sdc3 rw,data=ordered
51 25 8:42 / /home rw,relatime shared:32 - ext4 /dev/sdc10 rw,data=ordered
53 25 8:40 / /boot rw,relatime shared:33 - ext4 /dev/sdc8 rw,data=ordered
52 25 8:36 / /mnt/snd480_home_01 rw,relatime shared:34 - ext4 /dev/sdc4 rw,data=ordered
56 25 8:39 / /mnt/snd480_boot_03_wintemp rw,relatime shared:35 - vfat /dev/sdc7 rw,gid=46,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro
54 25 8:51 / /mnt/wd2t_home_01 rw,relatime shared:36 - ext4 /dev/sdd3 rw,data=ordered
55 25 8:1 / /mnt/video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
57 25 8:50 / /mnt/wd2t_root_01 rw,relatime shared:38 - ext4 /dev/sdd2 rw,data=ordered
58 25 8:17 / /mnt/dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
60 25 8:18 / /mnt/music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
59 25 8:49 / /mnt/wd2t_boot_01 rw,relatime shared:41 - ext4 /dev/sdd1 rw,data=ordered
61 25 252:4 / /mnt/lxc_images rw,noatime shared:42 - ext4 /dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
63 25 0:41 / /var/lib/lxd rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/
64 61 252:4 /containers/utopic_browse_normal_backup_151115/rootfs/home/ubuntu/Downloads /mnt/lxc_images/containers/trusty-mythserver/rootfs/mnt/lxc_container_normal_downloads rw,noatime shared:42 - ext4 /dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
65 61 8:1 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
62 25 252:0 / /mnt/programming_data rw,relatime shared:44 - ext4 /dev/mapper/wd2t--lvm--data-programming_data rw,data=ordered
66 61 8:17 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
67 61 8:1 / /mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
68 61 8:18 / /mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
69 61 8:18 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
70 61 8:17 / /mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
71 25 252:4 /containers/utopic_browse_normal_backup_151115/rootfs/home/ubuntu/Downloads /mnt/lxc_container_normal_downloads rw,noatime shared:42 - ext4 /dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
72 25 0:41 / /mnt/lxd_images rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/
142 23 0:45 / /run/cgmanager/fs rw,relatime shared:113 - tmpfs cgmfs rw,size=100k,mode=755
146 23 0:47 / /run/lxcfs/controllers rw,relatime shared:115 - tmpfs tmpfs rw,size=100k,mode=700
148 146 0:35 / /run/lxcfs/controllers/blkio rw,relatime shared:117 - cgroup blkio rw,blkio
150 146 0:34 / /run/lxcfs/controllers/hugetlb rw,relatime shared:119 - cgroup hugetlb rw,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb
152 146 0:33 / /run/lxcfs/controllers/freezer rw,relatime shared:121 - cgroup freezer rw,freezer
155 146 0:32 / /run/lxcfs/controllers/pids rw,relatime shared:123 - cgroup pids rw,pids,release_agent=/run/cgmanager/agents/cgm-release-agent.pids
157 146 0:31 / /run/lxcfs/controllers/perf_event rw,relatime shared:125 - cgroup perf_event rw,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event
159 146 0:30 / /run/lxcfs/controllers/cpu,cpuacct rw,relatime shared:127 - cgroup cpu,cpuacct rw,cpu,cpuacct
161 146 0:29 / /run/lxcfs/controllers/net_cls,net_prio rw,relatime shared:129 - cgroup net_cls,net_prio rw,net_cls,net_prio
163 146 0:28 / /run/lxcfs/controllers/devices rw,relatime shared:131 - cgroup devices rw,devices
165 146 0:27 / /run/lxcfs/controllers/memory rw,relatime shared:133 - cgroup memory rw,memory
167 146 0:26 / /run/lxcfs/controllers/cpuset rw,relatime shared:135 - cgroup cpuset rw,cpuset,clone_children
169 146 0:24 / /run/lxcfs/controllers/name=systemd rw,relatime shared:137 - cgroup name=systemd rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
171 25 0:48 / /var/lib/lxcfs rw,nosuid,nodev,relatime shared:139 - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
176 63 0:41 /shmounts /var/lib/lxd/shmounts rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/shmounts
177 72 0:41 /shmounts /mnt/lxd_images/shmounts rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/shmounts
180 23 0:50 / /run/user/1000 rw,nosuid,nodev,relatime shared:143 - tmpfs tmpfs rw,size=1615856k,mode=700,uid=1000,gid=1000
292 42 0:55 / /proc/sys/fs/binfmt_misc rw,relatime shared:151 - binfmt_misc binfmt_misc rw
182 63 8:1 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-video rw,relatime master:37 - ext4 /dev/sda1 rw,data=ordered
183 72 8:1 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
188 63 8:18 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-music rw,relatime master:40 - ext4 /dev/sdb2 rw,data=ordered
189 72 8:18 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
194 63 8:17 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-dvd_rips rw,relatime master:39 - ext4 /dev/sdb1 rw,data=ordered
195 72 8:17 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
thanks for your help  Rob

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20160630/1663237c/attachment-0001.html>