[lxc-users] Enabling sys_nice in a privileged container

[Peter Steele]

On 09/05/2015 10:35 AM, Peter Steele wrote:
> I have a privileged container that runs ctdb and needs to have real 
> time scheduling enabled. The error reported by ctdb is:
>
> Sep 05 10:27:05 pws-01-vm-05 systemd[1]: Starting CTDB...
> Sep 05 10:27:06 pws-01-vm-05 ctdbd[1598]: CTDB starting on node
> Sep 05 10:27:06 pws-01-vm-05 ctdbd[1599]: Starting CTDBD (Version 
> 2.5.4) as PID: 1599
> Sep 05 10:27:06 pws-01-vm-05 ctdbd[1599]: Created PID file 
> /run/ctdb/ctdbd.pid
> Sep 05 10:27:06 pws-01-vm-05 ctdbd[1599]: Unable to set scheduler to 
> SCHED_FIFO (Operation not permitted)
> Sep 05 10:27:06 pws-01-vm-05 ctdbd[1599]: CTDB daemon shutting down
>
> Apparently. my container is dropping the sys_nice capability which is 
> needed for real time scheduling. I thought I could just add the line
>
> lxc.cap.keep = sys_nice
>
> but this has the side effect of dropping all capabilities except this 
> one so that just made things worse. What is the correct way to enable 
> a specific capability for a container?
>
> I'm running CentOS 7 and am using a custom template. By config is 
> pretty basic with just the following parameters defined:
>
> lxc.tty = 4
> lxc.pts = 1024
> lxc.utsname = test
> lxc.network.type = veth
> lxc.network.flags = up
> lxc.network.link = br0
> lxc.network.veth.pair = veth-test
> lxc.network.hwaddr = 00:16:3e:16:ef:32
> lxc.rootfs = /lxc/test
>
> Peter
>

For what it's worth, I hit this very same problem when we were using 
libvirt-lxc containers. In that case though the containers did have 
sys_nice enabled but were still unable to schedule real time jobs 
because by default libvirt doesn't give any real time bandwidth to 
containers. This is defined by the cgroup parameter

/sys/fs/cgroup/cpu/cpu.rt_runtime_us

This is zero by default for libvirt containers, and setting this solved 
the CTDB issue. LXC containers *do* set this parameter though, so the 
error in this case is because sys_nice has been dropped for the 
containers. Or at least that's my assumption. The underlying call being 
made is pthread_create, and our own real-time threads are failing as 
well with the same "operation not permitted" error. So the containers 
are clearly lacking the required permissions.