[lxc-devel] Unprivileged setns
S.Çağlar Onur
caglar at 10ur.org
Mon Jan 20 21:59:47 UTC 2014
Hi Stéphane,
On Mon, Jan 20, 2014 at 4:41 PM, Stéphane Graber <stgraber at ubuntu.com> wrote:
> On Mon, Jan 20, 2014 at 04:19:33PM -0500, S.Çağlar Onur wrote:
>> Hi Stéphane,
>>
>> On Mon, Jan 20, 2014 at 4:06 PM, S.Çağlar Onur <caglar at 10ur.org> wrote:
>> > Hi Stéphane,
>> >
>> > On Mon, Jan 20, 2014 at 3:08 PM, Stéphane Graber <stgraber at ubuntu.com> wrote:
>> >> Hey everyone,
>> >>
>> >> So I spent some time this morning trying to figure out if setns to
>> >> unprivileged containers was even possible.
>> >>
>> >> The good news is that it is, the bad news is that setns back to the
>> >> original namespace isn't allowed.
>> >>
>> >> So in practice, with this patch[0] I can get lxc-info to show
>> >> information from an unprivileged containers like so:
>> >>
>> >> stgraber at castiana:~/Desktop$ lxc-info -n p1
>> >> Name: p1
>> >> State: RUNNING
>> >> PID: 24704
>> >> lxc_container: Operation not permitted - failed to setns
>> >> lxc_container: Operation not permitted - failed to setns
>> >> IP: 10.0.3.124
>> >> IP: 10.0.4.1
>> >> IP: 2607:f2c0:f00f:2751:5c41:a8fd:1169:6041
>> >> IP: 2607:f2c0:f00f:2751:ac15:54ff:fed5:8b4a
>> >
>> > Cool, I was trying to do the same this morning with no luck. I just
>> > tried your patch to see what will happen and unfortunately it's pretty
>> > much the same error that I was getting.
>> >
>> > I'm on saucy with trusy kernel and now started to wonder whether this
>> > could be related with the libc version that I'm using or something
>> > else. Here is what happens with your patch applied (with two
>> > additional lines for writing the function name for debugging).
>>
>> Hmm, it looks like go runtime is causing some trouble because calling
>> the get_ips directly via C API works as expected.
>
> Good to hear it's just a binding issue and not some very weird and
> twisted userspace library problem :)
I'm not :) Still trying to figure out the reason but looks like setns
only reassociates the calling thread with the namespace and since go
runtime executes c code in a different thread (and possibly moves it
from one OS thread to another before or after the execution of the C
code) things go crazy.
> Btw, I just got lxc-attach and attach() in the API to work with
> unprivileged containers though I have to track down and fix a cgroup
> problem before I send the patch to the mailinglist (attach works but not
> if you have it join the cgroup too...).
>
>
> Now for get_ips and get_interfaces, based on Eric's reply, there's no
> way we can do it by just tweaking our current implementation as even
> when keeping an open fd to the old namespace, setns will fail...
>
> I had a thought of instead adding those two functions to the monitor API
> and have lxc-start query those values for us but unfortunately lxc-start
> is also outside the network namespace so would also have to setns and
> would also be stuck with no way to revert back to the original
> namespace.
>
> So it seems like our only way out is to have a function which opens a
> communication fd (or something), forks, switches namespace, does what we
> want and returns the data over the fd. The same function would obviously
> work with both privileged and unprivileged containers, so we probably
> shouldn't make that a special case for unprivileged containers and just
> consistently get rid of in-process setns calls (all the attach functions
> already do it in a child).
Yeah this also sounds like a solution to my go <-> C interaction
problem. Will try to implement something messy for myself to see
whether it really solves my go problem. In the meantime if anyone
wants to implement this for the upstream, please do no't hesitate as
I'm not sure whether I'll succeed with my try.
>
>>
>> [caglar at oOo:~/Projects/lxc-c] cat ip.c
>> #include <stdio.h>
>> #include <stdlib.h>
>>
>> #include <lxc/lxccontainer.h>
>>
>> int main(int argc, char *argv[]) {
>> int i;
>> struct lxc_container *c;
>>
>> c = lxc_container_new("rubik", NULL);
>>
>> char **addresses = c->get_ips(c, NULL, NULL, 0);
>> if (addresses) {
>> char *address;
>> i = 0;
>> while (addresses[i]) {
>> address = addresses[i];
>> printf("IP: %s\n", address);
>> i++;
>> }
>> }
>>
>> lxc_container_put(c);
>>
>> return 0;
>> }
>>
>> [caglar at oOo:~/Projects/lxc-c] gcc ip.c -llxc -o ip
>> [caglar at oOo:~/Projects/lxc-c] ./ip
>> lxc_container: enter_to_ns
>> lxc_container: exit_from_ns
>> lxc_container: Operation not permitted - failed to setns CLONE_NEWUSER
>> lxc_container: Operation not permitted - failed to setns CLONE_NEWNET
>> IP: 10.0.3.153
>> [caglar at oOo:~/Projects/lxc-c]
>>
>>
>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./list
>> > You are using LXC as an unprivileged user. Some functionality may not work.
>> >
>> > 2014/01/20 16:03:42 Defined containers:
>> > 2014/01/20 16:03:42 rubik (STOPPED)
>> > 2014/01/20 16:03:42
>> > 2014/01/20 16:03:42 Active containers:
>> > 2014/01/20 16:03:42
>> > 2014/01/20 16:03:42 Active and Defined containers:
>> >
>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./start
>> > You are using LXC as an unprivileged user. Some functionality may not work.
>> >
>> > 2014/01/20 16:03:43 Starting the container...
>> >
>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./list
>> > You are using LXC as an unprivileged user. Some functionality may not work.
>> >
>> > 2014/01/20 16:03:44 Defined containers:
>> > 2014/01/20 16:03:44 rubik (RUNNING)
>> > 2014/01/20 16:03:44
>> > 2014/01/20 16:03:44 Active containers:
>> > 2014/01/20 16:03:44 rubik (RUNNING)
>> > 2014/01/20 16:03:44
>> > 2014/01/20 16:03:44 Active and Defined containers:
>> > 2014/01/20 16:03:44 rubik (RUNNING)
>> >
>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./ipaddress
>> > You are using LXC as an unprivileged user. Some functionality may not work.
>> >
>> > 2014/01/20 16:03:47 IPAddress("lo")
>> > lxc_container: enter_to_ns
>> > lxc_container: Invalid argument - failed to setns CLONE_NEWUSER
>> > lxc_container: exit_from_ns
>> > lxc_container: Invalid argument - failed to setns CLONE_NEWUSER
>> > lxc_container: Operation not permitted - failed to setns CLONE_NEWNET
>> > lxc_container: exit_from_ns
>> > lxc_container: Bad file descriptor - failed to setns CLONE_NEWUSER
>> > lxc_container: Bad file descriptor - failed to setns CLONE_NEWNET
>> > 2014/01/20 16:03:47 ERROR: getting IP address on the interface lo of
>> > the container "rubik" failed
>> >
>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] sudo
>> > ./ipaddress -lxcpath=/home/caglar/.local/share/lxc
>> > 2014/01/20 16:05:37 IPAddress("lo")
>> > lxc_container: enter_to_ns
>> > lxc_container: exit_from_ns
>> > 2014/01/20 16:05:37 0) 127.0.0.1
>> > 2014/01/20 16:05:37 1) ::1
>> > 2014/01/20 16:05:37 IPAddresses()
>> > lxc_container: enter_to_ns
>> > lxc_container: exit_from_ns
>> > 2014/01/20 16:05:37 0) 10.0.3.153
>> > 2014/01/20 16:05:37 IPv4Addresses()
>> > lxc_container: enter_to_ns
>> > lxc_container: exit_from_ns
>> > 2014/01/20 16:05:37 0) 10.0.3.153
>> > 2014/01/20 16:05:37 IPv6Addresses()
>> > lxc_container: enter_to_ns
>> > lxc_container: exit_from_ns
>> > 2014/01/20 16:05:37 ERROR: getting IPv6 addresses of the container
>> > "rubik" failed
>> >
>> >
>> >>
>> >> The problem obviously comes from those two error messages which say that
>> >> setns back to the original namespace failed.
>> >>
>> >> I can't think of a nice way around this particular limitation nor am I
>> >> convinced that there is any safe way to fix that at the kernel level.
>> >> (CCing Eric in case there's something I missed)
>> >>
>> >> The obvious thing we could do is instead of doing the setns calls in
>> >> process, instead fork a child, have it do the setns and send the result
>> >> of the command back to the original caller. However doing that is likely
>> >> going to take a bit more time than I have available right now, so if
>> >> someone's interested, that'd be a great thing to have for 1.0.
>> >>
>> >> [0] http://paste.ubuntu.com/6787869/
>> >>
>> >> --
>> >> Stéphane Graber
>> >> Ubuntu developer
>> >> http://www.ubuntu.com
>> >>
>> >> _______________________________________________
>> >> lxc-devel mailing list
>> >> lxc-devel at lists.linuxcontainers.org
>> >> http://lists.linuxcontainers.org/listinfo/lxc-devel
>> >>
>> >
>> > Best,
>> > --
>> > S.Çağlar Onur <caglar at 10ur.org>
>>
>>
>>
>> --
>> S.Çağlar Onur <caglar at 10ur.org>
>> _______________________________________________
>> lxc-devel mailing list
>> lxc-devel at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
> --
> Stéphane Graber
> Ubuntu developer
> http://www.ubuntu.com
>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
--
S.Çağlar Onur <caglar at 10ur.org>
More information about the lxc-devel
mailing list