[lxc-devel] Unprivileged setns
S.Çağlar Onur
caglar at 10ur.org
Tue Jan 21 00:17:57 UTC 2014
On Mon, Jan 20, 2014 at 4:59 PM, S.Çağlar Onur <caglar at 10ur.org> wrote:
> Hi Stéphane,
>
> On Mon, Jan 20, 2014 at 4:41 PM, Stéphane Graber <stgraber at ubuntu.com> wrote:
>> On Mon, Jan 20, 2014 at 04:19:33PM -0500, S.Çağlar Onur wrote:
>>> Hi Stéphane,
>>>
>>> On Mon, Jan 20, 2014 at 4:06 PM, S.Çağlar Onur <caglar at 10ur.org> wrote:
>>> > Hi Stéphane,
>>> >
>>> > On Mon, Jan 20, 2014 at 3:08 PM, Stéphane Graber <stgraber at ubuntu.com> wrote:
>>> >> Hey everyone,
>>> >>
>>> >> So I spent some time this morning trying to figure out if setns to
>>> >> unprivileged containers was even possible.
>>> >>
>>> >> The good news is that it is, the bad news is that setns back to the
>>> >> original namespace isn't allowed.
>>> >>
>>> >> So in practice, with this patch[0] I can get lxc-info to show
>>> >> information from an unprivileged containers like so:
>>> >>
>>> >> stgraber at castiana:~/Desktop$ lxc-info -n p1
>>> >> Name: p1
>>> >> State: RUNNING
>>> >> PID: 24704
>>> >> lxc_container: Operation not permitted - failed to setns
>>> >> lxc_container: Operation not permitted - failed to setns
>>> >> IP: 10.0.3.124
>>> >> IP: 10.0.4.1
>>> >> IP: 2607:f2c0:f00f:2751:5c41:a8fd:1169:6041
>>> >> IP: 2607:f2c0:f00f:2751:ac15:54ff:fed5:8b4a
>>> >
>>> > Cool, I was trying to do the same this morning with no luck. I just
>>> > tried your patch to see what will happen and unfortunately it's pretty
>>> > much the same error that I was getting.
>>> >
>>> > I'm on saucy with trusy kernel and now started to wonder whether this
>>> > could be related with the libc version that I'm using or something
>>> > else. Here is what happens with your patch applied (with two
>>> > additional lines for writing the function name for debugging).
>>>
>>> Hmm, it looks like go runtime is causing some trouble because calling
>>> the get_ips directly via C API works as expected.
>>
>> Good to hear it's just a binding issue and not some very weird and
>> twisted userspace library problem :)
>
> I'm not :) Still trying to figure out the reason but looks like setns
> only reassociates the calling thread with the namespace and since go
> runtime executes c code in a different thread (and possibly moves it
> from one OS thread to another before or after the execution of the C
> code) things go crazy.
>
>> Btw, I just got lxc-attach and attach() in the API to work with
>> unprivileged containers though I have to track down and fix a cgroup
>> problem before I send the patch to the mailinglist (attach works but not
>> if you have it join the cgroup too...).
>>
>>
>> Now for get_ips and get_interfaces, based on Eric's reply, there's no
>> way we can do it by just tweaking our current implementation as even
>> when keeping an open fd to the old namespace, setns will fail...
>>
>> I had a thought of instead adding those two functions to the monitor API
>> and have lxc-start query those values for us but unfortunately lxc-start
>> is also outside the network namespace so would also have to setns and
>> would also be stuck with no way to revert back to the original
>> namespace.
>>
>> So it seems like our only way out is to have a function which opens a
>> communication fd (or something), forks, switches namespace, does what we
>> want and returns the data over the fd. The same function would obviously
>> work with both privileged and unprivileged containers, so we probably
>> shouldn't make that a special case for unprivileged containers and just
>> consistently get rid of in-process setns calls (all the attach functions
>> already do it in a child).
>
> Yeah this also sounds like a solution to my go <-> C interaction
> problem. Will try to implement something messy for myself to see
> whether it really solves my go problem. In the meantime if anyone
> wants to implement this for the upstream, please do no't hesitate as
> I'm not sure whether I'll succeed with my try.
Fork method seems to be working fine here. Will civilize the code and
submit for review.
[caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./ipaddress
You are using LXC as an unprivileged user. Some functionality may not work.
2014/01/20 19:15:51 IPAddress("lo")
2014/01/20 19:15:51 0) 127.0.0.1
2014/01/20 19:15:51 1) ::1
2014/01/20 19:15:51 IPAddresses()
2014/01/20 19:15:51 0) 10.0.3.153
2014/01/20 19:15:51 IPv4Addresses()
2014/01/20 19:15:51 0) 10.0.3.153
2014/01/20 19:15:51 IPv6Addresses()
>>
>>>
>>> [caglar at oOo:~/Projects/lxc-c] cat ip.c
>>> #include <stdio.h>
>>> #include <stdlib.h>
>>>
>>> #include <lxc/lxccontainer.h>
>>>
>>> int main(int argc, char *argv[]) {
>>> int i;
>>> struct lxc_container *c;
>>>
>>> c = lxc_container_new("rubik", NULL);
>>>
>>> char **addresses = c->get_ips(c, NULL, NULL, 0);
>>> if (addresses) {
>>> char *address;
>>> i = 0;
>>> while (addresses[i]) {
>>> address = addresses[i];
>>> printf("IP: %s\n", address);
>>> i++;
>>> }
>>> }
>>>
>>> lxc_container_put(c);
>>>
>>> return 0;
>>> }
>>>
>>> [caglar at oOo:~/Projects/lxc-c] gcc ip.c -llxc -o ip
>>> [caglar at oOo:~/Projects/lxc-c] ./ip
>>> lxc_container: enter_to_ns
>>> lxc_container: exit_from_ns
>>> lxc_container: Operation not permitted - failed to setns CLONE_NEWUSER
>>> lxc_container: Operation not permitted - failed to setns CLONE_NEWNET
>>> IP: 10.0.3.153
>>> [caglar at oOo:~/Projects/lxc-c]
>>>
>>>
>>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./list
>>> > You are using LXC as an unprivileged user. Some functionality may not work.
>>> >
>>> > 2014/01/20 16:03:42 Defined containers:
>>> > 2014/01/20 16:03:42 rubik (STOPPED)
>>> > 2014/01/20 16:03:42
>>> > 2014/01/20 16:03:42 Active containers:
>>> > 2014/01/20 16:03:42
>>> > 2014/01/20 16:03:42 Active and Defined containers:
>>> >
>>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./start
>>> > You are using LXC as an unprivileged user. Some functionality may not work.
>>> >
>>> > 2014/01/20 16:03:43 Starting the container...
>>> >
>>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./list
>>> > You are using LXC as an unprivileged user. Some functionality may not work.
>>> >
>>> > 2014/01/20 16:03:44 Defined containers:
>>> > 2014/01/20 16:03:44 rubik (RUNNING)
>>> > 2014/01/20 16:03:44
>>> > 2014/01/20 16:03:44 Active containers:
>>> > 2014/01/20 16:03:44 rubik (RUNNING)
>>> > 2014/01/20 16:03:44
>>> > 2014/01/20 16:03:44 Active and Defined containers:
>>> > 2014/01/20 16:03:44 rubik (RUNNING)
>>> >
>>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] ./ipaddress
>>> > You are using LXC as an unprivileged user. Some functionality may not work.
>>> >
>>> > 2014/01/20 16:03:47 IPAddress("lo")
>>> > lxc_container: enter_to_ns
>>> > lxc_container: Invalid argument - failed to setns CLONE_NEWUSER
>>> > lxc_container: exit_from_ns
>>> > lxc_container: Invalid argument - failed to setns CLONE_NEWUSER
>>> > lxc_container: Operation not permitted - failed to setns CLONE_NEWNET
>>> > lxc_container: exit_from_ns
>>> > lxc_container: Bad file descriptor - failed to setns CLONE_NEWUSER
>>> > lxc_container: Bad file descriptor - failed to setns CLONE_NEWNET
>>> > 2014/01/20 16:03:47 ERROR: getting IP address on the interface lo of
>>> > the container "rubik" failed
>>> >
>>> > [caglar at oOo:~/go/src/github.com/caglar10ur/lxc/examples] sudo
>>> > ./ipaddress -lxcpath=/home/caglar/.local/share/lxc
>>> > 2014/01/20 16:05:37 IPAddress("lo")
>>> > lxc_container: enter_to_ns
>>> > lxc_container: exit_from_ns
>>> > 2014/01/20 16:05:37 0) 127.0.0.1
>>> > 2014/01/20 16:05:37 1) ::1
>>> > 2014/01/20 16:05:37 IPAddresses()
>>> > lxc_container: enter_to_ns
>>> > lxc_container: exit_from_ns
>>> > 2014/01/20 16:05:37 0) 10.0.3.153
>>> > 2014/01/20 16:05:37 IPv4Addresses()
>>> > lxc_container: enter_to_ns
>>> > lxc_container: exit_from_ns
>>> > 2014/01/20 16:05:37 0) 10.0.3.153
>>> > 2014/01/20 16:05:37 IPv6Addresses()
>>> > lxc_container: enter_to_ns
>>> > lxc_container: exit_from_ns
>>> > 2014/01/20 16:05:37 ERROR: getting IPv6 addresses of the container
>>> > "rubik" failed
>>> >
>>> >
>>> >>
>>> >> The problem obviously comes from those two error messages which say that
>>> >> setns back to the original namespace failed.
>>> >>
>>> >> I can't think of a nice way around this particular limitation nor am I
>>> >> convinced that there is any safe way to fix that at the kernel level.
>>> >> (CCing Eric in case there's something I missed)
>>> >>
>>> >> The obvious thing we could do is instead of doing the setns calls in
>>> >> process, instead fork a child, have it do the setns and send the result
>>> >> of the command back to the original caller. However doing that is likely
>>> >> going to take a bit more time than I have available right now, so if
>>> >> someone's interested, that'd be a great thing to have for 1.0.
>>> >>
>>> >> [0] http://paste.ubuntu.com/6787869/
>>> >>
>>> >> --
>>> >> Stéphane Graber
>>> >> Ubuntu developer
>>> >> http://www.ubuntu.com
>>> >>
>>> >> _______________________________________________
>>> >> lxc-devel mailing list
>>> >> lxc-devel at lists.linuxcontainers.org
>>> >> http://lists.linuxcontainers.org/listinfo/lxc-devel
>>> >>
>>> >
>>> > Best,
>>> > --
>>> > S.Çağlar Onur <caglar at 10ur.org>
>>>
>>>
>>>
>>> --
>>> S.Çağlar Onur <caglar at 10ur.org>
>>> _______________________________________________
>>> lxc-devel mailing list
>>> lxc-devel at lists.linuxcontainers.org
>>> http://lists.linuxcontainers.org/listinfo/lxc-devel
>>
>> --
>> Stéphane Graber
>> Ubuntu developer
>> http://www.ubuntu.com
>>
>> _______________________________________________
>> lxc-devel mailing list
>> lxc-devel at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-devel
>>
>
>
>
> --
> S.Çağlar Onur <caglar at 10ur.org>
--
S.Çağlar Onur <caglar at 10ur.org>
More information about the lxc-devel
mailing list