<div dir="ltr">Hey Serge and Maxim,<div><br></div><div>I've been busy with some work here and haven't had a lot of time to look into this. I can spend sometime now to help out.</div><div><br></div><div>Since I don't have much idea of how to go about creating the graph driver proxy for docker, I started by <span style="line-height:1.5">trying to see what problems we get when starting docker 1.10 experimental daemon inside an unprivileged container and seems that it fails to start with an error "Error starting daemon: Devices cgroup isn't mounted". Now, this seems to be an error unrelated to what the graph driver would resolve, but please correct me if I'm wrong as I'm quite new to lxc or docker dev. Looking at the docker code [1], it looks like the libcontainer which does parsing of cgroup mount point doesn't take into consideration the fact that cgroups are running on lxcfs inside the container. I'm now investigating what the solution could be to solve this. Let me know if you have any ideas.</span></div><div><span style="line-height:1.5"><br></span></div><div><span style="line-height:1.5">Also, do you think it makes more sense to have this discussion on lxc-devel than lxc-users?</span></div><div><span style="line-height:1.5"><br></span></div><div><span style="line-height:1.5">1 - </span><span style="line-height:1.5"><a href="https://github.com/docker/docker/blob/master/vendor/src/github.com/opencontainers/runc/libcontainer/cgroups/utils.go#L22-L47" target="_blank">https://github.com/docker/docker/blob/master/vendor/src/github.com/opencontainers/runc/libcontainer/cgroups/utils.go#L22-L47</a></span><br></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Nov 9, 2015 at 5:25 PM Maxim Patlasov <<a href="mailto:mpatlasov@parallels.com" target="_blank">mpatlasov@parallels.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 11/06/2015 02:52 PM, Serge Hallyn wrote:<br>
> Quoting Maxim Patlasov (<a href="mailto:mpatlasov@parallels.com" target="_blank">mpatlasov@parallels.com</a>):<br>
>> Hi Serge,<br>
>><br>
>> I had been working for a while on porting proxy-graphdriver-daemon<br>
>> to extpoint feature, but then switched to another task. I hope to<br>
>> switch back in a week. It would be great if we all together come to<br>
>> agreement about universal way to mount something from host to<br>
>> container namespace. The simplest way would be to specify the pid of<br>
>> container "init" as a command-line arg of proxy-daemon, so it could<br>
>> use the pid for setns(2) directly. Is such an approach safe enough<br>
>> and will work for all of us?<br>
> Surely safe enough. It's too bad that it requires a graphdriver<br>
> per docker-using container, and that it breaks up the container<br>
> start (I can't start the graphdriver first and just pass the unix<br>
> socket into the container), but I think it's ok.<br>
><br>
> In general, what do we need exactly?<br>
><br>
> 1. Some way to identify target pid. We can<br>
> a. pass pid to graphdriver on cmdline<br>
> b. we can get pid from peercred<br>
> 2. A MS_SLAVE directory to allow the mount to be passed from the<br>
> host to the container (whereupon it can be moved to its final<br>
> destination. This path location needs to be passed to the<br>
> graphdriver somehow. In what you suggest we can just pass the<br>
> absolute paths (both on the host and in the container) on the<br>
> command line as well.<br>
> 3. An actual request, presumably sent as<br>
> some-host-dev-id destination-path<br>
> over a unix socket.<br>
<br>
After more thinking it came to me that may be we can avoid all this<br>
hassle altogether. When I started to work on proxy-grpahdriver, I tried<br>
to follow the behavior of devicemapper graphdriver as close as possible.<br>
In particular, I thought it's important to create mount-points where the<br>
client (docker inside container) would expect to find them (under<br>
/var/lib/docker by default). But now, having a look at<br>
integration-cli/docker_cli_external_graphdriver_unix_test.go from<br>
<a href="https://github.com/docker/docker/pull/13777/files" rel="noreferrer" target="_blank">https://github.com/docker/docker/pull/13777/files</a>, it seems that<br>
mount-points may be placed anywhere. If it's true we could place them in<br>
a directory visible both from host system and inside container -- no<br>
need for setns(2) at all!<br>
<br>
In fact, we need a place visible both from host and container anyway --<br>
to have unix socket accessible both by proxy-daemon (running on host)<br>
and docker-daemon (running inside container). In case of docker<br>
containers, such a common directory may be specified by "-v" option, like:<br>
<br>
$ docker run --privileged --rm -ti -v<br>
`pwd`:/go/src/<a href="http://github.com/docker/docker" rel="noreferrer" target="_blank">github.com/docker/docker</a> dry-run-test /bin/bash<br>
(see <a href="http://docs.docker.com/opensource/project/set-up-dev-env/" rel="noreferrer" target="_blank">http://docs.docker.com/opensource/project/set-up-dev-env/</a>)<br>
<br>
In case of OpenVZ containers, per-container /tmp is visible as<br>
/vz/root/<id>/tmp from the host, so proxy-daemon might mount requested<br>
dm-thin thin to /vz/root/<id>/tmp/tempXXX and pass "/tmp/tempXXX" back<br>
to container as result of Get() request.<br>
<br>
What about lxc -- does it have an option similar to docker' "-v"?<br>
<br>
Thanks,<br>
Maxim<br>
_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a></blockquote></div></div>