[lxc-users] Establish a bind mount to a running container

[Guido Jäkel]

On 07.10.2016 16:53, Jäkel, Guido wrote:
>> -----Original Message-----
>> From: lxc-users [mailto:lxc-users-bounces at lists.linuxcontainers.org] On Behalf Of Stéphane Graber
>> Sent: Friday, October 07, 2016 4:26 PM
>> To: LXC users mailing-list
>> Subject: Re: [lxc-users] Establish a bind mount to a running container
>>
>> On Fri, Oct 07, 2016 at 02:19:53PM +0000, Jäkel, Guido wrote:
>>>> -----Original Message-----
>>>> From: lxc-users [mailto:lxc-users-bounces at lists.linuxcontainers.org] On Behalf Of Stéphane Graber
>>>> Sent: Friday, October 07, 2016 3:38 PM
>>>> To: LXC users mailing-list
>>>> Subject: Re: [lxc-users] Establish a bind mount to a running container
>>>>
>>>> On Fri, Oct 07, 2016 at 01:26:26PM +0000, Jäkel, Guido wrote:
>>>>>> -----Original Message-----
>>>>>> From: lxc-users [mailto:lxc-users-bounces at lists.linuxcontainers.org] On Behalf Of Stéphane Graber
>>>>>> Sent: Friday, October 07, 2016 11:46 AM
>>>>>> To: LXC users mailing-list
>>>>>> Subject: Re: [lxc-users] Establish a bind mount to a running container
>>>>>>
>>>>>> On Fri, Oct 07, 2016 at 07:03:21AM +0000, Jäkel, Guido wrote:
>>>>>>> Dear experts,
>>>>>>>
>>>>>>> I wonder if it's possible to establish a bind mount filesystem resource from the LXC host to an already running
>>>> container in
>>>>>> an manual way, but analogous as it is done at startup time.
>>>>>>>
>>>>>>> I already figured out that the releasing an existing link is no thing; just umount it from inside the container. But is
>>>> there a
>>>>>> way to establish one while shifting the destination of a bind mount into the right namespace?
>>>>>>>
>>>>>>> I ask about, because in a couple of days I have to change a (NFS) filesystem source (because of an hardware
>> migration)
>>>>>> that is common to a large number of running containers but not frequently used and I want to avoid to restart all
>> the
>>>>>> containers with it services.
>>>>>>>
>>>>>>> thank you for advice
>>>>>>>
>>>>>>> Guido
>>>>>>
>>>>>> It's very difficult due to a number of restrictions in place in the kernel.
>>>>>>
>>>>>> The only way of doing this that I'm aware of is what we do in LXD. We
>>>>>> create a path on the host before the container starts, put that on a
>>>>>> rshared mountpoint, then bind-mount that directory into the container
>>>>>> under some arbitrary path.
>>>>>>
>>>>>> Then when you want to inject a new mount in the container, you can mount
>>>>>> it in a sub-directory of that path you create on the host, which will
>>>>>> then have the container inherit the mount entry thanks to the host
>>>>>> mountpoint being rshared and the container's mountpoint being rslave.
>>>>>>
>>>>>> Once the mountpoint shows up in the container, you can then move it to
>>>>>> whatever path you actually want it on.
>>>>>
>>>>>
>>>>> Dear Stéphane,
>>>>>
>>>>> I sorry, but I don't get it yet; some of your terms and where to do it are dubious to me. Maybe an example may light it
>> up
>>>> to me:
>>>>>
>>>>> Let say, I want to inject the path  host:/mnt/some_host_mountpoint/some_directory  as a bind mount to a running
>>>> container; it should end up on  container:/import/some_container_moutpoint  . On the host, the mountpoint
>>>> host:/mnt/some_host_mountpoint is mouted to a NFS source, let say  nfshost:/some_export
>>>>>
>>>>> Now please, where to issue which commands?
>>>>
>>>> You'll have to do some research yourself or hope that someone can give
>>>> you step by step instructions :)
>>>>
>>>>
>>>> A guestimate (completely untested) would be:
>>>>
>>>> Setup steps, before you first start the container:
>>>>
>>>> 1) mkdir /tmp/shared-c1
>>>> 2) mount --bind /tmp/shared-c1 /tmp/shared-c1
>>>> 3) mount --make-rshared /tmp/shared-c1
>>>> 4) Add to /var/lib/lxc/c1/config => lxc.mount.entry=/tmp/shared-c1 /.shared none bind,create=dir 0 0
>>>> 5) lxc-start -n c1
>>>>
>>>> At which point, you could inject a new mount with:
>>>>
>>>> 1) mkdir /tmp/share-c1/some_directory
>>>> 2) mount -- bind /mnt/some_host_mountpoint/some_directory /tmp/share-c1/some_directory
>>>> 3) lxc-attach -n c1 -- mkdir -p /import/some_container_mountpoint
>>>> 4) lxc-attach -n c1 -- mount --move /.shared/some_directory /import/some_container_mountpoint
>>>>
>>>>
>>>> Again, that's a very rough approximation from what I remember the LXD
>>>> code is doing (though we're doing it in a mix of Go and C).
>>>
>>>
>>> Thanks again. I don't read through it but the first thing I read: "Steps to to before first start of the container". Does that
>> mean, that there's no chance in my case, because all the containers are already running?
>>>
>>> Or may I (miss)use an established bind mount (or even the rootfs mount) to create the path /tmp/shared "here" at it's
>> source path on the host , then "over-bind-mount-and-rshared" it on the host and then continue the injection?
>>
>> If you're lucky enough that one of your established bind-mounts is
>> rshared, then you could. But given that it's not the default, your
>> chances are pretty slim.
>>
>> It's easy enough to try though, just create a directory on the host side
>> of your existing bind-mount, bind-mount the filesystem you want over
>> that and see if you can see it in the container.
> 
> There is no tags stating a shared state in /proc/self/mountinfo,neigther on the host nor in the container. As expected, in the simple way you described there's nothing to see inside the container.

Dear Stéphane,

Now I read "Mount namespaces and shared subtrees" (https://lwn.net/Articles/689856/). And I see: The subtree sharing is handled by tagging a mount with an unique slot (numeric index). Any new shared mount will get a new index. The only way to get the same index to become a member of this sharing group is the clone operation at new namespace creation and with this you get an "already known and therefore shared" mount. Therefore: If the bind mount was not tagged *before* the containers mount namespace was cloned at it's start, there seems no way to establish such a group anymore.