[Lxc-users] lxc-start leaves temporary pivot dir behind

[Daniel Lezcano]

Michael H. Warfield wrote:
> On Wed, 2010-05-12 at 23:18 +0200, Daniel Lezcano wrote: 
>   
>> Ferenc Wagner wrote:
>>     
>>> Daniel Lezcano <daniel.lezcano at free.fr> writes:
>>>
>>>   
>>>       
>>>> Ferenc Wagner wrote:
>>>>
>>>>     
>>>>         
>>>>> Daniel Lezcano <daniel.lezcano at free.fr> writes:
>>>>>   
>>>>>       
>>>>>           
>>>>>> Ferenc Wagner wrote:
>>>>>>     
>>>>>>         
>>>>>>             
>>>>>>> Actually, I'm not sure you can fully solve this.  If rootfs is a
>>>>>>> separate file system, this is only much ado about nothing.  If rootfs
>>>>>>> isn't a separate filesystem, you can't automatically find a good
>>>>>>> place and also clean it up.
>>>>>>>           
>>>>>>>               
>>>>>> Maybe a single /tmp/lxc directory may be used as the mount points are
>>>>>> private to the container. So it would be acceptable to have a single
>>>>>> directory for N containers, no ?
>>>>>>         
>>>>>>             
>>>>> Then why not /usr/lib/lxc/pivotdir or something like that?  Such a
>>>>> directory could belong to the lxc package and not clutter up /tmp.  As
>>>>> you pointed out, this directory would always be empty in the outer name
>>>>> space, so a single one would suffice.  Thus there would be no need
>>>>> cleaning it up, either.
>>>>>       
>>>>>           
>>>> Agree. Shall we consider $(prefix)/var/run/lxc ?
>>>>     
>>>>         
>>> Hmm, /var/run/lxc is inconvenient, because it disappears on each reboot
>>> if /var/run is on tmpfs.  This isn't variable data either, that's why I
>>> recommended /usr above.
>>>   
>>>       
>> Good point. I will change that to /usr/$(libdir)/lxc and let the distro 
>> maintainer to choose a better place if he wants with the configure option.
>>     
>
> Are you SURE you want /usr/${libdir}/lxc for this?  Some high security
> systems might mount /usr as a separate read-only partition (OK - I'm and
> old school old fart).  Part of the standard allows for /usr to be an RO
> file system.
>   
I am not sure it is the right location, but lxc does not create the 
directory and, as you mention, I don't want to do that due to RO fs. It 
is up to the admin to configure the system with this directory.

The configure option allows to specify the location of the directory, so 
I let the lxc distro maintainers to choose the location they want and to 
create the directory in the package post-install section.

Furthermore, in the config option of lxc, you can change the location 
with lxc.rootfs.mount=<path>

lxc mounts only the rootfs on this directory. As we are in the private 
mount namespace, the mount point is only visible from the container, and 
not externally, so we can reuse this directory to mount another rootfs 
for another container. In other words, one directory for all the containers.

> Wouldn't this be more appropriate in /var/${libdir}/lxc instead?  Maybe
> create a .tmp directory under it or .tmp.${CTID} or something?  Or,
> maybe, something under /var/${libdir}/lxc/${CTID}/tmp instead?  /var is
> for things that change and vary.  Wouldn't that be a better location and
> you've already got control of the /var/${libdir}/lxc location, don't
> you?
>   

We are talking about the default location when nothing is specified for 
the configure.
We need just one directory and I am not sure it is good to place it in 
/var/lib/lxc because these are for containers configuration and the rule 
is one directory in /var/lib/lxc == one container.

Putting it in /var/lib/{ctrid}/ is not good because we can spawn a 
'volatile' container without a previous creation (eg. lxc-start -n foo 
-f configfile, or lxc-start -n foo -s lxc.rootfs=myrootfs -s 
lxc.network.type=veth ...)

But I am open if there is a better default location. /var/run/lxc was 
proposed but Ferenc pointed it is mounted as a tmpfs on some distro 
(like ubuntu) and we will need to create it at each reboot.

Thanks
  -- Daniel





>>>>> Now the question is: if rootfs is a separate file system (which
>>>>> includes bind mounts), is the superfluous rbind of the original root
>>>>> worth skipping, or should we just do it to avoid needing an extra
>>>>> code path?
>>>>>       
>>>>>           
>>>> Good question. IMO, skipping the rbind is ok for this case but it may
>>>> be interesting from a coding point of view to have a single place
>>>> identified for the rootfs (especially for mounting an image). I will
>>>> cook a patchset to fix the rootfs location and then we can look at
>>>> removing the superfluous rbind.
>>>>     
>>>>         
>>> I'm testing your patchset now.  So far it seems to work as advertised.
>>>   
>>>       
>> Cool, thanks for testing.
>>     
>
> Regards,
> Mike
>