[lxc-users] lxc.mount.entry selectively mount parts of sys and proc read write

[GC]

On 03/26/2014 04:07 PM, Serge Hallyn wrote:
> Quoting catchall (catchall at gc9.org):
>>
>> On 03/26/2014 12:52 PM, Serge Hallyn wrote:
>>> Quoting catchall (catchall at gc9.org):
>>>>
>>>> On 03/24/2014 05:10 PM, Serge Hallyn wrote:
>>>>> Quoting GC (catchall at gc9.org):
>>>>>> On 03/21/2014 09:11 PM, Serge Hallyn wrote:
>>>>>>> Quoting GC (catchall at gc9.org):
>>>>>>>> On 03/21/2014 07:15 AM, Serge Hallyn wrote:
>>>>>>>>> Quoting GC (catchall at gc9.org):
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I want to selectively mount parts of sys and proc rw, but the rest
>>>>>>>>>> ro.  I thought I might be able to e.g., mount /sys ro (in the
>>>>>>>>>> container), and mount /.sys rw (in the container), then bind mount
>>>>>>>>>> bits from /.sys to /sys, and finally hide the rw /.sys by mounting
>>>>>>>>>> another directory on top of it, like:
>>>>>>>>>>
>>>>>>>>>> lxc.mount.entry = sysfs sys sysfs ro 0 0
>>>>>>>>>> lxc.mount.entry = sysfs .sys sysfs rw 0 0        # (dot)sys
>>>>>>>>>>
>>>>>>>>>> lxc.mount.entry = /var/lib/lxc/container/.sys/module/ipv6
>>>>>>>>>> sys/module/ipv6 none defaults,bind 0 0
>>>>>>>>>> # or alternatively (also doesn't work) this instead of line above
>>>>>>>>>> #lxc.mount.entry = .sys/module/ipv6 sys/module/ipv6 none defaults,bind 0 0
>>>>>>>>>>
>>>>>>>>>> lxc.mount.entry = /var/lib/lxc/dummy_mount .sys none ro,bind 0 0
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The part where I try to perform the bind mount of the read/write
>>>>>>>>>> .sys/module/ipv6 (in the container) on top of the read only
>>>>>>>>>> sys/module/ipv6  (in the container) fails.  Is there  a way to get
>>>>>>>>>> this to work?
>>>>>>>>> Wouldn't it be simpler to simply bind mount /sys ro from the host,
>>>>>>>>> then bind-mount /sys/module/ipv6 from the host rw into the container?
>>>>>>>> I thought there would be issues with namespace support.  I thought
>>>>>>>> it would break network namespaces, which appears to be wrong from
>>>>>>> Oh - yeah, right you are.
>>>>>>>
>>>>>> Still looking at docs, but I think it should be possible, but not as
>>>>>> clean as if lxc.mount.entry did it.  From docs it looks like I can
>>>>>> get a descriptor from /proc/containerPID/ns/mnt, and use sentns to
>>>>>> join the container's mount namespace, and then do the bind mounts.
>>>>> You'd want to also setns to the container netns so as to get the
>>>>> right /sys/class/net, of course.
>>>>>
>>>>>> Seems there must be an easier/better way though.
>>>>>>
>>>>>> Ideas?
>>>>> (Without looking back at previous messages,) Have you tried using a
>>>>> mount hook?
>>>>>
>>>> Whenever I try to use any of the mount hooks, I get this error and
>>>> the container doesn't start:
>>>>
>>>> lxc-start: command get_cgroup failed to receive response
>>>>
>>>>
>>>> I even tried a hook script that didn't do anything, just ran a
>>>> single "echo", so it isn't the content of the script.
>>>>
>>>> I am running lxc version: 1.0.0.alpha2.
>>> The hook was executable I assume?  Can you do
>> The real scrips were, the last test of just echo was not. Do'h
>>
>>> lxc-start -n container -l trace -o xxx
>>>
>>> and append xxx here?
>> Non-zero exit codes from the mount failing were causing the
>> lxc-start error.  Hook scripts run now, but bind mounts don't work
>> since it looks like sys and proc aren't mounted yet even though,
>> from the trace, it looks like they are:
>> .
>> .
>> .
>>        lxc-start 1395870561.553 DEBUG    lxc_conf - mounted 'proc' on
>> '/usr/lib/x86_64-linux-gnu/lxc/pr
>> oc', type 'proc'
>>         lxc-start 1395870561.553 DEBUG    lxc_conf - mounted 'proc'
>> on '/usr/lib/x86_64-linux-gnu/lxc/.p
>> roc', type 'proc'
>>        lxc-start 1395870561.553 INFO     lxc_conf - mount points have
>> been setup
>>        lxc-start 1395870561.553 INFO     lxc_conf - Executing script
>> '/var/lib/lxc/nsmaster/hook.mount' for container 'nsmaster', config
>> section 'lxc'
>> lxc-start 1395870561.569 INFO     lxc_conf - console has been setup
>>
>>
>> /var/lib/lxc/nsmaster/hook.mount:
>> #!/bin/sh
>> ls -a $LXC_ROOTFS_PATH/proc > /tmp/log 2>&1
>> echo "#######" >> /tmp/log
>> ls -a $LXC_ROOTFS_PATH/sys >> /tmp/log 2>&1
> $LXC_ROOTFS_PATH is probably not what you want, rather
> $LXC_ROOTFS_MOUNT.  You can look at
> /usr/share/lxc/hooks/ubuntu-cloud-prep for a nice list
> of the available variables.
>
Thanks for all your help, Serge.  Yes, this is exactly what I needed.  I 
did a hook script that just did env>file to see what was set, and I did 
not think $LXC_ROOTFS_MOUNT looked at all like what I wanted.

The bind mounts are working great now.

Thanks again,

g