[lxc-devel] Memory Resources

[Daniel Lezcano]

Krzysztof Taraszka wrote:
> 2009/8/24 Daniel Lezcano <daniel.lezcano at free.fr>
>
>   
>> Krzysztof Taraszka wrote:
>>
>>     
>>> 2009/8/24 Daniel Lezcano <daniel.lezcano at free.fr>
>>>
>>>
>>>
>>>       
>>>> Krzysztof Taraszka wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> 2009/8/24 Daniel Lezcano <dlezcano at fr.ibm.com>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Krzysztof Taraszka wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> 2009/8/24 Daniel Lezcano <daniel.lezcano at free.fr>
>>>>>>>
>>>>>>>  Krzysztof Taraszka wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>  2009/8/23 Daniel Lezcano <daniel.lezcano at free.fr>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> (...)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  With the lxc tools I did:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>    lxc-execute -n foo /bin/bash
>>>>>>>>>>    echo 268435456 > /cgroup/foo/memory.limit_in_bytes
>>>>>>>>>>    mount --bind /cgroup/foo/memory.meminfo /proc/meminfo
>>>>>>>>>>    for i in $(seq 1 100); do sleep 3600 & done
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> (...)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  :)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>  hmmm... I think that access to the cgroup inside container is very
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> risk
>>>>>>>>> because I am able to manage for example memory resources (what if I
>>>>>>>>> am
>>>>>>>>> not
>>>>>>>>> the host owner and... I can give me via non-secure mounted /cgroup
>>>>>>>>> (inside
>>>>>>>>> container) all available memory resources...).
>>>>>>>>> I think that the /proc/meminfo should be pass to the container in
>>>>>>>>> the
>>>>>>>>> other
>>>>>>>>> way, but this is the topic for the other thread.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  It is not a problem, I did it in this way because it's easy to test
>>>>>>>>> but
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> in
>>>>>>>> a real use case, the memory limit is setup by the lxc configuration
>>>>>>>> file
>>>>>>>> and
>>>>>>>> the cgroup directory will be no longer accessible from the container.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> So.. how there will be another method (more secure) for giving
>>>>>>> /proc/meminfo
>>>>>>> with limits to the container, right?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Same method. The lxc tools can be configured with a fstab to mount more
>>>>>> mount points, furthermore if memory.meminfo is available I will add the
>>>>>> code
>>>>>> to mount it automatically to /proc/meminfo in the lxc tools.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> Hmm... setup_fs() from lxc_init.c or another way?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> No, I was thinking in the setup_cgroup() function in conf.c.
>>>>
>>>> Something like:
>>>>
>>>> ...
>>>>
>>>> if (!access("/var/lib/lxc/mycontainer/nsgroup/memory.meminfo"), F_OK) {
>>>>  mount("/var/lib/lxc/mycontainer/nsgroup/memory.meminfo",
>>>> "/proc/meminfo",
>>>> MS_BIND, ...);
>>>> }
>>>>
>>>> ...
>>>>
>>>>
>>>> but a bit more clean :)
>>>>
>>>>
>>>>
>>>>         
>>> hmm... ok, got it, but don't know why does it won't work ;)
>>>
>>> @@ -999,12 +999,14 @@
>>>  static int setup_cgroup(const char *name)
>>>  {
>>>     char filename[MAXPATHLEN];
>>> +    char meminfofilename[MAXPATHLEN];
>>>     char line[MAXPATHLEN];
>>>     struct stat s;
>>>     int ret;
>>>
>>>     snprintf(filename, MAXPATHLEN, LXCPATH "/%s/cgroup", name);
>>> -
>>> +    snprintf(meminfofilename, MAXPATHLEN, LXCPATH
>>> "/%s/nsgroup/memory.meminfo", name);
>>> +
>>>     if (stat(filename, &s)) {
>>>         SYSERROR("failed to stat '%s'", filename);
>>>         return -1;
>>> @@ -1024,6 +1026,10 @@
>>>
>>>     INFO("cgroup has been setup");
>>>
>>> +    /* mount memory.meminfo as /proc/meminfo */
>>> +    if (!access(meminfofilename, F_OK)) {
>>> +        mount(meminfofilename, "/proc/meminfo", "none", MS_BIND, 0);
>>> +     }
>>>     return 0;
>>>  }
>>>
>>>
>>> hmm... any idea Daniel? :)
>>>
>>>
>>>       
>> Yep, can you check the return code of the mount call and return an error ?
>> if (mount(....)) {
>>   SYSERROR("failed to mount '%s' to '/proc/meminfo'", meminfofilename);
>>   return -1;
>> }
>> at least to verify if this does not fail.
>> and maybe add an INFO trace if the mount is successful saying
>> "/proc/meminfo" is setup with the cgroup.
>>
>> ps : you should launch the command with the "-l INFO" to see the message.
>>
>>     
>
>
>
>
> Hmmm....
> i think that I know where the problem might be:
>
> look here:
>
> lxc1:~# cat debin.log
>       lxc-start 1251109397.922 INFO     lxc_conf - tty's configured
>       lxc-start 1251109397.922 INFO     lxc_start - 'debian' is initialized
>       lxc-start 1251109397.974 INFO     lxc_conf - 'debian' hostname has
> been setup
>       lxc-start 1251109397.975 INFO     lxc_conf - network has been setup
>       lxc-start 1251109397.976 INFO     lxc_conf - cgroup has been setup
>       lxc-start 1251109397.976 INFO     lxc_conf - /proc/meminfo is setup
> with the cgroup
>       lxc-start 1251109397.976 INFO     lxc_conf - mount points have been
> setup
>       lxc-start 1251109397.976 INFO     lxc_conf - console '/dev/pts/1'
> mounted to '/usr/local/var/lib/lxc/debian/rootfs/dev/console'
>       lxc-start 1251109397.977 INFO     lxc_conf - 4 tty(s) has been setup
>       lxc-start 1251109397.977 INFO     lxc_conf - chrooted to
> '/usr/local/var/lib/lxc/debian/rootfs'
>       lxc-start 1251109397.977 INFO     lxc_conf - created new pts instance
>       lxc-start 1251109397.977 NOTICE   lxc_conf - 'debian' is setup.
>       lxc-start 1251109397.977 NOTICE   lxc_start - exec'ing '/sbin/init'
>       lxc-start 1251109397.978 NOTICE   lxc_start - '/sbin/init' started
> with pid '24339'
>
> i think that /proc/meminfo should be mounted after /proc . why? i think
> that, because mounting /proc may override /proc/meminfo
> Am I right? :)
>   
Ha ! haha ! arrgh ! no way ! You are right :/

In the case of application container, lxc mounts /proc but in the case 
of system container it is the system who do that so after the 
/proc/meminfo has been mounted.

Maybe we can look at modifying fs/proc/meminfo.c instead. Let me do a 
small patch for the kernel...