[lxc-users] LXCFS installation effects

Martín Fernández fmartin91 at gmail.com
Tue Jun 5 19:29:39 UTC 2018 

Content of the omitted include:

# Default pivot location
lxc.pivotdir = lxc_putold

# Default mount entries
lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry = sysfs sys sysfs defaults 0 0
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none bind,optional 0 0
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0

# Default console settings
lxc.devttydir = lxc
lxc.tty = 4
lxc.pts = 1024

# Default capabilities
lxc.cap.drop = sys_module mac_admin mac_override sys_time

# When using LXC with apparmor, the container will be confined by default.
# If you wish for it to instead run unconfined, copy the following line
# (uncommented) to the container's configuration file.
#lxc.aa_profile = unconfined

# To support container nesting on an Ubuntu host while retaining most of
# apparmor's added security, use the following two lines instead.
#lxc.aa_profile = lxc-container-default-with-nesting
#lxc.mount.auto = cgroup:mixed

# Uncomment the following line to autodetect squid-deb-proxy configuration on the
# host and forward it to the guest at start time.
#lxc.hook.pre-start = /usr/share/lxc/hooks/squid-deb-proxy-client

# If you wish to allow mounting block filesystems, then use the following
# line instead, and make sure to grant access to the block device and/or loop
# devices below in lxc.cgroup.devices.allow.
#lxc.aa_profile = lxc-container-default-with-mounting

# Default cgroup limits
lxc.cgroup.devices.deny = a
## Allow any mknod (but not using the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
## /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
## consoles
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 5:1 rwm
## /dev/{,u}random
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 1:9 rwm
## /dev/pts/*
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 136:* rwm
## rtc
lxc.cgroup.devices.allow = c 254:0 rm
## fuse
lxc.cgroup.devices.allow = c 10:229 rwm
## tun
lxc.cgroup.devices.allow = c 10:200 rwm
## full
lxc.cgroup.devices.allow = c 1:7 rwm
## hpet
lxc.cgroup.devices.allow = c 10:228 rwm
## kvm
lxc.cgroup.devices.allow = c 10:232 rwm
## To use loop devices, copy the following line to the container's
## configuration file (uncommented).
#lxc.cgroup.devices.allow = b 7:* rwm

# Blacklist some syscalls which are not safe in privileged
# containers
lxc.seccomp = /usr/share/lxc/config/common.seccomp

Martín

On Tue, Jun 05, 2018 at 4:28 PM fmartin91 at gmail.com < fmartin91 at gmail.com > wrote:

> 
> 
> I omitted this line that is probably important!
> 
> 
> # Common configuration
> lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> 
> 
> Best,
> Martín
> 
> On Tue, Jun 05, 2018 at 4:24 PM "Stéphane Graber" < ">"Stéphane Graber" >
> wrote:
> 
> 
>> 
>> 
>> Is that all you have or is there some lines before that?
>> 
>> 
>> 
>> On Tue, Jun 05, 2018 at 12:16:48PM -0700, Martín Fernández wrote:
>> > Stéphane,
>> >
>> > I think this could be the issue in the configuration:
>> >
>> > ```
>> > # Container specific configuration
>> > lxc.rootfs = /dev/Main/app1-dev
>> > lxc.mount = /var/lib/lxc/app1-dev/fstab
>> > lxc.utsname = app1-dev
>> > lxc.arch = amd64
>> > ```
>> >
>> > Best,
>> > Martín
>> >
>> > On Tue, Jun 05, 2018 at 4:14 PM "Stéphane Graber" < ">"Stéphane Graber"
>> > wrote:
>> >
>> > >
>> > >
>> > >
>> > > /var/lib/lxc/ /config for the container you're testing things with.
>> > >
>> > >
>> > >
>> > >
>> > > On Tue, Jun 05, 2018 at 12:09:52PM -0700, Martín Fernández wrote:
>> > > > Stéphane,
>> > > >
>> > > > Not sure what configuration file you are talking about.
>> Configuration
>> > > file under /etc/lxc/default.conf looks like this:
>> > > >
>> > > > ```
>> > > > lxc.network.type = veth
>> > > > lxc.network.link ( http://lxc.network.link ) ( http://lxc.network.link
>> ) = br0
>> > > > lxc.network.flags = up
>> > > > lxc.network.hwaddr = XXXXX
>> > > > ```
>> > > >
>> > > > Any lxc-* command that I could use to introspect the containers and
>> get
>> > > more information to troubleshoot ?
>> > > >
>> > > > Thanks again!
>> > > >
>> > > > Best,
>> > > > Martín
>> > > >
>> > > > On Tue, Jun 05, 2018 at 4:05 PM "Stéphane Graber" < ">"Stéphane
>> Graber"
>> > > > wrote:
>> > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > What's your container's config like?
>> > > > >
>> > > > > I wonder if it's somehow missing the include (usually indirect
>> through
>> > >
>> > > > > common.conf) that's needed for the lxcfs hook.
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Tue, Jun 05, 2018 at 11:57:39AM -0700, Martín Fernández wrote:
>> > > > > > Stéphane,
>> > > > > >
>> > > > > > `grep lxcfs /proc/1/mountinfo` doesn’t return any output.
>> > > > > >
>> > > > > > On the other hand,  /var/lib/lxcfs/ shows `cgroup` and `proc`
>> > > folders
>> > > > > with multiple files.
>> > > > > >
>> > > > > > Best,
>> > > > > > Martín
>> > > > > >
>> > > > > > On Tue, Jun 05, 2018 at 3:54 PM "Stéphane Graber" < ">"Stéphane
>> > > Graber"
>> > > > > > wrote:
>> > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > What do you see if you run "grep lxcfs /proc/1/mountinfo"
>> inside
>> > > the
>> > > > > > > container?
>> > > > > > >
>> > > > > > > And do you see the lxcfs tree at /var/lib/lxcfs/ on the host?
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Tue, Jun 05, 2018 at 11:50:51AM -0700, Martín Fernández
>> wrote:
>> > > > > > > > Stéphane,
>> > > > > > > >
>> > > > > > > > I just got time to do my work on lxcfs. Installed lxcfs
>> running
>> > > on a
>> > > > >
>> > > > > > > Ubuntu 14.04 box, installed version is 2.0.8. 
>> > > > > > > >
>> > > > > > > > I restarted one of our containers and “I think” I see wrong
>> > > output
>> > > > > when
>> > > > > > > running `free` for example. 
>> > > > > > > >
>> > > > > > > > lxc-info shows 1GB of memory usage and `free` shows 24GB of
>> > > memory
>> > > > > usage
>> > > > > > > which is the same as the host memory usage. Anything I could
>> be
>> > > > > missing ?
>> > > > > > > >
>> > > > > > > > Short version of the process done would be:
>> > > > > > > >
>> > > > > > > > - apt-get install lxcfs
>> > > > > > > > - sudo init 0 (in container)
>> > > > > > > > - lxc-start -n container-name -d 
>> > > > > > > >
>> > > > > > > > Best,
>> > > > > > > > Martín
>> > > > > > > >
>> > > > > > > > On Thu, May 31, 2018 at 12:39 AM "Stéphane Graber" <
>> ">"Stéphane
>> > >
>> > > > > Graber"
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Wed, May 30, 2018 at 07:16:04PM -0700, Martín Fernández
>> 
>> > > wrote:
>> > > > > > > > > > Stéphane,
>> > > > > > > > > >
>> > > > > > > > > > Thank you very much for the quick reply!
>> > > > > > > > > >
>> > > > > > > > > > What are you are saying is pretty awesome! That would
>> make
>> > > it
>> > > > > super
>> > > > > > > easy
>> > > > > > > > > to start using it. Is there any constraint in terms of
>> what
>> > > > > versions
>> > > > > > > of
>> > > > > > > > > LXC are supported ? I can run LXCFS with LXC 1.0.10 ? 
>> > > > > > > > >
>> > > > > > > > > 1.0.10 should be fine though we certainly don't have all
>> that
>> > > many
>> > > > >
>> > > > > > > users
>> > > > > > > > > of that release now that it's two LTS ago :)
>> > > > > > > > >
>> > > > > > > > > In any case, it'll be safe to install LXCFS, then create a
>> 
>> > > test
>> > > > > > > > > container, confirm it behaves and if it does then start
>> > > restarting
>> > > > >
>> > > > > > > your
>> > > > > > > > > existing containers, if it doesn't, let us know and we'll
>> try
>> > > to
>> > > > > > > figure
>> > > > > > > > > out why.
>> > > > > > > > >
>> > > > > > > > > > In order to understand a little bit more about how LXCFS
>> 
>> > > works,
>> > > > > does
>> > > > > > >
>> > > > > > > > > LXCFS hook into LXC starting process and mount /proc/*
>> files ?
>> > >
>> > > > > > > > >
>> > > > > > > > > That's correct, LXCFS when installed will create a tree at
>> 
>> > > > > > > > > /var/lib/lxcfs those files then get bind-mounted on top of
>> the
>> > >
>> > > > > > > > > containers /proc/* files through a LXC startup hook.
>> > > > > > > > >
>> > > > > > > > > > Thank you very much again!
>> > > > > > > > > >
>> > > > > > > > > > Best,
>> > > > > > > > > > Martín
>> > > > > > > > > >
>> > > > > > > > > > On Wed, May 30, 2018 at 10:52 PM "Stéphane Graber" <
>> > > ">"Stéphane
>> > > > >
>> > > > > > > Graber"
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > _______________________________________________
>> > > > > > > > > > > lxc-users mailing list
>> > > > > > > > > > > lxc-users at lists.linuxcontainers.org
>> > > > > > > > > > > http://lists.linuxcontainers.org/listinfo/lxc-users
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Wed, May 30, 2018 at 05:08:59PM -0700, Martín
>> Fernández
>> > >
>> > > > > wrote:
>> > > > > > > > > > > > Hello,
>> > > > > > > > > > > >
>> > > > > > > > > > > > We are using LXC to virtualize containers in
>> multiple of
>> > > our
>> > > > >
>> > > > > > > hosts.
>> > > > > > > > > We
>> > > > > > > > > > > have been running with LXC for a while now. 
>> > > > > > > > > > > >
>> > > > > > > > > > > > We started adding monitoring tools to our systems
>> and
>> > > found
>> > > > > the
>> > > > > > > > > known
>> > > > > > > > > > > issue that LXC containers show the host information on
>> 
>> > > > > > > /proc/meminfo
>> > > > > > > > > and
>> > > > > > > > > > > /proc/cpuinfo.  
>> > > > > > > > > > > >
>> > > > > > > > > > > > I found that LXCFS solves the problems mentioned
>> above.
>> > > What
>> > > > >
>> > > > > > > would
>> > > > > > > > > be
>> > > > > > > > > > > required to setup LXCFS in my hosts ? Would I need to
>> > > reboot
>> > > > > all
>> > > > > > > the
>> > > > > > > > > > > containers ? Do I need to restore my containers
>> filesystem
>> > > ?
>> > > > > Is
>> > > > > > > there
>> > > > > > > > > any
>> > > > > > > > > > > guide/documentation around it ?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thanks before hand!
>> > > > > > > > > > > >
>> > > > > > > > > > > > Best,
>> > > > > > > > > > > > Martín
>> > > > > > > > > > >
>> > > > > > > > > > > Hey there,
>> > > > > > > > > > >
>> > > > > > > > > > > You should just need to install lxcfs and then any
>> > > container
>> > > > > you
>> > > > > > > start
>> > > > > > > > >
>> > > > > > > > > > > or restart will be using it. There's no way to set it
>> up
>> > > > > against a
>> > > > > > >
>> > > > > > > > > > > running container, but there's also no need to restart
>> all
>> > >
>> > > > > your
>> > > > > > > > > > > containers immediately, you can slowly roll it out if
>> that
>> > >
>> > > > > helps.
>> > > > > > > > > > >
>> > > > > > > > > > > And no changes needed to the containers, it gets setup
>> 
>> > > > > > > automatically
>> > > > > > > > > > > through a lxc hook when the container starts.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > --
>> > > > > > > > > > > Stéphane Graber
>> > > > > > > > > > > Ubuntu developer
>> > > > > > > > > > > http://www.ubuntu.com
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Stéphane Graber
>> > > > > > > > > Ubuntu developer
>> > > > > > > > > http://www.ubuntu.com
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Stéphane Graber
>> > > > > > > Ubuntu developer
>> > > > > > > http://www.ubuntu.com
>> > > > > > >
>> > > > >
>> > > > > --
>> > > > > Stéphane Graber
>> > > > > Ubuntu developer
>> > > > > http://www.ubuntu.com
>> > > > >
>> > >
>> > > --
>> > > Stéphane Graber
>> > > Ubuntu developer
>> > > http://www.ubuntu.com
>> > >
>> 
>> --
>> Stéphane Graber
>> Ubuntu developer
>> http://www.ubuntu.com
>> 
> 
> 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20180605/16f98ca0/attachment.html>

More information about the lxc-users mailing list