[lxc-users] LXCFS installation effects

Tue Jun 5 19:38:12 UTC 2018

Ah, that is missing a bit that I'd have expected common.conf to contain.

Can you try adding this to your container's config:

    lxc.include = /usr/share/lxc/config/common.conf.d/00-lxcfs.conf

On Tue, Jun 05, 2018 at 12:29:39PM -0700, Martín Fernández wrote:
> Content of the omitted include:
> 
> # Default pivot location
> lxc.pivotdir = lxc_putold
> 
> # Default mount entries
> lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
> lxc.mount.entry = sysfs sys sysfs defaults 0 0
> lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none bind,optional 0 0
> lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
> lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
> lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
> 
> # Default console settings
> lxc.devttydir = lxc
> lxc.tty = 4
> lxc.pts = 1024
> 
> # Default capabilities
> lxc.cap.drop = sys_module mac_admin mac_override sys_time
> 
> # When using LXC with apparmor, the container will be confined by default.
> # If you wish for it to instead run unconfined, copy the following line
> # (uncommented) to the container's configuration file.
> #lxc.aa_profile = unconfined
> 
> # To support container nesting on an Ubuntu host while retaining most of
> # apparmor's added security, use the following two lines instead.
> #lxc.aa_profile = lxc-container-default-with-nesting
> #lxc.mount.auto = cgroup:mixed
> 
> # Uncomment the following line to autodetect squid-deb-proxy configuration on the
> # host and forward it to the guest at start time.
> #lxc.hook.pre-start = /usr/share/lxc/hooks/squid-deb-proxy-client
> 
> # If you wish to allow mounting block filesystems, then use the following
> # line instead, and make sure to grant access to the block device and/or loop
> # devices below in lxc.cgroup.devices.allow.
> #lxc.aa_profile = lxc-container-default-with-mounting
> 
> # Default cgroup limits
> lxc.cgroup.devices.deny = a
> ## Allow any mknod (but not using the node)
> lxc.cgroup.devices.allow = c *:* m
> lxc.cgroup.devices.allow = b *:* m
> ## /dev/null and zero
> lxc.cgroup.devices.allow = c 1:3 rwm
> lxc.cgroup.devices.allow = c 1:5 rwm
> ## consoles
> lxc.cgroup.devices.allow = c 5:0 rwm
> lxc.cgroup.devices.allow = c 5:1 rwm
> ## /dev/{,u}random
> lxc.cgroup.devices.allow = c 1:8 rwm
> lxc.cgroup.devices.allow = c 1:9 rwm
> ## /dev/pts/*
> lxc.cgroup.devices.allow = c 5:2 rwm
> lxc.cgroup.devices.allow = c 136:* rwm
> ## rtc
> lxc.cgroup.devices.allow = c 254:0 rm
> ## fuse
> lxc.cgroup.devices.allow = c 10:229 rwm
> ## tun
> lxc.cgroup.devices.allow = c 10:200 rwm
> ## full
> lxc.cgroup.devices.allow = c 1:7 rwm
> ## hpet
> lxc.cgroup.devices.allow = c 10:228 rwm
> ## kvm
> lxc.cgroup.devices.allow = c 10:232 rwm
> ## To use loop devices, copy the following line to the container's
> ## configuration file (uncommented).
> #lxc.cgroup.devices.allow = b 7:* rwm
> 
> # Blacklist some syscalls which are not safe in privileged
> # containers
> lxc.seccomp = /usr/share/lxc/config/common.seccomp
> 
> Martín
> 
> On Tue, Jun 05, 2018 at 4:28 PM fmartin91 at gmail.com < fmartin91 at gmail.com > wrote:
> 
> > 
> > 
> > I omitted this line that is probably important!
> > 
> > 
> > # Common configuration
> > lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> > 
> > 
> > Best,
> > Martín
> > 
> > On Tue, Jun 05, 2018 at 4:24 PM "Stéphane Graber" < ">"Stéphane Graber" >
> > wrote:
> > 
> > 
> >> 
> >> 
> >> Is that all you have or is there some lines before that?
> >> 
> >> 
> >> 
> >> On Tue, Jun 05, 2018 at 12:16:48PM -0700, Martín Fernández wrote:
> >> > Stéphane,
> >> >
> >> > I think this could be the issue in the configuration:
> >> >
> >> > ```
> >> > # Container specific configuration
> >> > lxc.rootfs = /dev/Main/app1-dev
> >> > lxc.mount = /var/lib/lxc/app1-dev/fstab
> >> > lxc.utsname = app1-dev
> >> > lxc.arch = amd64
> >> > ```
> >> >
> >> > Best,
> >> > Martín
> >> >
> >> > On Tue, Jun 05, 2018 at 4:14 PM "Stéphane Graber" < ">"Stéphane Graber"
> >> > wrote:
> >> >
> >> > >
> >> > >
> >> > >
> >> > > /var/lib/lxc/ /config for the container you're testing things with.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Tue, Jun 05, 2018 at 12:09:52PM -0700, Martín Fernández wrote:
> >> > > > Stéphane,
> >> > > >
> >> > > > Not sure what configuration file you are talking about.
> >> Configuration
> >> > > file under /etc/lxc/default.conf looks like this:
> >> > > >
> >> > > > ```
> >> > > > lxc.network.type = veth
> >> > > > lxc.network.link ( http://lxc.network.link ) ( http://lxc.network.link
> >> ) = br0
> >> > > > lxc.network.flags = up
> >> > > > lxc.network.hwaddr = XXXXX
> >> > > > ```
> >> > > >
> >> > > > Any lxc-* command that I could use to introspect the containers and
> >> get
> >> > > more information to troubleshoot ?
> >> > > >
> >> > > > Thanks again!
> >> > > >
> >> > > > Best,
> >> > > > Martín
> >> > > >
> >> > > > On Tue, Jun 05, 2018 at 4:05 PM "Stéphane Graber" < ">"Stéphane
> >> Graber"
> >> > > > wrote:
> >> > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > What's your container's config like?
> >> > > > >
> >> > > > > I wonder if it's somehow missing the include (usually indirect
> >> through
> >> > >
> >> > > > > common.conf) that's needed for the lxcfs hook.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Tue, Jun 05, 2018 at 11:57:39AM -0700, Martín Fernández wrote:
> >> > > > > > Stéphane,
> >> > > > > >
> >> > > > > > `grep lxcfs /proc/1/mountinfo` doesn’t return any output.
> >> > > > > >
> >> > > > > > On the other hand,  /var/lib/lxcfs/ shows `cgroup` and `proc`
> >> > > folders
> >> > > > > with multiple files.
> >> > > > > >
> >> > > > > > Best,
> >> > > > > > Martín
> >> > > > > >
> >> > > > > > On Tue, Jun 05, 2018 at 3:54 PM "Stéphane Graber" < ">"Stéphane
> >> > > Graber"
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > What do you see if you run "grep lxcfs /proc/1/mountinfo"
> >> inside
> >> > > the
> >> > > > > > > container?
> >> > > > > > >
> >> > > > > > > And do you see the lxcfs tree at /var/lib/lxcfs/ on the host?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Tue, Jun 05, 2018 at 11:50:51AM -0700, Martín Fernández
> >> wrote:
> >> > > > > > > > Stéphane,
> >> > > > > > > >
> >> > > > > > > > I just got time to do my work on lxcfs. Installed lxcfs
> >> running
> >> > > on a
> >> > > > >
> >> > > > > > > Ubuntu 14.04 box, installed version is 2.0.8. 
> >> > > > > > > >
> >> > > > > > > > I restarted one of our containers and “I think” I see wrong
> >> > > output
> >> > > > > when
> >> > > > > > > running `free` for example. 
> >> > > > > > > >
> >> > > > > > > > lxc-info shows 1GB of memory usage and `free` shows 24GB of
> >> > > memory
> >> > > > > usage
> >> > > > > > > which is the same as the host memory usage. Anything I could
> >> be
> >> > > > > missing ?
> >> > > > > > > >
> >> > > > > > > > Short version of the process done would be:
> >> > > > > > > >
> >> > > > > > > > - apt-get install lxcfs
> >> > > > > > > > - sudo init 0 (in container)
> >> > > > > > > > - lxc-start -n container-name -d 
> >> > > > > > > >
> >> > > > > > > > Best,
> >> > > > > > > > Martín
> >> > > > > > > >
> >> > > > > > > > On Thu, May 31, 2018 at 12:39 AM "Stéphane Graber" <
> >> ">"Stéphane
> >> > >
> >> > > > > Graber"
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > On Wed, May 30, 2018 at 07:16:04PM -0700, Martín Fernández
> >> 
> >> > > wrote:
> >> > > > > > > > > > Stéphane,
> >> > > > > > > > > >
> >> > > > > > > > > > Thank you very much for the quick reply!
> >> > > > > > > > > >
> >> > > > > > > > > > What are you are saying is pretty awesome! That would
> >> make
> >> > > it
> >> > > > > super
> >> > > > > > > easy
> >> > > > > > > > > to start using it. Is there any constraint in terms of
> >> what
> >> > > > > versions
> >> > > > > > > of
> >> > > > > > > > > LXC are supported ? I can run LXCFS with LXC 1.0.10 ? 
> >> > > > > > > > >
> >> > > > > > > > > 1.0.10 should be fine though we certainly don't have all
> >> that
> >> > > many
> >> > > > >
> >> > > > > > > users
> >> > > > > > > > > of that release now that it's two LTS ago :)
> >> > > > > > > > >
> >> > > > > > > > > In any case, it'll be safe to install LXCFS, then create a
> >> 
> >> > > test
> >> > > > > > > > > container, confirm it behaves and if it does then start
> >> > > restarting
> >> > > > >
> >> > > > > > > your
> >> > > > > > > > > existing containers, if it doesn't, let us know and we'll
> >> try
> >> > > to
> >> > > > > > > figure
> >> > > > > > > > > out why.
> >> > > > > > > > >
> >> > > > > > > > > > In order to understand a little bit more about how LXCFS
> >> 
> >> > > works,
> >> > > > > does
> >> > > > > > >
> >> > > > > > > > > LXCFS hook into LXC starting process and mount /proc/*
> >> files ?
> >> > >
> >> > > > > > > > >
> >> > > > > > > > > That's correct, LXCFS when installed will create a tree at
> >> 
> >> > > > > > > > > /var/lib/lxcfs those files then get bind-mounted on top of
> >> the
> >> > >
> >> > > > > > > > > containers /proc/* files through a LXC startup hook.
> >> > > > > > > > >
> >> > > > > > > > > > Thank you very much again!
> >> > > > > > > > > >
> >> > > > > > > > > > Best,
> >> > > > > > > > > > Martín
> >> > > > > > > > > >
> >> > > > > > > > > > On Wed, May 30, 2018 at 10:52 PM "Stéphane Graber" <
> >> > > ">"Stéphane
> >> > > > >
> >> > > > > > > Graber"
> >> > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > _______________________________________________
> >> > > > > > > > > > > lxc-users mailing list
> >> > > > > > > > > > > lxc-users at lists.linuxcontainers.org
> >> > > > > > > > > > > http://lists.linuxcontainers.org/listinfo/lxc-users
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Wed, May 30, 2018 at 05:08:59PM -0700, Martín
> >> Fernández
> >> > >
> >> > > > > wrote:
> >> > > > > > > > > > > > Hello,
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > We are using LXC to virtualize containers in
> >> multiple of
> >> > > our
> >> > > > >
> >> > > > > > > hosts.
> >> > > > > > > > > We
> >> > > > > > > > > > > have been running with LXC for a while now. 
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > We started adding monitoring tools to our systems
> >> and
> >> > > found
> >> > > > > the
> >> > > > > > > > > known
> >> > > > > > > > > > > issue that LXC containers show the host information on
> >> 
> >> > > > > > > /proc/meminfo
> >> > > > > > > > > and
> >> > > > > > > > > > > /proc/cpuinfo.  
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > I found that LXCFS solves the problems mentioned
> >> above.
> >> > > What
> >> > > > >
> >> > > > > > > would
> >> > > > > > > > > be
> >> > > > > > > > > > > required to setup LXCFS in my hosts ? Would I need to
> >> > > reboot
> >> > > > > all
> >> > > > > > > the
> >> > > > > > > > > > > containers ? Do I need to restore my containers
> >> filesystem
> >> > > ?
> >> > > > > Is
> >> > > > > > > there
> >> > > > > > > > > any
> >> > > > > > > > > > > guide/documentation around it ?
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Thanks before hand!
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Best,
> >> > > > > > > > > > > > Martín
> >> > > > > > > > > > >
> >> > > > > > > > > > > Hey there,
> >> > > > > > > > > > >
> >> > > > > > > > > > > You should just need to install lxcfs and then any
> >> > > container
> >> > > > > you
> >> > > > > > > start
> >> > > > > > > > >
> >> > > > > > > > > > > or restart will be using it. There's no way to set it
> >> up
> >> > > > > against a
> >> > > > > > >
> >> > > > > > > > > > > running container, but there's also no need to restart
> >> all
> >> > >
> >> > > > > your
> >> > > > > > > > > > > containers immediately, you can slowly roll it out if
> >> that
> >> > >
> >> > > > > helps.
> >> > > > > > > > > > >
> >> > > > > > > > > > > And no changes needed to the containers, it gets setup
> >> 
> >> > > > > > > automatically
> >> > > > > > > > > > > through a lxc hook when the container starts.
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > --
> >> > > > > > > > > > > Stéphane Graber
> >> > > > > > > > > > > Ubuntu developer
> >> > > > > > > > > > > http://www.ubuntu.com
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > Stéphane Graber
> >> > > > > > > > > Ubuntu developer
> >> > > > > > > > > http://www.ubuntu.com
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > Stéphane Graber
> >> > > > > > > Ubuntu developer
> >> > > > > > > http://www.ubuntu.com
> >> > > > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Stéphane Graber
> >> > > > > Ubuntu developer
> >> > > > > http://www.ubuntu.com
> >> > > > >
> >> > >
> >> > > --
> >> > > Stéphane Graber
> >> > > Ubuntu developer
> >> > > http://www.ubuntu.com
> >> > >
> >> 
> >> --
> >> Stéphane Graber
> >> Ubuntu developer
> >> http://www.ubuntu.com
> >> 
> > 
> > 
> >

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20180605/8d3d9f3b/attachment-0001.sig>