[lxc-users] LXCFS installation effects

Stéphane Graber stgraber at ubuntu.com
Tue Jun 5 22:26:47 UTC 2018 

On Tue, Jun 05, 2018 at 03:19:08PM -0700, Martín Fernández wrote:
> Awesome!
> 
> You mean overriding my current `lxc.include` or adding an additional `lxc.include` ? Not sure if lxc supports multiple includes.


Adding a separate line with the second include, you can have as many
lxc.include in your config as you want.

> 
> Sorry for the delay!
> 
> Best,
> Martín
> 
> On Tue, Jun 05, 2018 at 4:38 PM "Stéphane Graber" < ">"Stéphane Graber" > wrote:
> 
> > 
> > 
> > 
> > Ah, that is missing a bit that I'd have expected common.conf to contain.
> > 
> > Can you try adding this to your container's config:
> > 
> > lxc.include = /usr/share/lxc/config/common.conf.d/00-lxcfs.conf
> > 
> > 
> > 
> > On Tue, Jun 05, 2018 at 12:29:39PM -0700, Martín Fernández wrote:
> > > Content of the omitted include:
> > >
> > > # Default pivot location
> > > lxc.pivotdir = lxc_putold
> > >
> > > # Default mount entries
> > > lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
> > > lxc.mount.entry = sysfs sys sysfs defaults 0 0
> > > lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
> > bind,optional 0 0
> > > lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional
> > 0 0
> > > lxc.mount.entry = /sys/kernel/security sys/kernel/security none
> > bind,optional 0 0
> > > lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
> > >
> > > # Default console settings
> > > lxc.devttydir = lxc
> > > lxc.tty = 4
> > > lxc.pts = 1024
> > >
> > > # Default capabilities
> > > lxc.cap.drop = sys_module mac_admin mac_override sys_time
> > >
> > > # When using LXC with apparmor, the container will be confined by
> > default.
> > > # If you wish for it to instead run unconfined, copy the following line
> > > # (uncommented) to the container's configuration file.
> > > #lxc.aa_profile = unconfined
> > >
> > > # To support container nesting on an Ubuntu host while retaining most of
> > 
> > > # apparmor's added security, use the following two lines instead.
> > > #lxc.aa_profile = lxc-container-default-with-nesting
> > > #lxc.mount.auto = cgroup:mixed
> > >
> > > # Uncomment the following line to autodetect squid-deb-proxy
> > configuration on the
> > > # host and forward it to the guest at start time.
> > > #lxc.hook.pre-start = /usr/share/lxc/hooks/squid-deb-proxy-client
> > >
> > > # If you wish to allow mounting block filesystems, then use the
> > following
> > > # line instead, and make sure to grant access to the block device and/or
> > loop
> > > # devices below in lxc.cgroup.devices.allow.
> > > #lxc.aa_profile = lxc-container-default-with-mounting
> > >
> > > # Default cgroup limits
> > > lxc.cgroup.devices.deny = a
> > > ## Allow any mknod (but not using the node)
> > > lxc.cgroup.devices.allow = c *:* m
> > > lxc.cgroup.devices.allow = b *:* m
> > > ## /dev/null and zero
> > > lxc.cgroup.devices.allow = c 1:3 rwm
> > > lxc.cgroup.devices.allow = c 1:5 rwm
> > > ## consoles
> > > lxc.cgroup.devices.allow = c 5:0 rwm
> > > lxc.cgroup.devices.allow = c 5:1 rwm
> > > ## /dev/{,u}random
> > > lxc.cgroup.devices.allow = c 1:8 rwm
> > > lxc.cgroup.devices.allow = c 1:9 rwm
> > > ## /dev/pts/*
> > > lxc.cgroup.devices.allow = c 5:2 rwm
> > > lxc.cgroup.devices.allow = c 136:* rwm
> > > ## rtc
> > > lxc.cgroup.devices.allow = c 254:0 rm
> > > ## fuse
> > > lxc.cgroup.devices.allow = c 10:229 rwm
> > > ## tun
> > > lxc.cgroup.devices.allow = c 10:200 rwm
> > > ## full
> > > lxc.cgroup.devices.allow = c 1:7 rwm
> > > ## hpet
> > > lxc.cgroup.devices.allow = c 10:228 rwm
> > > ## kvm
> > > lxc.cgroup.devices.allow = c 10:232 rwm
> > > ## To use loop devices, copy the following line to the container's
> > > ## configuration file (uncommented).
> > > #lxc.cgroup.devices.allow = b 7:* rwm
> > >
> > > # Blacklist some syscalls which are not safe in privileged
> > > # containers
> > > lxc.seccomp = /usr/share/lxc/config/common.seccomp
> > >
> > > Martín
> > >
> > > On Tue, Jun 05, 2018 at 4:28 PM fmartin91 at gmail.com < fmartin91 at gmail.com
> > > wrote:
> > >
> > > >
> > > >
> > > > I omitted this line that is probably important!
> > > >
> > > >
> > > > # Common configuration
> > > > lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> > > >
> > > >
> > > > Best,
> > > > Martín
> > > >
> > > > On Tue, Jun 05, 2018 at 4:24 PM "Stéphane Graber" < ">"Stéphane
> > Graber" >
> > > > wrote:
> > > >
> > > >
> > > >>
> > > >>
> > > >> Is that all you have or is there some lines before that?
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Jun 05, 2018 at 12:16:48PM -0700, Martín Fernández wrote:
> > > >> > Stéphane,
> > > >> >
> > > >> > I think this could be the issue in the configuration:
> > > >> >
> > > >> > ```
> > > >> > # Container specific configuration
> > > >> > lxc.rootfs = /dev/Main/app1-dev
> > > >> > lxc.mount = /var/lib/lxc/app1-dev/fstab
> > > >> > lxc.utsname = app1-dev
> > > >> > lxc.arch = amd64
> > > >> > ```
> > > >> >
> > > >> > Best,
> > > >> > Martín
> > > >> >
> > > >> > On Tue, Jun 05, 2018 at 4:14 PM "Stéphane Graber" < ">"Stéphane
> > Graber"
> > > >> > wrote:
> > > >> >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > /var/lib/lxc/ /config for the container you're testing things
> > with.
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Tue, Jun 05, 2018 at 12:09:52PM -0700, Martín Fernández wrote:
> > 
> > > >> > > > Stéphane,
> > > >> > > >
> > > >> > > > Not sure what configuration file you are talking about.
> > > >> Configuration
> > > >> > > file under /etc/lxc/default.conf looks like this:
> > > >> > > >
> > > >> > > > ```
> > > >> > > > lxc.network.type = veth
> > > >> > > > lxc.network.link ( http://lxc.network.link ) ( http://lxc.network.link
> > ) ( http://lxc.network.link
> > > >> ) = br0
> > > >> > > > lxc.network.flags = up
> > > >> > > > lxc.network.hwaddr = XXXXX
> > > >> > > > ```
> > > >> > > >
> > > >> > > > Any lxc-* command that I could use to introspect the containers
> > and
> > > >> get
> > > >> > > more information to troubleshoot ?
> > > >> > > >
> > > >> > > > Thanks again!
> > > >> > > >
> > > >> > > > Best,
> > > >> > > > Martín
> > > >> > > >
> > > >> > > > On Tue, Jun 05, 2018 at 4:05 PM "Stéphane Graber" < ">"Stéphane
> > 
> > > >> Graber"
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > What's your container's config like?
> > > >> > > > >
> > > >> > > > > I wonder if it's somehow missing the include (usually
> > indirect
> > > >> through
> > > >> > >
> > > >> > > > > common.conf) that's needed for the lxcfs hook.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Tue, Jun 05, 2018 at 11:57:39AM -0700, Martín Fernández
> > wrote:
> > > >> > > > > > Stéphane,
> > > >> > > > > >
> > > >> > > > > > `grep lxcfs /proc/1/mountinfo` doesn’t return any output.
> > > >> > > > > >
> > > >> > > > > > On the other hand,  /var/lib/lxcfs/ shows `cgroup` and
> > `proc`
> > > >> > > folders
> > > >> > > > > with multiple files.
> > > >> > > > > >
> > > >> > > > > > Best,
> > > >> > > > > > Martín
> > > >> > > > > >
> > > >> > > > > > On Tue, Jun 05, 2018 at 3:54 PM "Stéphane Graber" <
> > ">"Stéphane
> > > >> > > Graber"
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > What do you see if you run "grep lxcfs /proc/1/mountinfo"
> > 
> > > >> inside
> > > >> > > the
> > > >> > > > > > > container?
> > > >> > > > > > >
> > > >> > > > > > > And do you see the lxcfs tree at /var/lib/lxcfs/ on the
> > host?
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > On Tue, Jun 05, 2018 at 11:50:51AM -0700, Martín
> > Fernández
> > > >> wrote:
> > > >> > > > > > > > Stéphane,
> > > >> > > > > > > >
> > > >> > > > > > > > I just got time to do my work on lxcfs. Installed lxcfs
> > 
> > > >> running
> > > >> > > on a
> > > >> > > > >
> > > >> > > > > > > Ubuntu 14.04 box, installed version is 2.0.8. 
> > > >> > > > > > > >
> > > >> > > > > > > > I restarted one of our containers and “I think” I see
> > wrong
> > > >> > > output
> > > >> > > > > when
> > > >> > > > > > > running `free` for example. 
> > > >> > > > > > > >
> > > >> > > > > > > > lxc-info shows 1GB of memory usage and `free` shows
> > 24GB of
> > > >> > > memory
> > > >> > > > > usage
> > > >> > > > > > > which is the same as the host memory usage. Anything I
> > could
> > > >> be
> > > >> > > > > missing ?
> > > >> > > > > > > >
> > > >> > > > > > > > Short version of the process done would be:
> > > >> > > > > > > >
> > > >> > > > > > > > - apt-get install lxcfs
> > > >> > > > > > > > - sudo init 0 (in container)
> > > >> > > > > > > > - lxc-start -n container-name -d 
> > > >> > > > > > > >
> > > >> > > > > > > > Best,
> > > >> > > > > > > > Martín
> > > >> > > > > > > >
> > > >> > > > > > > > On Thu, May 31, 2018 at 12:39 AM "Stéphane Graber" <
> > > >> ">"Stéphane
> > > >> > >
> > > >> > > > > Graber"
> > > >> > > > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > On Wed, May 30, 2018 at 07:16:04PM -0700, Martín
> > Fernández
> > > >>
> > > >> > > wrote:
> > > >> > > > > > > > > > Stéphane,
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Thank you very much for the quick reply!
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > What are you are saying is pretty awesome! That
> > would
> > > >> make
> > > >> > > it
> > > >> > > > > super
> > > >> > > > > > > easy
> > > >> > > > > > > > > to start using it. Is there any constraint in terms
> > of
> > > >> what
> > > >> > > > > versions
> > > >> > > > > > > of
> > > >> > > > > > > > > LXC are supported ? I can run LXCFS with LXC 1.0.10 ?
> > 
> > > >> > > > > > > > >
> > > >> > > > > > > > > 1.0.10 should be fine though we certainly don't have
> > all
> > > >> that
> > > >> > > many
> > > >> > > > >
> > > >> > > > > > > users
> > > >> > > > > > > > > of that release now that it's two LTS ago :)
> > > >> > > > > > > > >
> > > >> > > > > > > > > In any case, it'll be safe to install LXCFS, then
> > create a
> > > >>
> > > >> > > test
> > > >> > > > > > > > > container, confirm it behaves and if it does then
> > start
> > > >> > > restarting
> > > >> > > > >
> > > >> > > > > > > your
> > > >> > > > > > > > > existing containers, if it doesn't, let us know and
> > we'll
> > > >> try
> > > >> > > to
> > > >> > > > > > > figure
> > > >> > > > > > > > > out why.
> > > >> > > > > > > > >
> > > >> > > > > > > > > > In order to understand a little bit more about how
> > LXCFS
> > > >>
> > > >> > > works,
> > > >> > > > > does
> > > >> > > > > > >
> > > >> > > > > > > > > LXCFS hook into LXC starting process and mount
> > /proc/*
> > > >> files ?
> > > >> > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > That's correct, LXCFS when installed will create a
> > tree at
> > > >>
> > > >> > > > > > > > > /var/lib/lxcfs those files then get bind-mounted on
> > top of
> > > >> the
> > > >> > >
> > > >> > > > > > > > > containers /proc/* files through a LXC startup hook.
> > > >> > > > > > > > >
> > > >> > > > > > > > > > Thank you very much again!
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Best,
> > > >> > > > > > > > > > Martín
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > On Wed, May 30, 2018 at 10:52 PM "Stéphane Graber"
> > <
> > > >> > > ">"Stéphane
> > > >> > > > >
> > > >> > > > > > > Graber"
> > > >> > > > > > > > > > wrote:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > _______________________________________________
> > > >> > > > > > > > > > > lxc-users mailing list
> > > >> > > > > > > > > > > lxc-users at lists.linuxcontainers.org
> > > >> > > > > > > > > > > http://lists.linuxcontainers.org/listinfo/lxc-users
> > 
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > On Wed, May 30, 2018 at 05:08:59PM -0700, Martín
> > > >> Fernández
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > > > > > > > > Hello,
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > We are using LXC to virtualize containers in
> > > >> multiple of
> > > >> > > our
> > > >> > > > >
> > > >> > > > > > > hosts.
> > > >> > > > > > > > > We
> > > >> > > > > > > > > > > have been running with LXC for a while now. 
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > We started adding monitoring tools to our
> > systems
> > > >> and
> > > >> > > found
> > > >> > > > > the
> > > >> > > > > > > > > known
> > > >> > > > > > > > > > > issue that LXC containers show the host
> > information on
> > > >>
> > > >> > > > > > > /proc/meminfo
> > > >> > > > > > > > > and
> > > >> > > > > > > > > > > /proc/cpuinfo.  
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > I found that LXCFS solves the problems
> > mentioned
> > > >> above.
> > > >> > > What
> > > >> > > > >
> > > >> > > > > > > would
> > > >> > > > > > > > > be
> > > >> > > > > > > > > > > required to setup LXCFS in my hosts ? Would I
> > need to
> > > >> > > reboot
> > > >> > > > > all
> > > >> > > > > > > the
> > > >> > > > > > > > > > > containers ? Do I need to restore my containers
> > > >> filesystem
> > > >> > > ?
> > > >> > > > > Is
> > > >> > > > > > > there
> > > >> > > > > > > > > any
> > > >> > > > > > > > > > > guide/documentation around it ?
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > Thanks before hand!
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > Best,
> > > >> > > > > > > > > > > > Martín
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Hey there,
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > You should just need to install lxcfs and then
> > any
> > > >> > > container
> > > >> > > > > you
> > > >> > > > > > > start
> > > >> > > > > > > > >
> > > >> > > > > > > > > > > or restart will be using it. There's no way to
> > set it
> > > >> up
> > > >> > > > > against a
> > > >> > > > > > >
> > > >> > > > > > > > > > > running container, but there's also no need to
> > restart
> > > >> all
> > > >> > >
> > > >> > > > > your
> > > >> > > > > > > > > > > containers immediately, you can slowly roll it
> > out if
> > > >> that
> > > >> > >
> > > >> > > > > helps.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > And no changes needed to the containers, it gets
> > setup
> > > >>
> > > >> > > > > > > automatically
> > > >> > > > > > > > > > > through a lxc hook when the container starts.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > --
> > > >> > > > > > > > > > > Stéphane Graber
> > > >> > > > > > > > > > > Ubuntu developer
> > > >> > > > > > > > > > > http://www.ubuntu.com
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > --
> > > >> > > > > > > > > Stéphane Graber
> > > >> > > > > > > > > Ubuntu developer
> > > >> > > > > > > > > http://www.ubuntu.com
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > --
> > > >> > > > > > > Stéphane Graber
> > > >> > > > > > > Ubuntu developer
> > > >> > > > > > > http://www.ubuntu.com
> > > >> > > > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > Stéphane Graber
> > > >> > > > > Ubuntu developer
> > > >> > > > > http://www.ubuntu.com
> > > >> > > > >
> > > >> > >
> > > >> > > --
> > > >> > > Stéphane Graber
> > > >> > > Ubuntu developer
> > > >> > > http://www.ubuntu.com
> > > >> > >
> > > >>
> > > >> --
> > > >> Stéphane Graber
> > > >> Ubuntu developer
> > > >> http://www.ubuntu.com
> > > >>
> > > >
> > > >
> > > >
> > 
> > --
> > Stéphane Graber
> > Ubuntu developer
> > http://www.ubuntu.com
> >

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20180605/f10bcece/attachment.sig>

More information about the lxc-users mailing list