[lxc-users] Converting from libvirt lxc

[Peter Steele]

On 12/03/2015 07:25 AM, Fajar A. Nugraha wrote:
> On Thu, Dec 3, 2015 at 9:27 PM, Peter Steele <pwsteele at gmail.com 
> <mailto:pwsteele at gmail.com>>wrote:
>
>     On 12/02/2015 08:47 PM, Fajar A. Nugraha wrote:
>
>
> centos template -> download lost of packages (i.e. RPM) one by one 
> using yum, and then install it
>
> download template -> download one big tar.xz file (plus several small 
> config files), and then extract it. MUCH faster, and works for unpriv 
> containers as well (not sure what the current state of unpriv 
> containers on centos though)
>
> However I was actually more concerned about the fact that the 
> templates are maintained separately, so there could be some difference 
> in the resulting container/config. The download template works (I've 
> tested it), while (based on your previous output) the centos template 
> doesn't provide the desired /dev entries.
>
I just did a test using the download approach and it worked nicely, 
obviously much cleaner than downloading the rpms individually. The 
containers created from the two approaches seem to be identical, as far 
as a cursory glance is concerned, with identical config files.
>
> Which is why I suggested the download template.
>
> I also tested using the resulting config with rootfs replaced by a 
> "native" centos7 install (to be exact, a disk clone of minimal centos7 
> install on virtualbox), still result in the desired /dev entries (i.e. 
> minimal /dev entries, no /dev/snd).
>
I can't really use the downloaded template for our rootfs, as I 
explained earlier. We already have a process that generates a custom 
centos tar ball with the specific set of packages that we need in our 
containers. Our tarball includes other third party packages as well, 
such as supervisord and ctdb. I've used the downloaded template's config 
file to create a custom config for our containers. The container 
specific portion of the config looks something like this:

lxc.utsname = pws-vm-03
lxc.rootfs = /hf/cs/vm-03/rootfs
lxc.network.veth.pair = vm-03
lxc.network.hwaddr = fe:d6:e8:dc:c8:db
lxc.rootfs = /hf/cs/vm-03/rootfs
lxc.cgroup.memory.limit_in_bytes = 1073741824
lxc.cgroup.memory.memsw.limit_in_bytes = 2147483648
lxc.include = /var/lib/hf/lxc.conf

and the settings that are common to all containers (lxc.conf) include 
the following:

lxc.autodev = 1
lxc.devttydir = lxc
lxc.tty = 4
lxc.pts = 1024
lxc.kmsg = 0
lxc.arch = x86_64

# Networking defaults
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0

# Remove capabilities we don't want in containers
lxc.cap.drop = mac_admin mac_override sys_time sys_module

# Set the pivot directory
lxc.pivotdir = lxc_putold

# Control Group devices: all denied except those white-listed
lxc.cgroup.devices.deny = a
## Allow any mknod (but not reading/writing the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
## /dev/null
lxc.cgroup.devices.allow = c 1:3 rwm
## /dev/zero
lxc.cgroup.devices.allow = c 1:5 rwm
## /dev/full
lxc.cgroup.devices.allow = c 1:7 rwm
## /dev/tty
lxc.cgroup.devices.allow = c 5:0 rwm
## /dev/random
lxc.cgroup.devices.allow = c 1:8 rwm
## /dev/urandom
lxc.cgroup.devices.allow = c 1:9 rwm
## /dev/tty[1-4] ptys and lxc console
lxc.cgroup.devices.allow = c 136:* rwm
## /dev/ptmx pty master
lxc.cgroup.devices.allow = c 5:2 rwm

# Setup the default mounts
lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none 
bind,optional 0 0

As you can see this was largely pulled from centos.common.conf and 
common.conf.I assume something isn't quite right since I see more 
entries under /dev than I do when I'm running under libvirt, using the 
same custom tarball. I'll be satisfied with this for now though as long 
as the extra entries aren't causing issues.
>
>     There *are* other issues (our software isn't running properly),
>     but I think the major container issues have been resolved.
>
>
> Which is?
>
Well, mainly the udev issue, plus the fact that the containers booted 
*really* slowly.
>
>     I changed a few things, including the version of LXC that I'm
>     using, so it's hard to say what the culprit was with regards to
>     this udev issue.
>
>
>
> IIRC systemd containers are only supported on lxc-1.1.x, so upgrading 
> lxc probably has a big part in that.
Yeah, things definitely started working better after I upgraded.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20151203/f3846f31/attachment.html>