<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<tt>On 12/03/2015 07:25 AM, Fajar A. Nugraha wrote:</tt><tt><br>
</tt>
<blockquote
cite="mid:CAG1y0scNdS9sy_4=8cX3rL+OXYxqPaoqSR-L9a-rPbFsOe5fJg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><tt>On Thu, Dec 3, 2015 at 9:27 PM,
Peter Steele </tt><tt><span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:pwsteele@gmail.com" target="_blank">pwsteele@gmail.com</a>></span></tt><tt>
wrote:</tt><tt><br>
</tt>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><tt><span class="">
On 12/02/2015 08:47 PM, Fajar A. Nugraha wrote:</span></tt></div>
</blockquote>
<tt><br>
</tt>
<div><tt>centos template -> download lost of packages
(i.e. RPM) one by one using yum, and then install it</tt></div>
<div><tt><br>
</tt></div>
<div><tt>download template -> download one big tar.xz
file (plus several small config files), and then extract
it. MUCH faster, and works for unpriv containers as well
(not sure what the current state of unpriv containers on
centos though)</tt></div>
<div><tt><br>
</tt></div>
<div><tt>However I was actually more concerned about the
fact that the templates are maintained separately, so
there could be some difference in the resulting
container/config. The download template works (I've
tested it), while (based on your previous output) the
centos template doesn't provide the desired /dev
entries.</tt></div>
<div><tt><br>
</tt></div>
</div>
</div>
</div>
</blockquote>
<tt>I just did a test using the download approach and it worked
nicely, obviously much cleaner than downloading the rpms
individually. The containers created from the two approaches seem
to be identical, as far as a cursory glance is concerned, with
identical config files. </tt><tt><br>
</tt>
<blockquote
cite="mid:CAG1y0scNdS9sy_4=8cX3rL+OXYxqPaoqSR-L9a-rPbFsOe5fJg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><tt><br>
</tt></div>
<div><tt>Which is why I suggested the download template. </tt></div>
<div><tt><br>
</tt></div>
<div><tt>I also tested using the resulting config with
rootfs replaced by a "native" centos7 install (to be
exact, a disk clone of minimal centos7 install on
virtualbox), still result in the desired /dev entries
(i.e. minimal /dev entries, no /dev/snd).</tt></div>
<div><tt><br>
</tt></div>
</div>
</div>
</div>
</blockquote>
<tt>I can't really use the downloaded template for our rootfs, as I
explained earlier. We already have a process that generates a
custom centos tar ball with the specific set of packages that we
need in our containers. Our tarball includes other third party
packages as well, such as supervisord and ctdb. I've used the
downloaded template's config file to create a custom config for
our containers. The container specific portion of the config looks
something like this:</tt><tt><br>
</tt><tt><br>
</tt><tt>lxc.utsname = pws-vm-03</tt><tt><br>
</tt><tt>lxc.rootfs = /hf/cs/vm-03/rootfs</tt><tt><br>
</tt><tt>lxc.network.veth.pair = vm-03</tt><tt><br>
</tt><tt>lxc.network.hwaddr = fe:d6:e8:dc:c8:db</tt><tt><br>
</tt><tt>lxc.rootfs = /hf/cs/vm-03/rootfs</tt><tt><br>
</tt><tt>lxc.cgroup.memory.limit_in_bytes = 1073741824</tt><tt><br>
</tt><tt>lxc.cgroup.memory.memsw.limit_in_bytes = 2147483648</tt><tt><br>
</tt><tt>lxc.include = /var/lib/hf/lxc.conf</tt><tt><br>
<br>
</tt><tt>and the settings that are common to all containers
(lxc.conf) include the following:</tt><tt><br>
</tt><tt><br>
</tt><tt>lxc.autodev = 1</tt><tt><br>
</tt><tt>lxc.devttydir = lxc</tt><tt><br>
</tt><tt>lxc.tty = 4</tt><tt><br>
</tt><tt>lxc.pts = 1024</tt><tt><br>
</tt><tt>lxc.kmsg = 0</tt><tt><br>
</tt><tt>lxc.arch = x86_64</tt><tt><br>
</tt><tt><br>
</tt><tt># Networking defaults</tt><tt><br>
</tt><tt>lxc.network.type = veth</tt><tt><br>
</tt><tt>lxc.network.flags = up</tt><tt><br>
</tt><tt>lxc.network.link = br0</tt><tt><br>
</tt><tt><br>
</tt><tt># Remove capabilities we don't want in containers</tt><tt><br>
</tt><tt>lxc.cap.drop = mac_admin mac_override sys_time sys_module</tt><tt><br>
</tt><tt><br>
</tt><tt># Set the pivot directory</tt><tt><br>
</tt><tt>lxc.pivotdir = lxc_putold</tt><tt><br>
</tt><tt><br>
</tt><tt># Control Group devices: all denied except those
white-listed</tt><tt><br>
</tt><tt>lxc.cgroup.devices.deny = a</tt><tt><br>
</tt><tt>## Allow any mknod (but not reading/writing the node)</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c *:* m</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = b *:* m</tt><tt><br>
</tt><tt>## /dev/null</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 1:3 rwm</tt><tt><br>
</tt><tt>## /dev/zero</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 1:5 rwm</tt><tt><br>
</tt><tt>## /dev/full</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 1:7 rwm</tt><tt><br>
</tt><tt>## /dev/tty</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 5:0 rwm</tt><tt><br>
</tt><tt>## /dev/random</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 1:8 rwm</tt><tt><br>
</tt><tt>## /dev/urandom</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 1:9 rwm</tt><tt><br>
</tt><tt>## /dev/tty[1-4] ptys and lxc console</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 136:* rwm</tt><tt><br>
</tt><tt>## /dev/ptmx pty master</tt><tt><br>
</tt><tt>lxc.cgroup.devices.allow = c 5:2 rwm</tt><tt><br>
</tt><tt><br>
</tt><tt># Setup the default mounts</tt><tt><br>
</tt><tt>lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed</tt><tt><br>
</tt><tt>lxc.mount.entry = /sys/fs/fuse/connections
sys/fs/fuse/connections none bind,optional 0 0</tt><tt><br>
</tt><tt><br>
</tt><tt>As you can see this was largely pulled from
centos.common.conf and common.conf.</tt><tt> I assume something
isn't quite right since I see more entries under /dev than I do
when I'm running under libvirt, using the same custom tarball.
I'll be satisfied with this for now though as long as the extra
entries aren't causing issues.<br>
</tt>
<blockquote
cite="mid:CAG1y0scNdS9sy_4=8cX3rL+OXYxqPaoqSR-L9a-rPbFsOe5fJg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><tt><br>
</tt></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><tt> There *are*
other issues (our software isn't running properly),
but I think the major container issues have been
resolved.</tt></div>
</blockquote>
<div><tt><br>
</tt></div>
<div><tt>Which is?</tt></div>
<div><tt><br>
</tt></div>
</div>
</div>
</div>
</blockquote>
<tt>Well, mainly the udev issue, plus the fact that the containers
booted *really* slowly.</tt><tt><br>
</tt>
<blockquote
cite="mid:CAG1y0scNdS9sy_4=8cX3rL+OXYxqPaoqSR-L9a-rPbFsOe5fJg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><tt> </tt></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><tt> I changed a few
things, including the version of LXC that I'm using,
so it's hard to say what the culprit was with regards
to this udev issue.</tt><tt><span class=""><br>
<br>
</span></tt></div>
</blockquote>
<div><tt><br>
</tt></div>
<div><tt><br>
</tt></div>
<div><tt>IIRC systemd containers are only supported on
lxc-1.1.x, so upgrading lxc probably has a big part in
that.</tt></div>
</div>
</div>
</div>
</blockquote>
<tt>Yeah, things definitely started working better after I upgraded.</tt><tt><br>
<br>
</tt><br>
</body>
</html>