[lxc-devel] [Not A Patch] [POC] Proof of concept code for using devtmpfs for autodev and more...

[Michael H. Warfield]

On Fri, 2013-11-01 at 17:25 -0500, Serge Hallyn wrote: 
> Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > On Fri, 2013-11-01 at 16:30 -0500, Serge Hallyn wrote: 
> > > Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > > > On Fri, 2013-11-01 at 15:03 -0500, Serge Hallyn wrote: 
> > > > > Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > > > > > The only place that's being used is in creating a symlink...
> > > > > > 
> > > > > > /dev/.lxc/$name -> /dev/.lxc/$pathhash
> > > > > > 
> > > > > > I use it for the same reason you wanted the extra bind mounts to
> > > > > > $lxcpath/$lxcname.dev.  In your case, you wanted to see the dev mappings
> > > > > 
> > > > > Oh - gotcha.  Well in that case I'd say just create your own unique
> > > > > $name.$index.  that should be enough info.
> > > > 
> > > > > Oh now unprivileged container creation of course will not be able
> > > > > to do this as I won't be able to create /dev/.lxc/anything as uid
> > > > > 1000.
> > > > 
> > > > Oh, we're going to have to look into that then.  We're doing other
> > > > privileged operations like the bind mounts...  Hmmm...  It may have to
> > 
> > > bind mounts are ok.  we can do this in a private mntns.  That's how
> > > I currently get around our inability to mknod in a userns - I
> > > bind mount devices from the host into the container's /dev.
> > 
> > Ok...  How are you handling the creation of objects under $lxc_path
> > then?  Obviously, I haven't been paying much attention to the unpriv
> > user angle of things here.  Is it like many of the other virt systems
> > where the user needs to be part of a particular group?  Could we do
> > something similar?

> No.  I mkdir /home/serge/lxcbase and do

> 	lxc-create -t tarball -n b1 -P /home/serge/lxcbase -f /home/serge/lxc.conf -- -T /home/serge/ubuntu.tgz

> (tarball is a special template that just extracts the given tarball.
> I'm working on a patch to not need to do that, but I've been very
> distracted by other issues)

> So the key is the "-P" which specifies that the container lives in a
> directory which I own.

Ah!  Ok...  So, once again, the key to that is that -P switch.  I hadn't
realized that but it makes a lot of sense.  That's certainly moving my
thought processes along.

The /dev/.lxc issue is going to be tricky and we may have to fall back
to tmpfs in that case.  We could still mount tmpfs to a directory
$lxc_path/$container/.lxc.dev (or $lxc_path/$container/lxc.dev without
the dot or $lxc_path/$container/dev without the lxc. at all) and then
bind mount it across and retain the local ability to act on the
container's /dev file system just like the static fs case.  It means
inverting or convoluting a bit of my logic in mount_autodev() but that
shouldn't be difficult and I see where it needs to happen.

Case 1: root w/ devtmpfs on /dev
Create /dev/.lxc/${hash}
Bind mount /dev/.lxc/${hash} to /$lxcpath/$container/.lxc.dev
Bind mount /dev/.lxc/${hash} to /$rootfs/dev

Case 2: root w/ no devtmpfs on /dev
Mount tmpfs on /dev/.lxc if needed
Recurse to #1

Case 3: No permissions to create in /dev/.lxc (non-root case)
Mount tmfs on /$lxcpath/$container/.lxc.dev
Bind mount /$lxcpath/$container/.lxc.dev to /$rootfs/dev

In all three cases we end up with /$lxcpath/$container/.lxc.dev where
the host can manipulate the containers /dev system via udev rules or
whatever.  We can handle both the devtmpfs and non-devtmpfs cases and it
handles the priv user and non-priv cases.

We could drop case #2 and immediately drop to case #3 if devtmpfs is not
mounted on /dev.  The difference is really very minor and may make a lot
of sense.  In principle, I guess, we could even do the same thing with
case one and just use devtmpfs but I just feel like I would like to
stick with case 1 where we could to keep things open like hard links for
shared devices or /dev symlinks and the host udev rules.  Just thoughts.

Which would you prefer for that subdirectory (it's just a string)...

/$lxcpath/$container/.lxc.dev
/$lxcpath/$container/lxc.dev
/$lxcpath/$container/dev

My personal preference would be to just use the third option unless you
have an objection.

Regards
Mike

> Ok, so really to get this to work I first need to:
> 
> 1.  cat > /home/serge/lxc.conf << EOF
> lxc.network.type = veth
> lxc.network.link = lxcbr0
> lxc.network.flags = up
> lxc.id_map = u 0 100000 10000
> lxc.id_map = g 0 100000 10000
> EOF
> 
> 2. sudo usermod -v 100000-200000 -w 100000-200000 serge
> 
> And then if I want to actually start the container (since I specified a
> nic) I need to
> 
> 3. cat >> /etc/lxc/lxc-usernet  << EOF
> serge veth lxcbr0 2
> EOF
> 

-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20131101/4cf3223a/attachment.pgp>