[Lxc-users] LVM in LXC

[Serge Hallyn]

Quoting Benjamin Kiessling (mittagessen at l.unchti.me):
> Hi,
> 
> > That's still doable, just a bit more work.  Take a look at
> > 
> > ls -l /dev/lxc
> > 
> > (or whatever is the vg you're looking at).  It has symlinks to the real
> > devices.  When you look at the link targets, you can find their maj:min.
> > For me,
> > 
> > serge at sergelap:~$ ls -l /dev/lxc
> > total 0
> > lrwxrwxrwx 1 root root 7 2011-05-13 17:26 build1 -> ../dm-1
> > lrwxrwxrwx 1 root root 7 2011-05-13 17:26 delme -> ../dm-4
> > lrwxrwxrwx 1 root root 7 2011-05-13 17:26 nattylvm -> ../dm-0
> > serge at sergelap:~$ ls -l /dev/dm-1
> > brw-rw---- 1 root disk 252, 1 2011-05-13 17:26 /dev/dm-1
> > 
> > So if I only wanted /dev/lxc/build1 to be available to container nattylvm,
> > then in it's config I would keep the existing lxc.cgroup.devices entries,
> > and add
> > 
> > lxc.cgroup.devices.allow = b 252:1 rwm
> > 
> > To actually give the container access to the vg so it can create LVM
> > devices, I'm afraid I don't know enough about how lvcreate to be sure.
> > 
> > But here's my guess (based on a quick read of strace -f lvcreate output):
> > 
> > Use a different physical partition for each container's pv, and give
> > the container full access to that partition.  Then pvscan/pvcreate
> > will have access to the full drive, and all metadata is on there.
> > vgscan/vgcreate and lvscan/lvcreate likewise I believe will then
> > be able to create vgs and lvs on that partition.
> 
> That's what I was basically trying to do (and doesn't work this way as far as I
> can see). Currently I'm granting access to specific /dev/dm-* files to the 
> container. For example:
> /dev/dm-2 is the "partition"/logical volume of vm0 with maj:min 252:2. So I
> set lxc.cgroup.devices.allow = b 252:2 rwm. In the container I create a 
> vg on /dev/dm-2 (works so far) with name vg-vm0. Then I create a logical volume
> on vg-vm0 in the container. This pseudo-fails as the container doesn't have
> the rights to create any /dev/dm-* (or else an container could just create /dev/dm-n
> and access data on other logical volumes). On the host system the corresponding
> /dev/dm-7 of the new container lv has been created and I grant access to create
> the device node to the container: lxc.cgroup.devices.allow = b 252:7 rwm. vm0 
> is now able to create the device node and access the new lv.
> So either users have to contact me each time they want to create a new logical
> volume in their vm (so I can enable device node access) or they can create arbitrary
> /dev/dm-* nodes and access data from other users.

Ah yeah.  Of course.

I wonder if there is a not-too-hacky way that we could prealloc certain
dm-N ranges to containers, and get those to be used at lvcreate.

-serge