[lxc-users] unprivileged container with zfs backing

Wed Feb 4 11:59:08 UTC 2015

On 04/02/2015 11:54, Fajar A. Nugraha wrote:
> On Wed, Feb 4, 2015 at 6:01 PM, Fajar A. Nugraha <list at fajar.net
> <mailto:list at fajar.net>> wrote:
> 
>     On Wed, Feb 4, 2015 at 5:46 PM, Adam Gold <awg1 at gmx.com
>     <mailto:awg1 at gmx.com>> wrote:
> 
> 
>           If so, what's the best way to manually create a template in
>         unprivileged mode to ensure all the subuids and subgids are assigned
>         correctly.
> 
> 
>     Create it using any normal method known to work. It should be OK
>     when .local/share/lxc is on the same filesystem as $HOME, right?
>     After that, copy it manually to your template dataset
>     (zfs/lxc/containers/template ?) using "rsync -avP" or whatever tool
>     of your choice. When you clone the template to a new directory,
>     don't forget to change these settings manually in the new container
>     config:
> 
> 
> 
> This is what I just tested. Environment:
> - there's a system container called "template"
> - user "user" is allowed to create its own container (including setting
> /etc/subuid)
> - I want to copy the system container to user container
> 
> Current permissions:
> # grep user /etc/subuid
> user:100000:65537
> 
> # grep user /etc/subgid
> user:100000:65537
> 
> # ls -la /var/lib/lxc/template/
> total 6
> drwxr-xr-x  3 root root   4 Jul 10  2014 .
> drwxr-xr-x  4 root root   4 Jan 22 19:26 ..
> -rw-r--r--  1 root root 691 Jul 10  2014 config
> drwxr-xr-x 21 root root  21 Feb  4 18:32 rootfs
> 
> # ls -la /home/user/.local/share/lxc/template/
> total 6
> drwxr-xr-x  3 user   user     4 Feb  4 18:39 .
> drwxr-xr-x  3 user   user     4 Feb  4 18:37 ..
> drwxr-xr-x 21 100000 100000  21 Feb  4 18:32 rootfs
> 
> # df -h /home/user/.local/share/lxc/template/
> Filesystem               Size  Used Avail Use% Mounted on
> rpool/lxc/user/template   46G  164M   46G   1%
> /home/user/.local/share/lxc/template
> 
> Note that /home/user/.local/share/lxc/template/rootfs is EMPTY. The zfs
> datasets, owner, and permission were created/set by root.
> I can then run this command to copy system's rootfs to users's rootfs,
> with the correct permissions:
> 
> # tar -C /var/lib/lxc/template/rootfs -cf - . | su - user -c
> "lxc-usernsexec -- tar -C /home/user/.local/share/lxc/template/rootfs
> -xf - --exclude ./dev/*"
> 
> WARN: could not reopen tty: No such file or directory
> tar: ./dev/log: socket ignored
> 
> Ignore the warning messages. Next step is to create
> /home/user/.local/share/lxc/template/config, like so:
> ###
> # Template used to create this container:
> /usr/share/lxc/templates/lxc-download
> # Parameters passed to the template: -d ubuntu -r trusty -a amd64
> # For additional config options, please look at lxc.conf(5)
> 
> # Distribution configuration
> lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
> lxc.arch = x86_64
> 
> # Container specific configuration
> lxc.id_map = u 0 100000 65536
> lxc.id_map = g 0 100000 65536
> lxc.rootfs = /home/user/.local/share/lxc/template/rootfs
> lxc.utsname = template
> 
> # Network configuration
> lxc.network.type = veth
> lxc.network.veth.pair = veth-u-te-0
> lxc.network.flags = up
> lxc.network.link = br0
> lxc.network.hwaddr = 00:16:3E:3A:53:E7
> ###
> 
> Note that I use br0, where the default bridge created by lxc is lxcbr0,
> so you need to make sure it's correct.
> 
> When creating another container for the same user "user", I can simply
> use zfs clone rpool/lxc/user/template. If I wanted to create container
> for another user (e.g. "user2"), then I need to repeat the "tar" method
> above as a different user (e.g. "| su - user2 -c ...")
>  
> 
> 
>         Also, for the additional containers that I get from zfs cloning,
>         will
>         they be recognised by 'lxc-* -n' commands?
> 
> 
>     AFAIK most lxc commands simply reads whatever is under the
>     directory, and doesn't really care what fs they are on.
> 
> 
> 
> After doing the above steps I can do this:
> 
> $ id
> uid=1000(user) gid=1000(user) groups=1000(user),4(adm),6(disk),27(sudo)
> 
> $ lxc-ls -f
> NAME      STATE    IPV4  IPV6  GROUPS  AUTOSTART  
> ------------------------------------------------
> template  STOPPED  -     -     -       NO         
> 
> $ lxc-start -d -n template
> 
> $ lxc-ls -f
> NAME      STATE    IPV4             IPV6  GROUPS  AUTOSTART  
> -----------------------------------------------------------
> template  RUNNING  192.168.124.104  -     -       NO         
> 
> $ lxc-attach -n template id
> uid=0(root) gid=0(root) groups=0(root)
> 
> $ lxc-attach -n template ip ad li eth0
> 25: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
> state UP group default qlen 1000
>     link/ether 00:16:3e:3a:53:e7 brd ff:ff:ff:ff:ff:ff
>     inet 192.168.124.104/24 <http://192.168.124.104/24> brd
> 192.168.124.255 scope global eth0
>        valid_lft forever preferred_lft forever
>     inet6 fe80::216:3eff:fe3a:53e7/64 scope link 
>        valid_lft forever preferred_lft forever
> 
> $ lxc-stop -n template
> 
> $ lxc-ls -f
> NAME      STATE    IPV4  IPV6  GROUPS  AUTOSTART  
> ------------------------------------------------
> template  STOPPED  -     -     -       NO         
> 
> -- 
> Fajar
> 

Thank you so much for doing all of that.  I will attempt to follow your
approach and hopefully reproduce the results.

FYI, I just tried using btrfs and creating containers at the root of a
sub volume in unprivileged mode and that worked just fine.  I guess it's
not surprising that it may be harder with zfs.