[lxc-users] Migrating from LXC to LXD

[Michael Eager]

Hi All --

I recently upgraded a server from CentOS 7 to CentOS 8 and wanted to
upgrade from LXC to LXD.  I have several LXC containers on the CentOS 8
host and (almost) everything appears to work correctly.  All containers
have a static IP address and use the host bridge.  All can be accessed
either from the host or from other systems on the network.

When I create and launch an LXD CentOS 8 container, it does not have
network connectivity and I'm at a loss to understand why.  There seem to
be multiple How-To's online, not all of which agree with each other,
perhaps because some are for LXD 2.x and very few are LXD 3.x.
I'm following this one: 
https://blog.simos.info/how-to-make-your-lxd-container-get-ip-addresses-from-your-lan/ 


This should create containers which get an IP address from the DHCP
server on the LAN.  I'll worry about static IP later.

$ lxd --version
3.18

$ ip route show default 0.0.0.0/0
default via 192.168.20.1 dev br0 proto static metric 425

$ lxc network list
+---------+----------+---------+-------------+---------+
|  NAME   |   TYPE   | MANAGED | DESCRIPTION | USED BY |
+---------+----------+---------+-------------+---------+
| br0     | bridge   | NO      |             | 5       |
+---------+----------+---------+-------------+---------+
| enp10s0 | physical | NO      |             | 0       |
+---------+----------+---------+-------------+---------+
| enp4s0  | physical | NO      |             | 0       |
+---------+----------+---------+-------------+---------+
| lxdbr0  | bridge   | YES     |             | 0       |
+---------+----------+---------+-------------+---------+

I'm not using lxdbr0 which was created by "lxd init".
br0 is a bridge to enp4s0 connected to the LAN.

$ lxc profile show lanprofile
config: {}
description: Default LXD profile
devices:
   eth0:
     name: eth0
     nictype: macvlan
     parent: br0
     type: nic
   root:
     path: /
     pool: lxd
     type: disk
name: lanprofile
used_by:
...

I can launch an Ubuntu image as shown in the How-To and it works:

$ lxc launch -p lanprofile ubuntu:16.04 net1
Creating net1
Starting net1
$ lxc exec net1 ip route
default via 192.168.20.1 dev eth0
192.168.20.0/24 dev eth0  proto kernel  scope link  src 192.168.20.134

When I try to do the same with a CentOS 8 image, it doesn't work.  Also, 
I'm a bit confused by the difference in the command line syntax used in
the "lxc launch" for ubuntu and the one's which I find for CentOS.

$ lxc launch -p lanprofile images:centos/8/amd64 net2
Creating net2
Starting net2

$ lxc exec net2 ip route
<nothing>

Also, the Ubuntu and CentOS containers are different; the CentOS
container doesn't have eth0 listed under IPs.

$ lxc info net1
Name: net1
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/01/22 01:11 UTC
Status: Running
Type: persistent
Profiles: lanprofile
Pid: 20412
Ips:
   eth0: inet    192.168.20.134  mac5c9226a5
   eth0: inet6   fd03:5cb0:9e91:0:216:3eff:fe2d:62ad     mac5c9226a5
   eth0: inet6   fe80::216:3eff:fe2d:62ad        mac5c9226a5
   lo:   inet    127.0.0.1
   lo:   inet6   ::1
Resources:
   Processes: 23
   CPU usage:
     CPU usage (in seconds): 3
   Memory usage:
     Memory (current): 77.71MB
     Memory (peak): 232.05MB
   Network usage:
     eth0:
       Bytes received: 581.39kB
       Bytes sent: 16.40kB
       Packets received: 623
       Packets sent: 214
     lo:
       Bytes received: 0B
       Bytes sent: 0B
       Packets received: 0
       Packets sent: 0

$ lxc info net2
Name: net2
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/01/22 01:19 UTC
Status: Running
Type: persistent
Profiles: lanprofile
Pid: 22074
Ips:
   lo:   inet    127.0.0.1
   lo:   inet6   ::1
Resources:
   Processes: 13
   CPU usage:
     CPU usage (in seconds): 1
   Memory usage:
     Memory (current): 48.82MB
     Memory (peak): 56.66MB
   Network usage:
     eth0:
       Bytes received: 0B
       Bytes sent: 176B
       Packets received: 0
       Packets sent: 2
     lo:
       Bytes received: 0B
       Bytes sent: 0B
       Packets received: 0
       Packets sent: 0

The CentOS container (net2) does have an eth0, but it is down:

$ lxc exec net2 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
group default qlen 1000
     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
     inet 127.0.0.1/8 scope host lo
        valid_lft forever preferred_lft forever
     inet6 ::1/128 scope host
        valid_lft forever preferred_lft forever
23: eth0 at if4: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN 
group default qlen 1000
     link/ether 00:16:3e:dc:68:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0

Trying to start eth0 gets the following error:

$ lxc exec net2 ifup eth0
Error: Connection activation failed: No suitable device found for this 
connection (device lo not available because device is strictly unmanaged).

I thought that maybe I was configuring something wrong with br0, but
the Ubuntu container works.

Can someone give me a clue about what is going on?

What is "lxc launch" or "lxc start" doing for the Ubuntu
container that isn't happening with the CentOS container?

Does "br0" have to be "managed" and what does that mean?