[lxc-users] Unable to join cluster

[Joshua Schaeffer]

Hey all, I'm trying to build a cluster on Ubuntu 18.04 with lxd 3.0.3. I was able to bootstrap the first node without any issues, but when I try to add a second node it just hangs and never returns the  terminal prompt. Here is my bootstrapped node:

lxcuser at blllxc02:~$ lxc cluster list
+----------+----------------------------------------------+----------+--------+-------------------+
|   NAME   |                     URL                      | DATABASE | STATE  |      MESSAGE      |
+----------+----------------------------------------------+----------+--------+-------------------+
| blllxc02 | https://blllxc02-mgmt.harmonywave.cloud:8443 | YES      | ONLINE | fully operational |
+----------+----------------------------------------------+----------+--------+-------------------+

And here is the second node I am trying to add:

lxcuser at blllxc01:~$ sudo lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: yes
What name should be used to identify this node in the cluster? [default=blllxc01]:
What IP address or DNS name should be used to reach this node? [default=fe80::6a1c:a2ff:fe13:1ec6]: blllxc01-mgmt.harmonywave.cloud
Are you joining an existing cluster? (yes/no) [default=no]: yes
IP address or FQDN of an existing cluster node: blllxc02-mgmt.harmonywave.cloud
Cluster fingerprint: 20b51145761f3444278317331feeded8492c263920889f5dccd83772da0c42cf
You can validate this fingerpring by running "lxc info" locally on an existing node.
Is this the correct fingerprint? (yes/no) [default=no]: yes
Cluster trust password:
All existing data is lost when joining a cluster, continue? (yes/no) [default=no] yes
Choose the local disk or dataset for storage pool "btrfspool1" (empty for loop disk): /dev/sdj
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:

^C
lxcuser at blllxc01:~$ lxc cluster list
Error: LXD server isn't part of a cluster

After the last question from lxd init my terminal never returns. I've left it like this overnight with no change. This is all I'm seeing in the logs as well from the time I run lxd init to when I abort the process:

Logs from the node trying to be added:
t=2020-03-18T20:17:07-0600 lvl=info msg="Creating BTRFS storage pool \"btrfspool1\""
t=2020-03-18T20:17:08-0600 lvl=warn msg="Failed to detect UUID by looking at /dev/disk/by-uuid"
t=2020-03-18T20:17:08-0600 lvl=info msg="Created BTRFS storage pool \"btrfspool1\""
t=2020-03-19T02:12:27-0600 lvl=info msg="Updating images"
t=2020-03-19T02:12:27-0600 lvl=info msg="Done updating images"
t=2020-03-19T08:12:27-0600 lvl=info msg="Updating images"
t=2020-03-19T08:12:27-0600 lvl=info msg="Done updating images"

Logs from the bootstrapped node:
t=2020-03-18T17:05:58-0600 lvl=info msg="Initializing global database"
t=2020-03-18T17:06:02-0600 lvl=warn msg="Raft: Heartbeat timeout from \"\" reached, starting election"
t=2020-03-18T17:06:03-0600 lvl=info msg="Initializing storage pools"
t=2020-03-18T17:06:03-0600 lvl=info msg="Initializing networks"
t=2020-03-18T17:06:03-0600 lvl=info msg="Pruning leftover image files"
t=2020-03-18T17:06:03-0600 lvl=info msg="Done pruning leftover image files"
t=2020-03-18T17:06:03-0600 lvl=info msg="Loading daemon configuration"
t=2020-03-18T17:06:03-0600 lvl=info msg="Pruning expired images"
t=2020-03-18T17:06:03-0600 lvl=info msg="Done pruning expired images"
t=2020-03-18T17:06:03-0600 lvl=info msg="Expiring log files"
t=2020-03-18T17:06:03-0600 lvl=info msg="Done expiring log files"
t=2020-03-18T17:06:03-0600 lvl=info msg="Updating images"
t=2020-03-18T17:06:03-0600 lvl=info msg="Done updating images"
t=2020-03-18T17:06:03-0600 lvl=info msg="Updating instance types"
t=2020-03-18T17:06:03-0600 lvl=info msg="Done updating instance types"
t=2020-03-18T23:06:03-0600 lvl=info msg="Updating images"
t=2020-03-18T23:06:03-0600 lvl=info msg="Done updating images"
t=2020-03-19T05:06:03-0600 lvl=info msg="Updating images"
t=2020-03-19T05:06:03-0600 lvl=info msg="Done updating images"
t=2020-03-19T11:06:03-0600 lvl=info msg="Updating images"
t=2020-03-19T11:06:03-0600 lvl=info msg="Done updating images"

Any idea where I can get more information about what is going on to successfully add the node to the cluster?

-- 
Thanks,
Joshua Schaeffer

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20200319/b965a91f/attachment.htm>