[lxc-users] Nova-lxd plugin installation
Martin Bobák
martin.bobak at savba.sk
Thu Sep 20 14:41:40 UTC 2018
Hi Alex,
you are right, I installed stable/queens version (I haven't been sure
about master branch compatibility) of nova-lxd via devstack using ZFS.
Meanwhile, I tried to create a new storage for LXD with different names
for LXD pool and ZFS pool, however, the problem was still there. But
when I created "None" storage with a following line
sudo lxc storage create None_pool zfs zfs.pool_name=None
The problem was solved, and I moved to another error -> "Host 'ubuntu'
is not mapped to any cell" (I will build OpenStack one again with the
nova cells service enabled, and I will write about the results. As I
wrote, I used local.conf from github repo. There isn't nova cell enabled),
Best,
Martin
On 19.09.2018 18:17, Alex Kavanagh wrote:
> Hi Martin
>
> Okay, some progress which is great!
>
> But, to help you further, a few more details:
>
> 1. Which Linux OS are you using?
>
I'm using Ubuntu 16.04.5 LTS.
> 1. Is the LXD snap installed? - if so, which version is it? "lxc
> --version" will give the answer.
>
When I realised that the plugin is expecting LXD installed by apt (I
came to this conclusion by
https://github.com/openstack/nova-lxd/search?q=%2Fvar%2Flib%2F&unscoped_q=%2Fvar%2Flib%2F),
I installed LXD/LXC 3.0.1 by "sudo apt install -t xenial-backports lxd lxc".
So is it possible to use a LXD snap installation also? Because, when I
used it, the devstack installation script didn't recognize the LXD
installation, and so "stack.sh" will install LXD version 2...
> 1. Have you configured a storage pool for LXD? (e.g. have you done
> "sudo lxd init" and created a default storage pool). What kind of
> storage pool is it? (e.g. dir, btrfs, zfs, etc.)
>
I am using a zfs storage pool. I configured it with "sudo lxd init".
However, the changing of its name doesn't work properly (maybe LXD snap
version fixed it) and so I created another zfs storage pool by LXC CLI,
however, it wasn't successful, until its name wasn't None.
> 1. It looks like your using master version of nova-lxd (i.e. via
> devstack and the plugin)? I think it's also going to use ZFS? If
> so, there's a bug in nova-lxd (that I will fix in the not too
> distant future) where unless the ZFS pool is the same string as
> the LXD pool for it, then nova-lxd can't find the storage pool.
>
> So, if you run the following commands:
>
> lxc storage list
> zpool list
>
default -> created by "lxd init" (also in previous cases)
stack -> created by "sudo lxc storage create stack zfs
zfs.pool_name=stack_pool"
None_pool -> created by LXC CLI also, just to see whether it will work,
or not.
sudo lxc storage list
+-----------+-------------+--------+----------------------------------+---------+
| NAME | DESCRIPTION | DRIVER | SOURCE | USED BY |
+-----------+-------------+--------+----------------------------------+---------+
| None_pool | | zfs | /var/lib/lxd/disks/None_pool.img |
0 |
+-----------+-------------+--------+----------------------------------+---------+
| default | | zfs | /var/lib/lxd/disks/default.img |
2 |
+-----------+-------------+--------+----------------------------------+---------+
| stack | | zfs | /var/lib/lxd/disks/stack.img |
0 |
+-----------+-------------+--------+----------------------------------+---------+
zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH
ALTROOT
None 14.9G 514K 14.9G - 0% 0% 1.00x ONLINE -
default 2.98G 307M 2.68G - 9% 10% 1.00x ONLINE -
stack_pool 14.9G 292K 14.9G - 0% 0% 1.00x ONLINE -
> That should show the current configuration.
>
> Hope that this helps.
> Cheers
> Alex.
>
> On Wed, Sep 19, 2018 at 2:48 PM, Martin Bobák <martin.bobak at savba.sk
> <mailto:martin.bobak at savba.sk>> wrote:
>
> Hi Alex,
>
> first off, thank you for your kind reply. I followed your advices.
> However, I still have a problem which looks like an incorrect
> setting up of a storage pool for lxd. I configured the
> /etc/nova-compute.conf as you suggested, but it didn't help.
>
> In my case those lines look like:
>
> [DEFAULT]
> compute_driver = nova_lxd.nova.virt.lxd.LXDDriver
>
> [lxd]
> allow_live_migration = True
> pool = lxd
>
> But the error is following (relevant parts from syslog --> it
> looks like lxd variables from the /etc/nova-compute.conf aren't
> delivered to the driver, or recognized by it, I tried to use
> different names for compute_driver e.g. lxd.LXDDriver,
> nova-lxd.nova.virt.lxd.(driver).LXDDriver -> however, it didn't help):
>
> Sep 19 09:15:08 localhost nova-conductor[1956]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-5cda2246-8087-4f49-b9b3-463d29fd7bf8 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mcompute_driver =
> nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=1956)
> log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> Sep 19 09:15:08 localhost nova-consoleauth[1984]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-2a9399e1-996d-42f4-899b-62db8d8e5afd #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mcompute_driver =
> nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=1984)
> log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> Sep 19 09:15:10 localhost nova-scheduler[2000]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-3d23848b-270b-4718-91dd-3b0417d27d21 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mcompute_driver =
> nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2000)
> log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> Sep 19 09:15:11 localhost devstack at placement-api.service
> <mailto:devstack at placement-api.service>[2030]: #033[00;32mDEBUG
> nova.api.openstack.placement.wsgi [#033[00;36m-#033[00;32m]
> #033[01;35m#033[00;32mcompute_driver =
> nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2294)
> log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> Sep 19 09:15:12 localhost devstack at n-api.service
> <mailto:devstack at n-api.service>[1999]: #033[00;32mDEBUG
> nova.api.openstack.wsgi_app [#033[01;36mNone
> req-1e20e359-2bcc-44e6-9c75-7422c5837c03 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mcompute_driver =
> nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2181)
> log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> Sep 19 09:15:14 localhost devstack at placement-api.service
> <mailto:devstack at placement-api.service>[2030]: #033[00;32mDEBUG
> nova.api.openstack.placement.wsgi [#033[00;36m-#033[00;32m]
> #033[01;35m#033[00;32mcompute_driver =
> nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2292)
> log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> Sep 19 09:15:15 localhost devstack at n-api.service
> <mailto:devstack at n-api.service>[1999]: #033[00;32mDEBUG
> nova.api.openstack.wsgi_app [#033[01;36mNone
> req-6d5745e7-3784-4b8e-8210-e7bf74fd9655 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mcompute_driver =
> nova_lxd.nova.virt.lxd.LXDDriver#033[00m #033[00;33m{{(pid=2182)
> log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> Sep 19 09:15:20 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-1bc67100-ffbe-4d22-be1f-42133eb60611 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mcompute_driver =
> lxd.LXDDriver#033[00m #033[00;33m{{(pid=1968) log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2890}}#033[00m
> ep 19 09:15:20 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-1bc67100-ffbe-4d22-be1f-42133eb60611 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mlxd.allow_live_migration =
> False#033[00m #033[00;33m{{(pid=1968) log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2898}}#033[00m
> Sep 19 09:15:20 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-1bc67100-ffbe-4d22-be1f-42133eb60611 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mlxd.pool = None#033[00m
> #033[00;33m{{(pid=1968) log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2898}}#033[00m
> Sep 19 09:15:20 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-1bc67100-ffbe-4d22-be1f-42133eb60611 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mlxd.root_dir =
> /var/lib/lxd/#033[00m #033[00;33m{{(pid=1968) log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2898}}#033[00m
> Sep 19 09:15:20 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_service.service [#033[01;36mNone
> req-1bc67100-ffbe-4d22-be1f-42133eb60611 #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mlxd.timeout = -1#033[00m
> #033[00;33m{{(pid=1968) log_opt_values
> /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py:2898}}#033[00m
> .........
> .........
> Sep 19 09:15:21 localhost nova-compute[1968]: #033[00;32mDEBUG
> oslo_concurrency.processutils [#033[01;36mNone
> req-87a543da-84db-4fe1-a743-75ca6525ac2a #033[00;36mNone
> None#033[00;32m] #033[01;35m#033[00;32mu'sudo nova-rootwrap
> /etc/nova/rootwrap.conf zpool list -o size -H None' failed. Not
> Retrying.#033[00m #033[00;33m{{(pid=1968) execute
> /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:457}}#033[00m
> Sep 19 09:15:21 localhost nova-compute[1968]: #033[01;31mERROR
> nova.compute.manager [#033[01;36mNone
> req-87a543da-84db-4fe1-a743-75ca6525ac2a #033[00;36mNone
> None#033[01;31m] #033[01;35m#033[01;31mError updating resources
> for node ubuntu.#033[00m: ProcessExecutionError: Unexpected error
> while running command.
> Sep 19 09:15:21 localhost nova-compute[1968]: Command: sudo
> nova-rootwrap /etc/nova/rootwrap.conf zpool list -o size -H None
> Sep 19 09:15:21 localhost nova-compute[1968]: Exit code: 1
> Sep 19 09:15:21 localhost nova-compute[1968]: Stdout: u''
> Sep 19 09:15:21 localhost nova-compute[1968]: Stderr: u"cannot
> open 'None': no such pool\n"
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00mTraceback (most recent
> call last):
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/opt/stack/nova/nova/compute/manager.py", line 7344, in
> update_available_resource_for_node
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m
> rt.update_available_resource(context, nodename)
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/opt/stack/nova/nova/compute/resource_tracker.py", line 673, in
> update_available_resource
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m resources =
> self.driver.get_available_resource(nodename)
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/opt/stack/nova-lxd/nova/virt/lxd/driver.py", line 1031, in
> get_available_resource
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m local_disk_info =
> _get_zpool_info(pool_name)
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/opt/stack/nova-lxd/nova/virt/lxd/driver.py", line 209, in
> _get_zpool_info
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m total =
> _get_zpool_attribute('size')
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/opt/stack/nova-lxd/nova/virt/lxd/driver.py", line 201, in
> _get_zpool_attribute
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m run_as_root=True)
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/opt/stack/nova/nova/utils.py", line 230, in execute
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m return
> RootwrapProcessHelper().execute(*cmd, **kwargs)
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/opt/stack/nova/nova/utils.py", line 113, in execute
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m return
> processutils.execute(*cmd, **kwargs)
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m File
> "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py",
> line 424, in execute
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m cmd=sanitized_cmd)
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00mProcessExecutionError:
> Unexpected error while running command.
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00mCommand: sudo
> nova-rootwrap /etc/nova/rootwrap.conf zpool list -o size -H None
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00mExit code: 1
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00mStdout: u''
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00mStderr: u"cannot open
> 'None': no such pool\n"
> Sep 19 09:15:21 localhost nova-compute[1968]: ERROR
> nova.compute.manager #033[01;35m#033[00m
>
> The local.conf used by devstack is looks like:
>
> [[local|localrc]]
> ############################################################
> # Customize the following HOST_IP based on your installation
> ############################################################
> HOST_IP=127.0.0.1
>
> ADMIN_PASSWORD=devstack
> MYSQL_PASSWORD=devstack
> RABBIT_PASSWORD=devstack
> SERVICE_PASSWORD=$ADMIN_PASSWORD
> SERVICE_TOKEN=devstack
>
> # run the services you want to use
> ENABLED_SERVICES=rabbit,mysql,key
> ENABLED_SERVICES+=,g-api,g-reg
> ENABLED_SERVICES+=,n-cpu,n-api,n-crt,n-obj,n-cond,n-sch,n-novnc,n-cauth,placement-api,placement-client
> ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-meta,q-l3
> ENABLED_SERVICES+=,cinder,c-sch,c-api,c-vol
> ENABLED_SERVICES+=,horizon
>
> # disabled services
> disable_service n-net
>
> # enable nova-lxd
> enable_plugin nova-lxd
> https://git.openstack.org/openstack/nova-lxd
> <https://git.openstack.org/openstack/nova-lxd> stable/queens
>
> Best regards,
> Martin.
>
>
> On 17.09.2018 15:29, Alex Kavanagh wrote:
>> Hi Martin
>>
>> On Sun, Sep 16, 2018 at 8:46 PM, Martin Bobák
>> <martin.bobak at savba.sk <mailto:martin.bobak at savba.sk>> wrote:
>>
>> Hi all,
>>
>> what is the recommended way of nova-lxd plugin installation
>> on a fresh xenial host running a pure OpenStack devstack
>> (Queens) installation? I have tried to install the nova-lxd
>> plugin by pip install, or allowing it during OpenStack
>> devstack installation, but each attempt lead to the same
>> result. The plugin is either not recognized or the
>> installation doesn't finish successfully. I went through the
>> nova-lxd homepage as well as its github repo, but I wasn't
>> able to solve the whole installation problem (e.g. I found
>> out that the installation of the newest version of pylxd
>> helps with installation of the plugin, however, the plugin
>> isn't still recognized. So additional configuration is
>> needed...).
>>
>> Do you have any thoughts about it?
>>
>>
>> I'm one of the maintainers for nova-lxd, so hopefully can get you
>> up and running.
>>
>> In order for nova-lxd to be configured in nova, the
>> /etc/nova-compute.conf needs to contain the lines:
>>
>> [DEFAULT]
>> compute_driver = nova_lxd.nova.virt.lxd.LXDDriver
>>
>> This little 'fact' is hidden away in the "nova-compute-lxd"
>> debian package, unfortunately.
>>
>> You'll also need to configure an [lxd] section in nova.conf to
>> control the storage pool in LXD for containers to use when
>> launching instances.
>>
>> [lxd]
>> allow_live_migration = True
>> pool = {{ storage_pool }}
>>
>> The storage pool will need to be set up separately in lxd.
>> --
>>
>> However, an 'easy' way to test OpenStack with nova-lxd, is to use
>> charms. We have a number of bundles that work with Juju. For
>> example we have a deployable bundle for xenial and queens at
>> https://github.com/openstack-charmers/openstack-bundles/tree/master/development/openstack-lxd-xenial-queens
>> <https://github.com/openstack-charmers/openstack-bundles/tree/master/development/openstack-lxd-xenial-queens>
>> which also has some (hopefully) useful instructions on how to get
>> it going.
>>
>> Note the instructions say you have to use MaaS, but you should be
>> able to adjust using it to the hardware you are using.
>>
>> As an alternative, the openstack-ansible project also supports
>> nova-lxd, but I don't have any experience with that.
>>
>> Do come back if you have any further questions; do let me know
>> how you get on.
>>
>> Best regards
>> Alex.
>>
>>
>> Best,
>> Martin.
>>
>> --
>> Martin Bobák, PhD.
>> Researcher
>> Institute of Informatics
>> Slovak Academy of Sciences
>> Dubravska cesta 9
>> <https://maps.google.com/?q=Dubravska+cesta+9&entry=gmail&source=g>,
>> SK-845 07 Bratislava, Slovakia
>> Room: 311, Phone: +421 (0)2 5941-1278
>> E-mail: martin.bobak at savba.sk <mailto:martin.bobak at savba.sk>
>> URL: http://www.ui.sav.sk/w/odd/pdip/
>> <http://www.ui.sav.sk/w/odd/pdip/>
>> LinkedIn: https://www.linkedin.com/in/martin-bobak/
>> <https://www.linkedin.com/in/martin-bobak/>
>>
>> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> <mailto:lxc-users at lists.linuxcontainers.org>
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>> <http://lists.linuxcontainers.org/listinfo/lxc-users>
>>
>>
>>
>>
>> --
>> Alex Kavanagh - Software Engineer
>> Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd
>>
>>
>> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> <mailto:lxc-users at lists.linuxcontainers.org>
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>> <http://lists.linuxcontainers.org/listinfo/lxc-users>
>
>
>
>
> --
> Alex Kavanagh - Software Engineer
> Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd
>
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
--
Martin Bobák, PhD.
Researcher
Institute of Informatics
Slovak Academy of Sciences
Dubravska cesta 9, SK-845 07 Bratislava, Slovakia
Room: 311, Phone: +421 (0)2 5941-1278
E-mail: martin.bobak at savba.sk
URL: http://www.ui.sav.sk/w/odd/pdip/
LinkedIn: https://www.linkedin.com/in/martin-bobak/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20180920/bf6d3231/attachment-0001.html>
More information about the lxc-users
mailing list