[lxc-users] lxd live migration fail: "write AA policy failed: File exists"

[Brian Candler]

I got this error try to live-migrate an lxd container.  Both sides are 
running a fully-patched 16.04 with lxd 2.0.11.  They are different 
chipsets - the source is AMD Opteron X3216 and the destination is Intel 
N2820 - but I don't think that's the problem here.


root at nuc2:~# lxc move storage1:apt-cacher apt-cacher
error: Failed container creation:
  - https://storage1.home.deploy2.net:8443: migration restore failed
(00.128764) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming usr.sbin.tcpdump.12 to 
:lxd-apt-cacher_<var-lib-lxd>:
(00.128847) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming 
usr.lib.snapd.snap-confine.9 to :lxd-apt-cacher_<var-lib-lxd>:
(00.128857) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming usr.bin.lxc-start.8 to 
:lxd-apt-cacher_<var-lib-lxd>:
(00.128864) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming 
usr.lib.connman.scripts.dhclient-script.7 to :lxd-apt-cacher_<var-lib-lxd>:
(00.128872) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming 
usr.lib.NetworkManager.nm-dhcp-helper.6 to :lxd-apt-cacher_<var-lib-lxd>:
(00.128879) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming 
usr.lib.NetworkManager.nm-dhcp-client.action.5 to 
:lxd-apt-cacher_<var-lib-lxd>:
(00.128886) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming sbin.dhclient.4 to 
:lxd-apt-cacher_<var-lib-lxd>:
(00.128893) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming 
lxc-container-default-with-nesting.3 to :lxd-apt-cacher_<var-lib-lxd>:
(00.128900) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming 
lxc-container-default-with-mounting.2 to :lxd-apt-cacher_<var-lib-lxd>:
(00.128907) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming 
lxc-container-default-cgns.1 to :lxd-apt-cacher_<var-lib-lxd>:
(00.128931) Warn  (criu/apparmor.c:401): binary rewriting of apparmor 
policies not supported right now, not renaming lxc-container-default.0 
to :lxd-apt-cacher_<var-lib-lxd>:
(00.128967) Warn  (criu/apparmor.c:421): apparmor namespace 
/sys/kernel/security/apparmor/policy/namespaces/lxd-apt-cacher_<var-lib-lxd> 
already exists, restoring into it
(00.134500) Error (criu/apparmor.c:460): write AA policy failed: File exists
(00.135844) Error (criu/apparmor.c:479): failed to restore aa namespace, 
worker exited: 256
  - https://10.12.255.3:8443: websocket: bad handshake
  - https://[2001:470:XXXX:XXXX::3]:8443: websocket: bad handshake
  - https://10.12.254.3:8443: Unable to connect to: 10.12.254.3:8443
  - https://10.12.3.1:8443: websocket: bad handshake
  - https://[2001:470:YYYY:3::1]:8443: websocket: bad handshake

The migration was successfully rolled back, and I was able to complete 
the migration by shutting down the container and migrating again.

However it looks like it *almost* managed a successful live migration, 
but there was some problem with apparmor.

Looking now on the target host:

root at nuc2:~# ls 
'/sys/kernel/security/apparmor/policy/namespaces/lxd-apt-cacher_<var-lib-lxd>'
namespaces  profiles  raw_data

On the source host this no longer exists, but then that's because the 
container "apt-cacher" has gone from there.

This isn't a crucial container so if there's anything useful I can do to 
help fix this I'm happy to give it a go.

Cheers,

Brian.