[Lxc-users] Broken my lxc vms

[Gordon Henderson]

On Sat, 3 Apr 2010, Nigel Magnay wrote:

>>> I'm seeing things like
>>>
>>> root <at> nirima-host:/home/magnayn# lxc-start -n ubuntu
>>> swapon: /dev/disk/by-uuid/35b40dbb-337c-4f46-a82f-642d6fbf3faa: swapon
>>> failed: Device or resource busy
>>>
>> OK - I'm relatively new to LXC (couple of months), I also use Debian not
>> Ubuntu, but to me, it looks like your container is executing a whole bunch
>> of init-scripts that it shouldn't be executing. (or really doesn't need
>> to)
>>
>> In my containers, I just have /etc/init.d/rcS with nothing more than a
>> line to create a default route (as the network is already created with
>> lxc-start). The line
>>
>>   exec /etc/init.d/rc S
>>
>> is commented out. That's the ones in Debian that'll normally do stuff with
>> hardware like activate swap, fsck, etc.
>>
>
> Hm - I've tried rebooting the machine, which didn't help. Note that
> this image *used* to work; something has
> gotten corrupted somewhere, possibly when it was non-gracefully shutdown.
>
> I'm a bit confused;
> /dev/disk/by-uuid/35b40dbb-337c-4f46-a82f-642d6fbf3faa is swap in the
> outer machine, not the
> image itself. Why is lxc-start trying to turn on the swapfile (when
> it's already on) ?
>
> Either way, it's probably a red-herring; I can get past that but it
> still freezes..

I'm not sure it should see that though - make sure your init scripts are 
not trying to swapon, or fsck, etc. and make sure it's not in your 
/etc/fstab.

Same for other things - really, in a container all I'm running is syslogd, 
sshd, apache and then either mysql or asterisk depending on it's use...

No NTP, hwmonitoring, mdadm, snmp, etc. as that's all being done on the 
host. (actually there may be a case for snmp if you're remotely checking 
the network stats)

>>> Also - lxc looks exactly what I want to isolate some app servers into
>>> individual units. Is lxc considered reasonably stable (as it's
>>> mainline now), or should I steer clear for a bit ?
>>
>> I've jumped in at the deep end - did a lot of local testing myself
>> including running 50 containers on an old server, starting/stopping,
>> running applications, etc. each running a standard LAMPy type thing - and
>> Asterisk... And was happy enough with it to start to migrate a lot of
>> remote hosted servers over to it, and have decided to build all my server
>> from now on with containers in-mind.
>
> Good to know; I'm just slightly nervous as these initial tests have
> broken an image that was working, so I'd like to know
> why before I get a problem if I run it live..

I had a lot of "breakages" when starting out too - but once I culled all 
the init scripts everything went much smoother. Also watch out for the 
shutdown scripts too - so that if you shotdown a container (from inside 
the container with shutdown/reboot/init 0) then it doesn't try to unmount, 
or shutdown mdadm devices, etc. which may affect the host.

I'm not sure an LXC container is secure enough at present to allow root 
access to someone you don't trust, but I guess with proper use of limiting 
the underlying hardware the container can access it might be...

I'm using it as a management tool to better maintain a small raft of 
servers that I operate.

Gordon