[Lxc-users] Container with different architecture like arm on x86 [How-to]

[Justin Cormack]

On Wed, 2011-04-06 at 16:45 -0400, Brian K. White wrote:

> A few questions,
> 
> The echo command references /usr/local/bin/qemu-arm, but I don't see 
> that anywhere else in the recipe. Is that a x86_64 binary on the host or 
> is that supposed to be a arm binary in the container, or is it simply 
> ignored in this case and doesn't matter that it doesn't exist?
> 
> It sort of looks like you are telling the host x86_64 kernel to run a 
> x86_64 qemu-arm executable any time it encounters an arm elf executable, 
> and then since you are placing an arm qemu-arm executable in the 
> container fs I guess you are implying that the arm executable you will 
> be trying to run will be that arm qemu executable? Why would you do that?
> 
> foo -> qemu -> qemu -> kernel ??
> 
> ie: arm-executable foo -> arm executable qemu-arm -> x86_64 executable 
> qemu-arm -> x86_64 host kernel ??
> 
> Assuming that even works. Doesn't there have to be an arm kernel in 
> there somewhere? Like:
> 
> arm-foo -> arm-kernel -> x86_64-qemu-arm -> x86_64-host-kernel
> 
> I don't see the point in this. As long as you have qemu in there 
> anywhere it means you are doing full cpu virtualization, avoiding which 
> is pretty much the sole purpose of containers.
> 
> If it's really true that you can have qemu provide _only_ cpu 
> virtualization yet somehow have the host kernel support the arm 
> executables through that I guess that's a win since you have a single 
> kernel doling out resources directly to all processes instead of kernels 
> within kernels. Then again wouldn't that result in every single arm 
> executable running inside it's own instance of qemu, auto launched by 
> the binfmt? That might be ok for application containers that only run 
> one process but that would be terrible for a full system container 
> unless that container really only ran one process directly, an arm 
> kernel. And in that case I don't see the point of doing that inside a 
> container. It's already even more isolated inside qemu than what the 
> container provides and the container layer just becomes pointless overhead.
> 
> But doesn't the arm kernel have rather a lot more differences than 
> merely understanding the arm binary format and cpu? I would have thought 
> the container would have to run an x86_64 (or i386) binary, which would 
> be qemu, and that qemu would have to run an arm kernel, and all other 
> arm processes would have to run in that arm kernel.
> 
> I think I need an example to illustrate a use case for this.
> 

Qemu is just being used as an arm instruction set interpreter, making
x86 system calls to the native kernel. binfmt_misc lets you run other
architecture binaries via emulation just by executing the binary.
Obviously its slow, but if you want to build an arm distro say it gives
another option other than cross compiling or a native compile on a slow
machine.

Justin