On Fri, Nov 9, 2012 at 12:09 AM, unicell <span dir="ltr"><<a href="mailto:unicell@gmail.com" target="_blank">unicell@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<div><br></div><div>I'm trying to manage LXC instances through OpenStack, which use libvirt as a virtualization driver layer. After launching LXC instance, I simply could not attach to the console.</div><div><br></div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div>virsh # list</div></div>
<div><div> Id Name State</div></div><div><div>----------------------------------</div></div><div><div>14366 instance-00000078 running</div></div><div><div><br></div></div><div><div>virsh # console 14366</div>
</div><div><div>Connected to domain instance-00000078</div></div><div><div>Escape character is ^]</div></div></blockquote><div><br></div><div>And it keeps stuck here.</div><div><br></div><div>Initially I thought it could be a cgroup device config issue, and tried to "echo c 5:1 rwm' >/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow, to add /dev/console access right to the container. But it does not work, console stilll not working.</div>
<div><br></div><div>Before the cgroup tweaking, devices.list are as follows</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>[root@localhost libvirt]# cat /cgroup/devices/libvirt/lxc/instance-0000007a/devices.list</div>
<div><br></div><div><div>c 1:3 rwm</div></div><div><div>c 1:5 rwm</div></div><div><div>c 1:7 rwm</div></div><div><div>c 1:8 rwm</div></div><div><div>c 1:9 rwm</div></div><div><div>c 5:0 rwm</div></div><div><div>c 5:2 rwm</div>
</div><div><div>c 136:* rwm</div></div></blockquote>
<div><br></div><div>I'm not quite sure what kind of issue could possibly cause this console access issue. Cause I'm now using two different guest OS templates on two different host OSes. Following are some symptoms I observed.</div>
<div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div><div>* (g) for container guest OS</div><div>* (h) for host OS</div><div>* NOTE: Ubuntu guest os are generated by lxc-create -t ubuntu, which is an Ubuntu 12.04 based lxc template.</div>
<div> CentOS guest os are handcrafted by myself, taking reference of <a href="http://wiki.1tux.org/wiki/Lxc/Installation/Guest/Centos/6" target="_blank">http://wiki.1tux.org/wiki/Lxc/Installation/Guest/Centos/6</a></div>
<div>
<ol><li>Ubuntu (g) launched by lxc-start on Ubuntu 12.04 host, console works</li><li>Ubuntu (g) launched by OpenStack / libvirt on Ubuntu 12.04 host, console works</li><li>Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host, console NOT works</li>
<li>CentOS (g) launched by lxc-start on Ubuntu 12.04 host, console works</li><li>CentOS (g) launched by OpenStack / libvirt on Ubuntu12.04 host, console NOT works</li><li>CentOS (g) launched by OpenStack / libvirt on CentOS 6.3 host, console NOT works</li>
</ol></div></blockquote></div><div>So somehow I feel it could be something related to LXC template rootfs, but Host OS do make a difference (2 vs. 3). lxc-start / libvirt also makes a difference (4 vs. 5), but that might be caused by cgroup setting.</div>
<div><br></div><div>Could someone shed me some clues to further digging this issue? Thanks! </div><div><br></div><div>--<br>Qiu Yu<br>
</div>
</blockquote></div><br><div>For the "Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host", I also tried following command and console still not working. </div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>echo "a *.* rwm" >/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow</div></blockquote><div><br></div><div>More information attached.</div><div><br></div><div><div>[root@localhost ~]# ls -l /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty*</div>
<div>crw-rw-rw- 1 root tty 5, 0 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty</div><div>crw------- 1 root tty 4, 0 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty0</div>
<div>crw------- 1 root tty 4, 1 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty1</div><div>crw------- 1 root tty 4, 2 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty2</div>
<div>crw------- 1 root tty 4, 3 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty3</div><div>crw------- 1 root tty 4, 4 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty4</div>
<div>crw------- 1 root tty 4, 5 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty5</div><div>crw------- 1 root tty 4, 6 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty6</div>
<div>crw------- 1 root tty 4, 7 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty7</div><div>crw------- 1 root tty 4, 8 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty8</div>
<div>crw------- 1 root tty 4, 9 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty9</div><div><br></div><div><br></div><div><div>[root@localhost ~]# cat /cgroup/devices/libvirt/lxc/instance-00000078/devices.list </div>
<div>c 1:3 rwm</div><div>c 1:5 rwm</div><div>c 1:7 rwm</div><div>c 1:8 rwm</div><div>c 1:9 rwm</div><div>c 5:0 rwm</div><div>c 5:2 rwm</div><div>c 136:* rwm</div><div>c 5:1 rwm</div><div>c 5:* rwm</div><div>a *:* rwm</div>
</div><div><br></div><div><br></div><div><div>[root@localhost ~]# cat /etc/libvirt/lxc/instance-00000078.xml </div><div><!--</div><div>WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE </div><div>
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:</div><div> virsh edit instance-00000078</div><div>or other application using the libvirt API.</div><div>--></div><div><br></div><div><domain type='lxc'></div>
<div> <name>instance-00000078</name></div><div> <uuid>55f225d1-e49c-4a51-9c17-0a5dde6d1917</uuid></div><div> <memory>524288</memory></div><div> <currentMemory>524288</currentMemory></div>
<div> <vcpu>2</vcpu></div><div> <os></div><div> <type arch='x86_64'>exe</type></div><div> <init>/sbin/init</init></div><div> <cmdline>console=ttyS0</cmdline></div>
<div> </os></div><div> <clock offset='utc'/></div><div> <on_poweroff>destroy</on_poweroff></div><div> <on_reboot>restart</on_reboot></div><div> <on_crash>destroy</on_crash></div>
<div> <devices></div><div> <emulator>/usr/libexec/libvirt_lxc</emulator></div><div> <filesystem type='mount' accessmode='passthrough'></div><div> <source dir='/home/stack/nova_state/instances/instance-00000078/rootfs'/></div>
<div> <target dir='/'/></div><div> </filesystem></div><div> <interface type='bridge'></div><div> <mac address='fa:16:3e:08:4f:b9'/></div><div> <source bridge='br100'/></div>
<div> <filterref filter='nova-instance-instance-00000078-fa163e084fb9'></div><div> <parameter name='DHCPSERVER' value='10.48.253.1'/></div><div> <parameter name='IP' value='10.48.253.2'/></div>
<div> <parameter name='PROJMASK' value='255.255.255.0'/></div><div> <parameter name='PROJNET' value='10.48.253.0'/></div><div> </filterref></div><div> </interface></div>
<div> <console type='pty'></div><div> <target type='lxc' port='0'/></div><div> </console></div><div> </devices></div><div></domain></div></div><div><br></div>
<div><br></div>--<br>Qiu Yu<br></div>