[Lxc-users] kernel bug?
Gary Ballantyne
gary.ballantyne at haulashore.com
Thu Mar 14 00:16:31 UTC 2013
Hi All
I have an intermittent, but crippling, problem on a raring EC2 instance
(also on quantal). Its a (raring) lvm-backed container --- I use cgroups
directly (via /sys/fs) and iptables in the instance (not sure if that's
relevant at all).
Occasionally, when stopping or starting the container (there is just
one), the instance becomes unreachable. Rebooting doesn't help, but
starting/stopping the instance, typically at least twice, fixes things
(the instance is reachable, and the container auto-starts).
There doesn't appear to be anything sinister in /var/log/dmesg (upon
restart), but the AWS system log is pasted below. I *think* the first
part corresponds to before the crash, and the interesting bit is:
[3587596.471053] ------------[ cut here ]------------
[3587596.471071] Kernel BUG at ffffffff816c7c2c [verbose debug info
...
[3587596.472282] ---[ end trace dc5c4320e1320f1d ]---
[3587596.472292] Fixing recursive fault but reboot is needed!
The log doesn't mention lxc, but it is definitely starting/stopping a
container that triggers the problem. Not sure if this is a lxc, or even
EC2 problem --- but greatly appreciate any ideas anyone might have.
Thanks
Gary
=================================================================================
Xen Minimal OS!
start_info: 0xce2000(VA)
nr_pages: 0x6a400
shared_inf: 0x7e02e000(MA)
pt_base: 0xce5000(VA)
nr_pt_frames: 0xb
mfn_list: 0x990000(VA)
mod_start: 0x0(VA)
mod_len: 0
flags: 0x0
cmd_line: root=/dev/sda1 ro 4
stack: 0x94f860-0x96f860
MM: Init
_text: 0x0(VA)
_etext: 0x5ffbd(VA)
_erodata: 0x78000(VA)
_edata: 0x80ae0(VA)
stack start: 0x94f860(VA)
_end: 0x98fe68(VA)
start_pfn: cf3
max_pfn: 6a400
Mapping memory range 0x1000000 - 0x6a400000
setting 0x0-0x78000 readonly
skipped 0x1000
MM: Initialise page allocator for 103e000(103e000)-6a400000(6a400000)
MM: done
Demand map pfns at 6a401000-206a401000.
Heap resides at 206a402000-406a402000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x6a401000.
Initialising scheduler
Thread "Idle": pointer: 0x206a402010, stack: 0x13b0000
Initialising xenbus
Thread "xenstore": pointer: 0x206a4027c0, stack: 0x13c0000
Dummy main: start_info=0x96f960
Thread "main": pointer: 0x206a402f70, stack: 0x13d0000
"main" "root=/dev/sda1" "ro" "4"
vbd 2049 is hd0
******************* BLKFRONT for device/vbd/2049 **********
backend at /local/domain/0/backend/vbd/2550/2049
Failed to read /local/domain/0/backend/vbd/2550/2049/feature-barrier.
Failed to read /local/domain/0/backend/vbd/2550/2049/feature-flush-cache.
16777216 sectors of 512 bytes
**************************
vbd 2051 is hd1
******************* BLKFRONT for device/vbd/2051 **********
backend at /local/domain/0/backend/vbd/2550/2051
Failed to read /local/domain/0/backend/vbd/2550/2051/feature-barrier.
Failed to read /local/domain/0/backend/vbd/2550/2051/feature-flush-cache.
1835008 sectors of 512 bytes
**************************
vbd 2064 is hd2
******************* BLKFRONT for device/vbd/2064 **********
backend at /local/domain/0/backend/vbd/2550/2064
Failed to read /local/domain/0/backend/vbd/2550/2064/feature-barrier.
Failed to read /local/domain/0/backend/vbd/2550/2064/feature-flush-cache.
312705024 sectors of 512 bytes
**************************
vbd 51792 is hd3
******************* BLKFRONT for device/vbd/51792 **********
backend at /local/domain/0/backend/vbd/2550/51792
Failed to read /local/domain/0/backend/vbd/2550/51792/feature-barrier.
Failed to read /local/domain/0/backend/vbd/2550/51792/feature-flush-cache.
2097152 sectors of 512 bytes
**************************
vbd 51904 is hd4
******************* BLKFRONT for device/vbd/51904 **********
backend at /local/domain/0/backend/vbd/2550/51904
Failed to read /local/domain/0/backend/vbd/2550/51904/feature-barrier.
Failed to read /local/domain/0/backend/vbd/2550/51904/feature-flush-cache.
2097152 sectors of 512 bytes
**************************
vbd 51952 is hd5
******************* BLKFRONT for device/vbd/51952 **********
backend at /local/domain/0/backend/vbd/2550/51952
Failed to read /local/domain/0/backend/vbd/2550/51952/feature-barrier.
Failed to read /local/domain/0/backend/vbd/2550/51952/feature-flush-cache.
2097152 sectors of 512 bytes
**************************
vbd 268441344 is hd6
******************* BLKFRONT for device/vbd/268441344 **********
backend at /local/domain/0/backend/vbd/2550/268441344
Failed to read /local/domain/0/backend/vbd/2550/268441344/feature-barrier.
Failed to read
/local/domain/0/backend/vbd/2550/268441344/feature-flush-cache.
33554432 sectors of 512 bytes
**************************
[H
[J Booting 'Ubuntu Raring Ringtail (development branch), kernel
3.8.0-6-generic'
root (hd0)
Filesystem type is ext2fs, using whole disk
kernel /boot/vmlinuz-3.8.0-6-generic root=LABEL=cloudimg-rootfs ro
console=hvc
0
initrd /boot/initrd.img-3.8.0-6-generic
close blk: backend at /local/domain/0/backend/vbd/2550/2049
close blk: backend at /local/domain/0/backend/vbd/2550/2051
close blk: backend at /local/domain/0/backend/vbd/2550/2064
close blk: backend at /local/domain/0/backend/vbd/2550/51792
close blk: backend at /local/domain/0/backend/vbd/2550/51904
close blk: backend at /local/domain/0/backend/vbd/2550/51952
close blk: backend at /local/domain/0/backend/vbd/2550/268441344
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 3.8.0-6-generic (buildd at allspice) (gcc
version 4.7.2 (Ubuntu/Linaro 4.7.2-21ubuntu1) ) #11-Ubuntu SMP Mon Feb
11 16:02:49 UTC 2013 (Ubuntu 3.8.0-6.11-generic 3.8.0-rc7)
[ 0.000000] Command line: root=LABEL=cloudimg-rootfs ro console=hvc0
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Centaur CentaurHauls
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
[ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[ 0.000000] Xen: [mem 0x0000000000100000-0x000000006abfffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI not present or invalid.
[ 0.000000] No AGP bridge found
[ 0.000000] e820: last_pfn = 0x6ac00 max_arch_pfn = 0x400000000
[ 0.000000] x2apic enabled by BIOS, switching to x2apic ops
[ 0.000000] init_memory_mapping: [mem 0x00000000-0x6abfffff]
[ 0.000000] RAMDISK: [mem 0x02363000-0x035b2fff]
[ 0.000000] NUMA turned off
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000006abfffff]
[ 0.000000] Initmem setup node 0 [mem 0x00000000-0x6abfffff]
[ 0.000000] NODE_DATA [mem 0x6a3fb000-0x6a3fffff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00010000-0x00ffffff]
[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00010000-0x0009ffff]
[ 0.000000] node 0: [mem 0x00100000-0x6abfffff]
[ 0.000000] SFI: Simple Firmware Interface v0.81
http://simplefirmware.org
[ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 -
0000000000100000
[ 0.000000] e820: [mem 0x6ac00000-0xffffffff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 3.4.3.amazon (preserve-AD)
[ 0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256
nr_cpu_ids:1 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff880069c00000 s85056
r8192 d21440 u2097152
[3587550.286346] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 430298
[3587550.286348] Policy zone: DMA32
[3587550.286351] Kernel command line: root=LABEL=cloudimg-rootfs ro
console=hvc0
[3587550.287079] PID hash table entries: 4096 (order: 3, 32768 bytes)
[3587550.287086] __ex_table already sorted, skipping sort
[3587550.287123] Checking aperture...
[3587550.296363] No AGP bridge found
[3587550.328186] Memory: 1669920k/1748992k available (6978k kernel code,
448k absent, 78624k reserved, 6261k data, 992k init)
[3587550.328298] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=1, Nodes=1
[3587550.328336] Hierarchical RCU implementation.
[3587550.328338] RCU dyntick-idle grace-period acceleration is enabled.
[3587550.328339] RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=1.
[3587550.328352] NR_IRQS:16640 nr_irqs:256 16
[3587550.328497] Console: colour dummy device 80x25
[3587550.328660] console [tty0] enabled
[3587550.328883] console [hvc0] enabled
[3587550.330054] allocated 7340032 bytes of page_cgroup
[3587550.330063] please try 'cgroup_disable=memory' option if you don't
want memory cgroups
[3587550.330111] installing Xen timer for CPU 0
[3587550.330156] tsc: Detected 1800.000 MHz processor
[3587550.330165] Calibrating delay loop (skipped), value calculated
using timer frequency.. 3600.00 BogoMIPS (lpj=7200000)
[3587550.330178] pid_max: default: 32768 minimum: 301
[3587550.330226] Security Framework initialized
[3587550.330245] AppArmor: AppArmor initialized
[3587550.330250] Yama: becoming mindful.
[3587550.330719] Dentry cache hash table entries: 262144 (order: 9,
2097152 bytes)
[3587550.331375] Inode-cache hash table entries: 131072 (order: 8,
1048576 bytes)
[3587550.331619] Mount-cache hash table entries: 256
[3587550.331929] Initializing cgroup subsys cpuacct
[3587550.331939] Initializing cgroup subsys memory
[3587550.331961] Initializing cgroup subsys devices
[3587550.331967] Initializing cgroup subsys freezer
[3587550.331973] Initializing cgroup subsys blkio
[3587550.331979] Initializing cgroup subsys perf_event
[3587550.331986] Initializing cgroup subsys hugetlb
[3587550.332075] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[3587550.332075] ENERGY_PERF_BIAS: View and update with
x86_energy_perf_policy(8)
[3587550.332095] CPU: Physical Processor ID: 1
[3587550.332100] CPU: Processor Core ID: 7
[3587550.332110] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
[3587550.332110] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32
[3587550.332110] tlb_flushall_shift: 5
[3587550.471697] Freeing SMP alternatives: 24k freed
[3587550.475411] ftrace: allocating 26566 entries in 104 pages
[3587550.518610] cpu 0 spinlock event irq 17
[3587550.518651] Performance Events: unsupported p6 CPU model 45 no PMU
driver, software events only.
[3587550.519921] Brought up 1 CPUs
[3587550.519992] NMI watchdog: disabled (cpu0): hardware events not enabled
[3587550.520138] devtmpfs: initialized
[3587550.520990] EVM: security.selinux
[3587550.520997] EVM: security.SMACK64
[3587550.521001] EVM: security.capability
[3587550.521974] Grant tables using version 1 layout.
[3587550.522003] Grant table initialized
[3587550.522068] regulator-dummy: no parameters
[3587550.522133] NET: Registered protocol family 16
[3587550.523364] PCI: setting up Xen PCI frontend stub
[3587550.524287] bio: create slab <bio-0> at 0
[3587550.524397] ACPI: Interpreter disabled.
[3587550.524688] xen/balloon: Initialising balloon driver.
[3587550.526315] xen-balloon: Initialising balloon driver.
[3587550.526568] vgaarb: loaded
[3587550.526789] SCSI subsystem initialized
[3587550.526879] usbcore: registered new interface driver usbfs
[3587550.526910] usbcore: registered new interface driver hub
[3587550.526951] usbcore: registered new device driver usb
[3587550.527059] PCI: System does not support PCI
[3587550.527068] PCI: System does not support PCI
[3587550.527171] NetLabel: Initializing
[3587550.527178] NetLabel: domain hash size = 128
[3587550.527183] NetLabel: protocols = UNLABELED CIPSOv4
[3587550.527198] NetLabel: unlabeled traffic allowed by default
[3587550.527254] Switching to clocksource xen
[3587550.535825] AppArmor: AppArmor Filesystem Enabled
[3587550.535852] pnp: PnP ACPI: disabled
[3587550.537442] NET: Registered protocol family 2
[3587550.537660] TCP established hash table entries: 16384 (order: 6,
262144 bytes)
[3587550.537761] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
[3587550.537815] TCP: Hash tables configured (established 16384 bind 16384)
[3587550.537864] TCP: reno registered
[3587550.537880] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[3587550.537899] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[3587550.537972] NET: Registered protocol family 1
[3587550.538040] Trying to unpack rootfs image as initramfs...
[3587550.596663] Freeing initrd memory: 18752k freed
[3587550.603768] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[3587550.604063] Initialise module verification
[3587550.604124] audit: initializing netlink socket (disabled)
[3587550.604146] type=2000 audit(1363215276.473:1): initialized
[3587550.686261] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[3587550.687788] VFS: Disk quotas dquot_6.5.2
[3587550.687839] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[3587550.688382] fuse init (API version 7.20)
[3587550.688473] msgmni has been set to 3298
[3587550.688829] Key type asymmetric registered
[3587550.688836] Asymmetric key parser 'x509' registered
[3587550.688884] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 252)
[3587550.688927] io scheduler noop registered
[3587550.688933] io scheduler deadline registered (default)
[3587550.688943] io scheduler cfq registered
[3587550.689026] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[3587550.689046] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[3587550.689456] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[3587550.690681] Linux agpgart interface v0.103
[3587550.692013] brd: module loaded
[3587550.692714] loop: module loaded
[3587550.777721] libphy: Fixed MDIO Bus: probed
[3587550.777804] tun: Universal TUN/TAP device driver, 1.6
[3587550.777810] tun: (C) 1999-2004 Max Krasnyansky <maxk at qualcomm.com>
[3587550.777903] PPP generic driver version 2.4.2
[3587550.777946] Initialising Xen virtual ethernet driver.
[3587550.780145] blkfront: xvda1: barrier or flush: disabled
[3587550.781597] Setting capacity to 16777216
[3587550.781612] xvda1: detected capacity change from 0 to 8589934592
[3587550.782579] blkfront: xvdb: barrier or flush: disabled
[3587550.784541] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[3587550.784551] ehci-pci: EHCI PCI platform driver
[3587550.784568] ehci-platform: EHCI generic platform driver
[3587550.784585] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[3587550.784601] uhci_hcd: USB Universal Host Controller Interface driver
[3587550.784655] i8042: PNP: No PS/2 controller found. Probing ports
directly.
[3587552.763783] i8042: No controller found
[3587552.764050] mousedev: PS/2 mouse device common for all mice
[3587552.893562] xvdb: unknown partition table
[3587552.893684] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[3587552.893763] rtc_cmos: probe of rtc_cmos failed with error -38
[3587552.893914] Setting capacity to 312705024
[3587552.893931] xvdb: detected capacity change from 0 to 160104972288
[3587552.894150] device-mapper: uevent: version 1.0.3
[3587552.894302] device-mapper: ioctl: 4.23.1-ioctl (2012-12-18)
initialised: dm-devel at redhat.com
[3587552.894330] ledtrig-cpu: registered to indicate activity on CPUs
[3587552.894337] EFI Variables Facility v0.08 2004-May-17
[3587552.894668] ashmem: initialized
[3587552.894868] TCP: cubic registered
[3587552.895029] NET: Registered protocol family 10
[3587552.895096] blkfront: xvdp: barrier or flush: disabled
[3587552.895389] NET: Registered protocol family 17
[3587552.895410] Key type dns_resolver registered
[3587552.895530] Loading module verification certificates
[3587552.896750] MODSIGN: Loaded cert 'Magrathea: Glacier signing key:
b83594932a8b165988e2d3fc7e4792d2945f8465'
[3587552.896793] registered taskstats version 1
[3587552.901587] Key type trusted registered
[3587552.905738] Key type encrypted registered
[3587552.911592] xvdp: unknown partition table
[3587552.911813] Setting capacity to 2097152
[3587552.911824] xvdp: detected capacity change from 0 to 1073741824
[3587552.912821] blkfront: xvdm: barrier or flush: disabled
[3587552.934372] xvdm: unknown partition table
[3587552.934608] Setting capacity to 2097152
[3587552.934618] xvdm: detected capacity change from 0 to 1073741824
[3587552.935627] blkfront: xvdf: barrier or flush: disabled
[3587552.948907] xvdf: unknown partition table
[3587552.949157] Setting capacity to 2097152
[3587552.949174] xvdf: detected capacity change from 0 to 1073741824
[3587552.950264] blkfront: xvdx: barrier or flush: disabled
[3587553.172533] xvdx: unknown partition table
[3587553.172835] Setting capacity to 33554432
[3587553.172846] xvdx: detected capacity change from 0 to 17179869184
[3587553.173927] blkfront: xvda3: barrier or flush: disabled
[3587553.175841] Setting capacity to 1835008
[3587553.175852] xvda3: detected capacity change from 0 to 939524096
[3587553.207384] /build/buildd/linux-3.8.0/drivers/rtc/hctosys.c: unable
to open rtc device (rtc0)
[3587553.207447] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[3587553.207454] EDD information not available.
[3587553.207888] Freeing unused kernel memory: 992k freed
[3587553.208248] Write protecting the kernel read-only data: 12288k
[3587553.212645] Freeing unused kernel memory: 1204k freed
[3587553.213577] Freeing unused kernel memory: 1112k freed
Loading, please wait...
[3587553.281264] udevd[92]: starting version 175
[3587553.669021] Disabling lock debugging due to kernel taint
[3587554.139661] bio: create slab <bio-1> at 1
error: unexpectedly disconnected from boot status daemon
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top
... done.
Begin: Running /scripts/local-premount ... done.
[3587555.675128] EXT4-fs (xvda1): INFO: recovery required on readonly
filesystem
[3587555.675145] EXT4-fs (xvda1): write access will be enabled during
recovery
[3587556.187163] EXT4-fs (xvda1): orphan cleanup on readonly fs
[3587556.187355] EXT4-fs (xvda1): 5 orphan inodes deleted
[3587556.187366] EXT4-fs (xvda1): recovery complete
[3587556.193434] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... done.
[3587557.325542] EXT4-fs (xvda1): re-mounted. Opts: (null)
[3587557.371338] Adding 917500k swap on /dev/xvda3. Priority:-1
extents:1 across:917500k SS
Cloud-init v. 0.7.1 running 'init' at Wed, 13 Mar 2013 22:54:49 +0000.
Up 13.33 seconds.
ci-info: +++++++++++++++++++++++++++Net device
info+++++++++++++++++++++++++++
ci-info:
+--------+------+-------------+-----------------+-------------------+
ci-info: | Device | Up | Address | Mask |
Hw-Address |
ci-info:
+--------+------+-------------+-----------------+-------------------+
ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | .
|
ci-info: | eth0 | True | 10.197.49.9 | 255.255.255.192 |
22:00:0a:c5:31:09 |
ci-info:
+--------+------+-------------+-----------------+-------------------+
ci-info: ++++++++++++++++++++++++++++++++Route
info+++++++++++++++++++++++++++++++++
ci-info:
+-------+-------------+-------------+-----------------+-----------+-------+
ci-info: | Route | Destination | Gateway | Genmask |
Interface | Flags |
ci-info:
+-------+-------------+-------------+-----------------+-----------+-------+
ci-info: | 0 | 0.0.0.0 | 10.197.49.1 | 0.0.0.0 | eth0
| UG |
ci-info: | 1 | 10.197.49.0 | 0.0.0.0 | 255.255.255.192 | eth0
| U |
ci-info:
+-------+-------------+-------------+-----------------+-----------+-------+
* Starting AppArmor profiles
[80G Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
[74G[ OK ]
landscape-client is not configured, please run landscape-config.
acpid: cannot open input layer
acpid: starting up with netlink and the input layer
acpid: 1 rule loaded
acpid: waiting for events: event logging is off
* Starting web server apache2
[80G [3587593.446438] EXT3-fs (dm-0): error: couldn't mount because
of unsupported optional features (240)
[74G[ OK ]
[3587596.471053] ------------[ cut here ]------------
[3587596.471071] Kernel BUG at ffffffff816c7c2c [verbose debug info
unavailable]
[3587596.471080] invalid opcode: 0000 [#1] SMP
[3587596.471087] Modules linked in: xt_multiport(F) iptable_filter(F)
veth(F) ipt_MASQUERADE(F) iptable_nat(F) nf_conntrack_ipv4(F)
nf_defrag_ipv4(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack(F) ip_tables(F)
x_tables(F) bridge(F) stp(F) llc(F) dm_crypt(F) isofs(F) microcode(F)
acpiphp(F) ghash_clmulni_intel(F) aesni_intel(F) aes_x86_64(F) xts(F)
lrw(F) gf128mul(F) ablk_helper(F) cryptd(F)
[3587596.471138] CPU 0
[3587596.471144] Pid: 1358, comm: sh Tainted: GF
3.8.0-6-generic #11-Ubuntu
[3587596.471152] RIP: e030:[<ffffffff816c7c2c>] [<ffffffff816c7c2c>]
vmalloc_fault+0x1dc/0x20e
[3587596.471171] RSP: e02b:ffff880066605a20 EFLAGS: 00010046
[3587596.471177] RAX: ffff8800031e5ff8 RBX: ffffe8ffffc01080 RCX:
0000000000000000
[3587596.471185] RDX: 00003ffffffff000 RSI: ffff880000000ff8 RDI:
0000000000000000
[3587596.471192] RBP: ffff880066605a40 R08: ffff880065caa0b8 R09:
000000000061a000
[3587596.471198] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff880002c21e88
[3587596.471205] R13: ffff8800031e5ff8 R14: ffff880000000ff8 R15:
0000000000000000
[3587596.471218] FS: 00007fbc62816740(0000) GS:ffff880069c00000(0000)
knlGS:0000000000000000
[3587596.471226] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[3587596.471232] CR2: ffffe8ffffc01080 CR3: 0000000002c21000 CR4:
0000000000002660
[3587596.471239] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[3587596.471247] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[3587596.471254] Process sh (pid: 1358, threadinfo ffff880066604000,
task ffff88006660c5c0)
[3587596.471260] Stack:
[3587596.471264] 0000000000000029 0000000000000002 ffffe8ffffc01080
ffff880066605b68
[3587596.471276] ffff880066605b38 ffffffff816c7f5c ffff880069c17220
ffff88006660c5c0
[3587596.471287] 0000000000000068 0000000000000000 002284d000000000
0000000000000200
[3587596.471297] Call Trace:
[3587596.471306] [<ffffffff816c7f5c>] __do_page_fault+0x1ec/0x4e0
[3587596.471317] [<ffffffff8100508c>] ? xen_mc_extend_args+0xfc/0x120
[3587596.471326] [<ffffffff816c827b>] do_page_fault+0x2b/0x50
[3587596.471334] [<ffffffff816c4818>] page_fault+0x28/0x30
[3587596.471344] [<ffffffff81187c24>] ?
mem_cgroup_charge_statistics.isra.20+0x14/0x50
[3587596.471354] [<ffffffff81189cd0>]
__mem_cgroup_uncharge_common+0xd0/0x2d0
[3587596.471363] [<ffffffff8118d21a>] mem_cgroup_uncharge_page+0x2a/0x30
[3587596.471371] [<ffffffff81162a09>] page_remove_rmap+0x89/0x160
[3587596.471380] [<ffffffff811568c4>] unmap_page_range+0x4a4/0x750
[3587596.471388] [<ffffffff8100122a>] ? xen_hypercall_xen_version+0xa/0x20
[3587596.471397] [<ffffffff81156bfa>] unmap_single_vma+0x8a/0x100
[3587596.471404] [<ffffffff81157459>] unmap_vmas+0x49/0x90
[3587596.471412] [<ffffffff8115f808>] exit_mmap+0x98/0x170
[3587596.471422] [<ffffffff81081580>] ? lock_hrtimer_base.isra.21+0x30/0x60
[3587596.471430] [<ffffffff816c3fbe>] ?
_raw_spin_unlock_irqrestore+0x1e/0x30
[3587596.471441] [<ffffffff810559a4>] mmput+0x64/0x100
[3587596.471449] [<ffffffff8105e2d2>] do_exit+0x242/0x9d0
[3587596.471456] [<ffffffff8105eadf>] do_group_exit+0x3f/0xa0
[3587596.471462] [<ffffffff8105eb57>] sys_exit_group+0x17/0x20
[3587596.471472] [<ffffffff816cc89d>] system_call_fastpath+0x1a/0x1f
[3587596.471478] Code: de 48 89 d7 e8 a6 ef fe ff 48 f7 00 01 01 00 00
49 89 c5 74 3c 4c 89 f7 48 89 de e8 8f ef fe ff 48 8b 38 f7 c7 01 01 00
00 75 02 <0f> 0b ff 14 25 a8 35 c2 81 48 89 c2 49 8b 7d 00 ff 14 25 a8 35
[3587596.471553] RIP [<ffffffff816c7c2c>] vmalloc_fault+0x1dc/0x20e
[3587596.471562] RSP <ffff880066605a20>
[3587596.471569] ---[ end trace dc5c4320e1320f1c ]---
[3587596.471575] Fixing recursive fault but reboot is needed!
[3587596.471585] ------------[ cut here ]------------
[3587596.471591] Kernel BUG at ffffffff816c7c2c [verbose debug info
unavailable]
[3587596.471598] invalid opcode: 0000 [#2] SMP
[3587596.471604] Modules linked in: xt_multiport(F) iptable_filter(F)
veth(F) ipt_MASQUERADE(F) iptable_nat(F) nf_conntrack_ipv4(F)
nf_defrag_ipv4(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack(F) ip_tables(F)
x_tables(F) bridge(F) stp(F) llc(F) dm_crypt(F) isofs(F) microcode(F)
acpiphp(F) ghash_clmulni_intel(F) aesni_intel(F) aes_x86_64(F) xts(F)
lrw(F) gf128mul(F) ablk_helper(F) cryptd(F)
[3587596.471654] CPU 0
[3587596.471658] Pid: 1358, comm: sh Tainted: GF D
3.8.0-6-generic #11-Ubuntu
[3587596.471666] RIP: e030:[<ffffffff816c7c2c>] [<ffffffff816c7c2c>]
vmalloc_fault+0x1dc/0x20e
[3587596.471677] RSP: e02b:ffff880066605420 EFLAGS: 00010046
[3587596.471682] RAX: ffff8800031e5ff8 RBX: ffffe8ffffc00fc0 RCX:
0000000000000020
[3587596.471689] RDX: 00003ffffffff000 RSI: ffff880000000ff8 RDI:
0000000000000000
[3587596.471696] RBP: ffff880066605440 R08: 0000000000000242 R09:
0000000000000000
[3587596.471703] R10: 0000000000000156 R11: 0720072007200720 R12:
ffff880002c21e88
[3587596.471712] R13: ffff8800031e5ff8 R14: ffff880000000ff8 R15:
0000000000000000
[3587596.471721] FS: 00007fbc62816740(0000) GS:ffff880069c00000(0000)
knlGS:0000000000000000
[3587596.471730] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[3587596.471736] CR2: ffffe8ffffc00fc0 CR3: 0000000002c21000 CR4:
0000000000002660
[3587596.471743] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[3587596.471749] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[3587596.471756] Process sh (pid: 1358, threadinfo ffff880066604000,
task ffff88006660c5c0)
[3587596.471763] Stack:
[3587596.471766] 0000000000000029 0000000000000002 ffffe8ffffc00fc0
ffff880066605568
[3587596.471777] ffff880066605538 ffffffff816c7f5c ffffffffffffffff
ffff88006660c5c0
[3587596.471787] 0000000000000068 ffffffff81e6be49 ffffffffffffffff
ffff880066605530
[3587596.471798] Call Trace:
[3587596.471805] [<ffffffff816c7f5c>] __do_page_fault+0x1ec/0x4e0
[3587596.471814] [<ffffffff8134fec2>] ? number.isra.2+0x322/0x360
[3587596.471821] [<ffffffff8134efb2>] ? put_dec+0x72/0x90
[3587596.471828] [<ffffffff8134fec2>] ? number.isra.2+0x322/0x360
[3587596.471838] [<ffffffff816c827b>] do_page_fault+0x2b/0x50
[3587596.471847] [<ffffffff816c4818>] page_fault+0x28/0x30
[3587596.471858] [<ffffffff8109147c>] ? cpuacct_charge+0x3c/0x60
[3587596.471866] [<ffffffff81094d6c>] update_curr+0xec/0x170
[3587596.471873] [<ffffffff810951b3>] dequeue_entity+0x33/0x4b0
[3587596.471880] [<ffffffff816c3fbe>] ?
_raw_spin_unlock_irqrestore+0x1e/0x30
[3587596.471888] [<ffffffff810957c5>] dequeue_task_fair+0x45/0x510
[3587596.471896] [<ffffffff8108bdbe>] dequeue_task+0x8e/0xa0
[3587596.471904] [<ffffffff8108c823>] deactivate_task+0x23/0x30
[3587596.471911] [<ffffffff816c2b19>] __schedule+0x5b9/0x7b0
[3587596.471918] [<ffffffff816c3059>] schedule+0x29/0x70
[3587596.471924] [<ffffffff8105ea19>] do_exit+0x989/0x9d0
[3587596.471932] [<ffffffff816b7a90>] ? printk+0x67/0x69
[3587596.471940] [<ffffffff816c5446>] oops_end+0xa6/0xf0
[3587596.471950] [<ffffffff81017c18>] die+0x58/0x90
[3587596.471957] [<ffffffff816c4c7b>] do_trap+0x6b/0x170
[3587596.471965] [<ffffffff816c8346>] ?
atomic_notifier_call_chain+0x16/0x20
[3587596.471974] [<ffffffff81015321>] do_invalid_op+0xa1/0xc0
[3587596.471982] [<ffffffff816c7c2c>] ? vmalloc_fault+0x1dc/0x20e
[3587596.471990] [<ffffffff8134ae1e>] ? radix_tree_lookup_slot+0xe/0x10
[3587596.471998] [<ffffffff8100508c>] ? xen_mc_extend_args+0xfc/0x120
[3587596.472006] [<ffffffff816c3f6e>] ? _raw_spin_lock+0xe/0x20
[3587596.472016] [<ffffffff811c573a>] ? __find_get_block_slow+0xda/0x190
[3587596.472025] [<ffffffff816cdb1e>] invalid_op+0x1e/0x30
[3587596.472032] [<ffffffff816c7c2c>] ? vmalloc_fault+0x1dc/0x20e
[3587596.472039] [<ffffffff816c7f5c>] __do_page_fault+0x1ec/0x4e0
[3587596.472052] [<ffffffff8100508c>] ? xen_mc_extend_args+0xfc/0x120
[3587596.472061] [<ffffffff816c827b>] do_page_fault+0x2b/0x50
[3587596.472068] [<ffffffff816c4818>] page_fault+0x28/0x30
[3587596.472076] [<ffffffff81187c24>] ?
mem_cgroup_charge_statistics.isra.20+0x14/0x50
[3587596.472085] [<ffffffff81189cd0>]
__mem_cgroup_uncharge_common+0xd0/0x2d0
[3587596.472093] [<ffffffff8118d21a>] mem_cgroup_uncharge_page+0x2a/0x30
[3587596.472101] [<ffffffff81162a09>] page_remove_rmap+0x89/0x160
[3587596.472109] [<ffffffff811568c4>] unmap_page_range+0x4a4/0x750
[3587596.472116] [<ffffffff8100122a>] ? xen_hypercall_xen_version+0xa/0x20
[3587596.472123] [<ffffffff81156bfa>] unmap_single_vma+0x8a/0x100
[3587596.472130] [<ffffffff81157459>] unmap_vmas+0x49/0x90
[3587596.472138] [<ffffffff8115f808>] exit_mmap+0x98/0x170
[3587596.472146] [<ffffffff81081580>] ? lock_hrtimer_base.isra.21+0x30/0x60
[3587596.472153] [<ffffffff816c3fbe>] ?
_raw_spin_unlock_irqrestore+0x1e/0x30
[3587596.472161] [<ffffffff810559a4>] mmput+0x64/0x100
[3587596.472168] [<ffffffff8105e2d2>] do_exit+0x242/0x9d0
[3587596.472176] [<ffffffff8105eadf>] do_group_exit+0x3f/0xa0
[3587596.472182] [<ffffffff8105eb57>] sys_exit_group+0x17/0x20
[3587596.472189] [<ffffffff816cc89d>] system_call_fastpath+0x1a/0x1f
[3587596.472195] Code: de 48 89 d7 e8 a6 ef fe ff 48 f7 00 01 01 00 00
49 89 c5 74 3c 4c 89 f7 48 89 de e8 8f ef fe ff 48 8b 38 f7 c7 01 01 00
00 75 02 <0f> 0b ff 14 25 a8 35 c2 81 48 89 c2 49 8b 7d 00 ff 14 25 a8 35
[3587596.472269] RIP [<ffffffff816c7c2c>] vmalloc_fault+0x1dc/0x20e
[3587596.472277] RSP <ffff880066605420>
[3587596.472282] ---[ end trace dc5c4320e1320f1d ]---
[3587596.472292] Fixing recursive fault but reboot is needed!
More information about the lxc-users
mailing list