Complete report #742716
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a6885
Build id: 3.10.0-514.el7.x86_64
|
0xa5
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a6cb5
Build id: 3.10.0-514.el7.x86_64
|
0x215
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e3969
Build id: 3.10.0-514.el7.x86_64
|
0x219
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a6aa0
Build id: 3.10.0-514.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a7bb9
Build id: 3.10.0-514.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a7dac
Build id: 3.10.0-514.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8161
Build id: 3.10.0-514.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8168139f
Build id: 3.10.0-514.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff81681873
Build id: 3.10.0-514.el7.x86_64
|
0x3ea
|
|
10
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118af55
Build id: 3.10.0-514.el7.x86_64
|
0x405
|
|
11
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8983
Build id: 3.10.0-514.el7.x86_64
|
0x433
|
|
12
|
|
vmlinux |
0xffffffff8168900d
Build id: 3.10.0-514.el7.x86_64
|
0x17d
|
|
13
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
14
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
15
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
16
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
17
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
18
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
19
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#50 stuck for 22s! [khugepaged:554]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun rdma_ucm(OE) ip6t_rpfilter ib_ucm(OE) ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack rdma_cm(OE) iw_cm(OE) ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 ib_srp(OE) nf_nat_ipv6 scsi_transport_srp(OE) ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat ib_ipoib(OE) ib_cm(OE) iptable_mangle iptable_security iptable_raw nf_conntrack ib_uverbs(OE) ib_umad(OE) ip_set nfnetlink ebtable_filter ebtables mlx5_fpga_tools(OE) mlx5_ib(OE) mlx5_core(OE) mlxfw(OE) ip6table_filter ip6_tables mlx4_en(OE) iptable_filter dm_mirror dm_region_hash dm_log dm_mod raid456 async_raid6_recov async_memcpy async_pq
raid6_pq libcrc32c async_xor xor async_tx iTCO_wdt iTCO_vendor_support intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg i2c_i801 lpc_ich shpchp wmi ipmi_devintf ipmi_si ipmi_msghandler acpi_pad acpi_power_meter acpi_cpufreq sch_fq_codel nfsd knem(OE) auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 mlx4_ib(OE) ib_core(OE) sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper igb mlx4_core(OE) syscopyarea devlink ptp sysfillrect mlx_compat(OE) sysimgblt pps_core fb_sys_fops dca ttm i2c_algo_bit drm ahci libahci i2c_core libata nfit libnvdimm fjes fuse
CPU: 50 PID: 554 Comm: khugepaged Tainted: G OE ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: Supermicro SYS-2049U-TR4/X11QPH+, BIOS 2.1 07/23/2018
task: ffff8817de818fb0 ti: ffff8817de83c000 task.ti: ffff8817de83c000
RIP: 0010:[<ffffffff811a6885>] [<ffffffff811a6885>] isolate_freepages_block+0xa5/0x2c0
RSP: 0018:ffff8817de83f8b8 EFLAGS: 00000216
RAX: 00000000ffffffff RBX: 0000000000000003 RCX: ffff8817de83faa0
RDX: 0000000004e47c00 RSI: ffff8817de83faf8 RDI: ffff8817de83faa0
RBP: ffff8817de83f948 R08: 0000000000000000 R09: ffff88603ffd7000
R10: 0000000004840400 R11: 0000000006040000 R12: ffff88603fbcc0a0
R13: ffff88603ffd7000 R14: 0000000004840400 R15: 0000000006040000
FS: 0000000000000000(0000) GS:ffff882fde800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000044becb CR3: 00000000019ba000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000006040000 00000000000000f8 ffff8817de83f998 00000000000000f8
ffff8817de83faa0 ffff8817de83fae0 ffff8817de83faf8 ffff8817de83faa0
00efffff00084000 0000000000000000 0000000004e47c00 0000000000000001
Call Trace:
[<ffffffff811a6cb5>] compaction_alloc+0x215/0x290
[<ffffffff811e3969>] migrate_pages+0x219/0x790
[<ffffffff811a6aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a7bb9>] compact_zone+0x2d9/0x430
[<ffffffff811a7dac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8161>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8168139f>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff81681873>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e8983>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff8168900d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: a0 48 89 4d 90 45 89 c5 83 f8 80 74 2e 45 84 ed 0f 85 10 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 fe 00 00 00 8b 43 18 <41> 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b 45 a8 0f
Complete report #744135
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
compaction_alloc
|
vmlinux |
0xffffffff811642f8
Build id: 3.10.0-123.el7.x86_64
|
0xf8
|
|
2
|
migrate_pages
|
vmlinux |
0xffffffff8119ac44
Build id: 3.10.0-123.el7.x86_64
|
0xd4
|
|
3
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff81164200
Build id: 3.10.0-123.el7.x86_64
|
0x380
|
|
4
|
compact_zone
|
vmlinux |
0xffffffff81165199
Build id: 3.10.0-123.el7.x86_64
|
0x299
|
|
5
|
compact_zone_order
|
vmlinux |
0xffffffff8116539c
Build id: 3.10.0-123.el7.x86_64
|
0x9c
|
|
6
|
try_to_compact_pages
|
vmlinux |
0xffffffff81165721
Build id: 3.10.0-123.el7.x86_64
|
0xf1
|
|
7
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff815dd419
Build id: 3.10.0-123.el7.x86_64
|
0xac
|
|
8
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8114b259
Build id: 3.10.0-123.el7.x86_64
|
0x6e9
|
|
9
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff8119d4b9
Build id: 3.10.0-123.el7.x86_64
|
0x3c9
|
|
10
|
|
vmlinux |
0xffffffff815e502d
Build id: 3.10.0-123.el7.x86_64
|
0x17d
|
|
11
|
khugepaged
|
vmlinux |
0xffffffff8119dfdf
Build id: 3.10.0-123.el7.x86_64
|
0x25f
|
|
12
|
|
vmlinux |
0xffffffff81086ab0
Build id: 3.10.0-123.el7.x86_64
|
0x30
|
|
13
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff8119dd80
Build id: 3.10.0-123.el7.x86_64
|
0xc90
|
|
14
|
kthread
|
vmlinux |
0xffffffff81085aef
Build id: 3.10.0-123.el7.x86_64
|
0xcf
|
|
15
|
|
vmlinux |
0xffffffff81085a20
Build id: 3.10.0-123.el7.x86_64
|
0x140
|
|
16
|
ret_from_fork
|
vmlinux |
0xffffffff815f206c
Build id: 3.10.0-123.el7.x86_64
|
0x7c
|
|
17
|
|
vmlinux |
0xffffffff81085a20
Build id: 3.10.0-123.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#0 stuck for 22s! [khugepaged:530]
Modules linked in: ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sg i2c_piix4 virtio_console virtio_balloon mperf serio_raw pcspkr ip_tables xfs libcrc32c sr_mod cdrom virtio_blk virtio_scsi virtio_net ata_generic pata_acpi cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper ttm ata_piix drm virtio_pci libata virtio_ring i2c_core virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 530 Comm: khugepaged Not tainted 3.10.0-123.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff880811aac440 ti: ffff88080edb4000 task.ti: ffff88080edb4000
RIP: 0010:[<ffffffff811642f8>] [<ffffffff811642f8>] compaction_alloc+0xf8/0x240
RSP: 0018:ffff88080edb59b8 EFLAGS: 00010202
RAX: ffff88083ff6a680 RBX: 00000000005a0800 RCX: 0000000000000000
RDX: ffff88083ff6a000 RSI: 0000000000000001 RDI: 0000000000000014
RBP: ffff88080edb59f8 R08: ffff88083fb68580 R09: ffff88083ffbce80
R10: 0000000000104220 R11: 0000000000840000 R12: 00000000005a0800
R13: ffffea0016820000 R14: 0000000000104220 R15: ffffffff81164173
FS: 0000000000000000(0000) GS:ffff88081e200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000082e96000 CR3: 00000000018ce000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000104220 ffff88083ffbce80 0000000000840000 ffffea00041007c0
ffff88080edb5b10 ffffea0004100780 ffffea00041007e0 ffff880811aac440
ffff88080edb5a98 ffffffff8119ac44 ffff880811aac440 0001880800000000
Call Trace:
[<ffffffff8119ac44>] migrate_pages+0xd4/0x600
[<ffffffff81164200>] ? isolate_freepages_block+0x380/0x380
[<ffffffff81165199>] compact_zone+0x299/0x400
[<ffffffff8116539c>] compact_zone_order+0x9c/0xf0
[<ffffffff81165721>] try_to_compact_pages+0xf1/0x120
[<ffffffff815dd419>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff8114b259>] __alloc_pages_nodemask+0x6e9/0xb10
[<ffffffff8119d4b9>] khugepaged_scan_mm_slot+0x3c9/0xc90
[<ffffffff815e502d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff8119dfdf>] khugepaged+0x25f/0x4a0
[<ffffffff81086ab0>] ? wake_up_bit+0x30/0x30
[<ffffffff8119dd80>] ? khugepaged_scan_mm_slot+0xc90/0xc90
[<ffffffff81085aef>] kthread+0xcf/0xe0
[<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff815f206c>] ret_from_fork+0x7c/0xb0
[<ffffffff81085a20>] ? kthread_create_on_node+0x140/0x140
Code: ff ff 07 00 77 4b 4c 89 e2 48 c1 ea 16 48 8b 14 d5 c0 8a dc 81 48 85 d2 74 37 83 e0 7f 48 c1 e0 05 48 01 d0 74 2b f6 00 02 74 26 <49> 8b 45 00 48 89 c2 48 c1 e8 34 83 e0 03 48 c1 ea 36 48 69 c0
Complete report #519825
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a6a85
Build id: 3.10.0-514.10.2.el7.x86_64
|
0xa5
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a6eb5
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x215
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e3b69
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x219
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a6ca0
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a7db9
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a7fac
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8361
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff816818e2
Build id: 3.10.0-514.10.2.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff81682070
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x6a4
|
|
10
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118b155
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x405
|
|
11
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8b83
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x433
|
|
12
|
|
vmlinux |
0xffffffff8168954d
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x17d
|
|
13
|
khugepaged
|
vmlinux |
0xffffffff811e9617
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x257
|
|
14
|
|
vmlinux |
0xffffffff810b17d0
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x30
|
|
15
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e93c0
Build id: 3.10.0-514.10.2.el7.x86_64
|
0xc70
|
|
16
|
kthread
|
vmlinux |
0xffffffff810b06ff
Build id: 3.10.0-514.10.2.el7.x86_64
|
0xcf
|
|
17
|
|
vmlinux |
0xffffffff810b0630
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x140
|
|
18
|
ret_from_fork
|
vmlinux |
0xffffffff81696a58
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x58
|
|
19
|
|
vmlinux |
0xffffffff810b0630
Build id: 3.10.0-514.10.2.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 40s! [gstd:8260]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 40s! [khugepaged:44]
Modules linked in: traps(OE) iptable_filter nsrbbb(POE) vmw_vsock_vmci_transport vsock intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev parport_pc vmw_balloon pcspkr vmw_vmci sg parport i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace binfmt_misc sunrpc ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul vmwgfx crct10dif_common crc32c_intel drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ata_piix libata vmw_pvscsi i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 44 Comm: khugepaged Tainted: P OEL ------------ 3.10.0-514.10.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88042c743ec0 ti: ffff88042c7c4000 task.ti: ffff88042c7c4000
RIP: 0010:[<ffffffff811a6a85>] [<ffffffff811a6a85>] isolate_freepages_block+0xa5/0x2c0
RSP: 0018:ffff88042c7c78b8 EFLAGS: 00000212
RAX: 00000000ffffffff RBX: ffffffff81189b16 RCX: ffff88042c7c7aa0
RDX: 00000000001fca00 RSI: ffff88042c7c7af8 RDI: ffff88042c7c7aa0
RBP: ffff88042c7c7948 R08: 0000000000000000 R09: ffff88043ffcd000
R10: 00000000001ff600 R11: 0000000000440000 R12: ffff88042c7c78a0
R13: ffff88042c7c7880 R14: 0000000000000000 R15: ffffea000499b000
FS: 0000000000000000(0000) GS:ffff88043fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f25333bc000 CR3: 000000041c33b000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00000000001fca50 ffffea0007f28000 ffffea0007f29400 0000000100000000
ffff88042c7c7aa0 ffff88042c7c7ae0 ffff88042c7c7af8 ffff88042c7c7aa0
ffff88043ffcd000 0000000000000000 00000000001fca00 0000000000000246
Call Trace:
[<ffffffff811a6eb5>] compaction_alloc+0x215/0x290
[<ffffffff811e3b69>] migrate_pages+0x219/0x790
[<ffffffff811a6ca0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a7db9>] compact_zone+0x2d9/0x430
[<ffffffff811a7fac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8361>] try_to_compact_pages+0x121/0x1a0
[<ffffffff816818e2>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff81682070>] __alloc_pages_slowpath+0x6a4/0x725
[<ffffffff8118b155>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e8b83>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff8168954d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff811e9617>] khugepaged+0x257/0x480
[<ffffffff810b17d0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e93c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b06ff>] kthread+0xcf/0xe0
[<ffffffff810b0630>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696a58>] ret_from_fork+0x58/0x90
[<ffffffff810b0630>] ? kthread_create_on_node+0x140/0x140
Code: a0 48 89 4d 90 45 89 c5 83 f8 80 74 2e 45 84 ed 0f 85 10 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 fe 00 00 00 8b 43 18 <41> 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b 45 a8 0f
Complete report #173396
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff8118df8c
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
2
|
?
free_hot_cold_page_list
|
vmlinux |
0xffffffff81171b46
Build id: 3.10.0-327.el7.x86_64
|
0x46
|
|
3
|
compaction_alloc
|
vmlinux |
0xffffffff8118e42f
Build id: 3.10.0-327.el7.x86_64
|
0x1cf
|
|
4
|
migrate_pages
|
vmlinux |
0xffffffff811c71f9
Build id: 3.10.0-327.el7.x86_64
|
0x1e9
|
|
5
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff8118e260
Build id: 3.10.0-327.el7.x86_64
|
0x380
|
|
6
|
compact_zone
|
vmlinux |
0xffffffff8118f259
Build id: 3.10.0-327.el7.x86_64
|
0x299
|
|
7
|
compact_zone_order
|
vmlinux |
0xffffffff8118f45c
Build id: 3.10.0-327.el7.x86_64
|
0x9c
|
|
8
|
try_to_compact_pages
|
vmlinux |
0xffffffff8118f811
Build id: 3.10.0-327.el7.x86_64
|
0x121
|
|
9
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff816305c8
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
10
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff811734e8
Build id: 3.10.0-327.el7.x86_64
|
0x788
|
|
11
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c9941
Build id: 3.10.0-327.el7.x86_64
|
0x3f1
|
|
12
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
13
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
14
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
15
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
16
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
17
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
18
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#0 stuck for 21s! [khugepaged:309]
Modules linked in: tcp_lp bnep bluetooth rfkill fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event ppdev vmw_balloon crc32_pclmul ghash_clmulni_intel snd_ens1371 aesni_intel snd_rawmidi snd_ac97_codec lrw gf128mul ac97_bus glue_helper snd_seq ablk_helper cryptd snd_seq_device snd_pcm snd_timer snd soundcore sg pcspkr parport_pc parport vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace binfmt_misc sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif
crct10dif_generic pata_acpi vmwgfx drm_kms_helper ttm mptspi scsi_transport_spi mptscsih mptbase crct10dif_pclmul crct10dif_common crc32c_intel drm serio_raw ata_piix e1000 libata i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 309 Comm: khugepaged Tainted: G L ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff88061b9ce780 ti: ffff880616890000 task.ti: ffff880616890000
RIP: 0010:[<ffffffff8118df8c>] [<ffffffff8118df8c>] isolate_freepages_block+0xac/0x380
RSP: 0018:ffff880616893908 EFLAGS: 00010286
RAX: 00000000ffffffff RBX: ffffea00040352c0 RCX: ffff880616893b58
RDX: 00000000004e8000 RSI: 00000000004e7e00 RDI: ffff880616893b00
RBP: ffff8806168939a0 R08: 0000000000000000 R09: ffff88063ffd8000
R10: 00000000139f8000 R11: 0000000000640000 R12: 0000000000000297
R13: ffff8806168938a0 R14: 0000000000000000 R15: ffff880626697bc0
FS: 0000000000000000(0000) GS:ffff880626600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f090f8e0000 CR3: 000000012f6a0000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880616893960 ffff880616893980 ffff880616893950 ffffffff81171b46
00000000004e7e00 ffff880616893b00 0000000000640000 0000000000000000
ffff880616893b40 ffff880616893b58 ffff880616893b00 0000000000000000
Call Trace:
[<ffffffff81171b46>] ? free_hot_cold_page_list+0x46/0xa0
[<ffffffff8118e42f>] compaction_alloc+0x1cf/0x240
[<ffffffff811c71f9>] migrate_pages+0x1e9/0x790
[<ffffffff8118e260>] ? isolate_freepages_block+0x380/0x380
[<ffffffff8118f259>] compact_zone+0x299/0x400
[<ffffffff8118f45c>] compact_zone_order+0x9c/0xf0
[<ffffffff8118f811>] try_to_compact_pages+0x121/0x1a0
[<ffffffff816305c8>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff811734e8>] __alloc_pages_nodemask+0x788/0xb90
[<ffffffff811c9941>] khugepaged_scan_mm_slot+0x3f1/0xca0
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: c0 00 4d 89 ee 48 89 4d b0 41 89 c5 eb 1d 90 49 83 c7 01 48 83 c3 40 4d 39 fc 0f 86 07 01 00 00 41 83 c5 01 4d 85 f6 4c 0f 44 f3 <8b> 43 18 83 f8 80 75 dc 48 8b 45 b8 0f b6 55 c0 48 8d 75 c8 4c
Complete report #319297
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff8118df8f
Build id: 3.10.0-327.el7.x86_64
|
0xaf
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff8118e42f
Build id: 3.10.0-327.el7.x86_64
|
0x1cf
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811c71f9
Build id: 3.10.0-327.el7.x86_64
|
0x1e9
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff8118e260
Build id: 3.10.0-327.el7.x86_64
|
0x380
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff8118f259
Build id: 3.10.0-327.el7.x86_64
|
0x299
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff8118f45c
Build id: 3.10.0-327.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff8118f811
Build id: 3.10.0-327.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff816305c8
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff811734e8
Build id: 3.10.0-327.el7.x86_64
|
0x788
|
|
10
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c9941
Build id: 3.10.0-327.el7.x86_64
|
0x3f1
|
|
11
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
12
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
13
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
14
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
15
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
16
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
17
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#61 stuck for 22s! [khugepaged:473]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm svcrdma xprtrdma sunrpc rdma_ucm ib_uverbs ib_iser ib_umad rdma_cm ib_ipoib iw_cm libiscsi ib_cm scsi_transport_iscsi mlx4_ib ib_sa ib_mad ib_core ib_addr iTCO_wdt iTCO_vendor_support fuse intel_powerclamp coretemp intel_rapl snd_hda_codec_hdmi kvm_intel kvm crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) pcspkr snd_hda_intel snd_hda_codec
snd_hda_core snd_hwdep snd_seq snd_seq_device sb_edac sg edac_core snd_pcm mei_me snd_timer mei i2c_i801 drm snd lpc_ich mfd_core soundcore ipmi_ssif ipmi_msghandler shpchp acpi_power_meter acpi_pad binfmt_misc ip_tables xfs libcrc32c mlx4_en vxlan ip6_udp_tunnel udp_tunnel sd_mod crc_t10dif crct10dif_generic serio_raw crct10dif_pclmul crct10dif_common crc32c_intel mlx4_core igb ptp pps_core dca i2c_algo_bit i2c_core ahci libahci libata wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 61 PID: 473 Comm: khugepaged Tainted: P OEL ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: ASUSTeK COMPUTER INC. Z10PA-D8 Series/Z10PA-D8 Series, BIOS 3202 04/18/2016
task: ffff881fd1df4500 ti: ffff881fcec48000 task.ti: ffff881fcec48000
RIP: 0010:[<ffffffff8118df8f>] [<ffffffff8118df8f>] isolate_freepages_block+0xaf/0x380
RSP: 0018:ffff881fcec4b908 EFLAGS: 00000286
RAX: 00000000ffffffff RBX: ffff881fcec4bb40 RCX: ffff881fcec4bb58
RDX: 000000000056ee00 RSI: 000000000056ec00 RDI: ffff881fcec4bb00
RBP: ffff881fcec4b9a0 R08: 0000000000000000 R09: ffff88207ffda000
R10: 0000000015bb0000 R11: 0000000002080000 R12: ffff881fcec4bb58
R13: ffff88207ffda000 R14: 00000000193b0000 R15: 0000000002080000
FS: 0000000000000000(0000) GS:ffff883fff5a0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001e91283df8 CR3: 000000000194a000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff88207ffda000 0000000000000000 00000000005d3400 ffff881fcec4bb00
000000000056ec00 ffff881fcec4bb00 0000000002080000 0000000000000000
ffff881fcec4bb40 ffff881fcec4bb58 ffff881fcec4bb00 ffff88207ffda000
Call Trace:
[<ffffffff8118e42f>] compaction_alloc+0x1cf/0x240
[<ffffffff811c71f9>] migrate_pages+0x1e9/0x790
[<ffffffff8118e260>] ? isolate_freepages_block+0x380/0x380
[<ffffffff8118f259>] compact_zone+0x299/0x400
[<ffffffff8118f45c>] compact_zone_order+0x9c/0xf0
[<ffffffff8118f811>] try_to_compact_pages+0x121/0x1a0
[<ffffffff816305c8>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff811734e8>] __alloc_pages_nodemask+0x788/0xb90
[<ffffffff811c9941>] khugepaged_scan_mm_slot+0x3f1/0xca0
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: 89 ee 48 89 4d b0 41 89 c5 eb 1d 90 49 83 c7 01 48 83 c3 40 4d 39 fc 0f 86 07 01 00 00 41 83 c5 01 4d 85 f6 4c 0f 44 f3 8b 43 18 <83> f8 80 75 dc 48 8b 45 b8 0f b6 55 c0 48 8d 75 c8 4c 8b 45 b0
Complete report #197703
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a6882
Build id: 3.10.0-514.el7.x86_64
|
0xa2
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a6cb5
Build id: 3.10.0-514.el7.x86_64
|
0x215
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e3969
Build id: 3.10.0-514.el7.x86_64
|
0x219
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a6aa0
Build id: 3.10.0-514.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a7bb9
Build id: 3.10.0-514.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a7dac
Build id: 3.10.0-514.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8161
Build id: 3.10.0-514.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8168139f
Build id: 3.10.0-514.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff81681873
Build id: 3.10.0-514.el7.x86_64
|
0x3ea
|
|
10
|
|
vmlinux |
0xffffffff8118ef5e
Build id: 3.10.0-514.el7.x86_64
|
0x24e
|
|
11
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118af55
Build id: 3.10.0-514.el7.x86_64
|
0x405
|
|
12
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8983
Build id: 3.10.0-514.el7.x86_64
|
0x433
|
|
13
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
14
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
15
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
16
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
17
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
18
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
19
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [khugepaged:28]
Modules linked in: binfmt_misc rfcomm bnep vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp ppdev snd_ens1371 iosf_mbi crc32_pclmul ghash_clmulni_intel snd_rawmidi snd_ac97_codec ac97_bus aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_seq snd_seq_device vmw_balloon pcspkr snd_pcm btusb btrtl btbcm btintel bluetooth rfkill snd_timer snd nfit soundcore sg libnvdimm parport_pc parport i2c_piix4 shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix vmwgfx e1000 mptspi scsi_transport_spi mptscsih mptbase drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm i2c_core libata
fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 28 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff880137202f10 ti: ffff880131c20000 task.ti: ffff880131c20000
RIP: 0010:[<ffffffff811a6882>] [<ffffffff811a6882>] isolate_freepages_block+0xa2/0x2c0
RSP: 0018:ffff880131c238b8 EFLAGS: 00010206
RAX: 00000000ffffffff RBX: ffffffff81189916 RCX: ffff880131c23aa0
RDX: 000000000006f800 RSI: ffff880131c23af8 RDI: ffff880131c23aa0
RBP: ffff880131c23948 R08: 0000000000000000 R09: ffff88013ffd7800
R10: 0000000000001e00 R11: 0000000000100000 R12: ffff880131c238a0
R13: ffff880131c23880 R14: 0000000000000000 R15: ffffea000430d2c0
FS: 0000000000000000(0000) GS:ffff880139600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f516d794000 CR3: 00000000708de000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffff880131c23998 ffff880134cd3e80
ffff880131c23aa0 ffff880131c23ae0 ffff880131c23af8 ffff880131c23aa0
ffff88013ffd7800 0000000000000000 000000000006f800 0000000000000003
Call Trace:
[<ffffffff811a6cb5>] compaction_alloc+0x215/0x290
[<ffffffff811e3969>] migrate_pages+0x219/0x790
[<ffffffff811a6aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a7bb9>] compact_zone+0x2d9/0x430
[<ffffffff811a7dac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8161>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8168139f>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff81681873>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118ef5e>] ? release_pages+0x24e/0x430
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e8983>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 48 89 75 a0 48 89 4d 90 45 89 c5 83 f8 80 74 2e 45 84 ed 0f 85 10 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 fe 00 00 00 <8b> 43 18 41 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b
Complete report #159381
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff8118df8c
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff8118e42f
Build id: 3.10.0-327.el7.x86_64
|
0x1cf
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811c71f9
Build id: 3.10.0-327.el7.x86_64
|
0x1e9
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff8118e260
Build id: 3.10.0-327.el7.x86_64
|
0x380
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff8118f259
Build id: 3.10.0-327.el7.x86_64
|
0x299
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff8118f45c
Build id: 3.10.0-327.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff8118f811
Build id: 3.10.0-327.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff816305c8
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff811734e8
Build id: 3.10.0-327.el7.x86_64
|
0x788
|
|
10
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c9941
Build id: 3.10.0-327.el7.x86_64
|
0x3f1
|
|
11
|
|
vmlinux |
0xffffffff8163d438
Build id: 3.10.0-327.el7.x86_64
|
0x28
|
|
12
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
13
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
14
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
15
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
16
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
17
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
18
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#6 stuck for 22s! [khugepaged:78]
Modules linked in: tcp_lp binfmt_misc ext4 mbcache jbd2 bnep bluetooth rfkill fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter kvm_intel kvm crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd i2c_piix4 virtio_balloon pcspkr parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi virtio_net virtio_blk cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper ttm virtio_pci crct10dif_pclmul drm virtio_ring crct10dif_common ata_piix libata crc32c_intel serio_raw i2c_core virtio floppy dm_mirror
dm_region_hash dm_log dm_mod
CPU: 6 PID: 78 Comm: khugepaged Tainted: G L ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: OpenStack Foundation OpenStack Nova, BIOS Bochs 01/01/2011
task: ffff88069b2a2280 ti: ffff880696660000 task.ti: ffff880696660000
RIP: 0010:[<ffffffff8118df8c>] [<ffffffff8118df8c>] isolate_freepages_block+0xac/0x380
RSP: 0018:ffff880696663908 EFLAGS: 00010286
RAX: 00000000ffffffff RBX: ffffea0011092000 RCX: ffff880696663b58
RDX: 0000000000442600 RSI: 0000000000442400 RDI: ffff880696663b00
RBP: ffff8806966639a0 R08: 0000000000000000 R09: ffff8806bffd8000
R10: 0000000011090000 R11: 00000000006c0000 R12: 0000000000442600
R13: 0000000000000081 R14: ffffea0011090000 R15: 0000000000442480
FS: 0000000000000000(0000) GS:ffff8806bfd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000e9a53006 CR3: 000000000194a000 CR4: 00000000001406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00000000006c0000 00000000000000a0 ffff8806bfb846a0 00000000ffffffff
0000000000442400 ffff880696663b00 0000000000000002 0000000000000000
ffff880696663b40 ffff880696663b58 ffff880696663b00 0000000000000000
Call Trace:
[<ffffffff8118e42f>] compaction_alloc+0x1cf/0x240
[<ffffffff811c71f9>] migrate_pages+0x1e9/0x790
[<ffffffff8118e260>] ? isolate_freepages_block+0x380/0x380
[<ffffffff8118f259>] compact_zone+0x299/0x400
[<ffffffff8118f45c>] compact_zone_order+0x9c/0xf0
[<ffffffff8118f811>] try_to_compact_pages+0x121/0x1a0
[<ffffffff816305c8>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff811734e8>] __alloc_pages_nodemask+0x788/0xb90
[<ffffffff811c9941>] khugepaged_scan_mm_slot+0x3f1/0xca0
[<ffffffff8163d438>] ? async_page_fault+0x28/0x30
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: c0 00 4d 89 ee 48 89 4d b0 41 89 c5 eb 1d 90 49 83 c7 01 48 83 c3 40 4d 39 fc 0f 86 07 01 00 00 41 83 c5 01 4d 85 f6 4c 0f 44 f3 <8b> 43 18 83 f8 80 75 dc 48 8b 45 b8 0f b6 55 c0 48 8d 75 c8 4c
Complete report #152891
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a6f82
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xa2
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a73b5
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x215
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e4059
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x219
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a71a0
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x2c0
|
|
5
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a71a0
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x2c0
|
|
6
|
compact_zone
|
vmlinux |
0xffffffff811a82b9
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x2d9
|
|
7
|
compact_zone_order
|
vmlinux |
0xffffffff811a84ac
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x9c
|
|
8
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8861
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x121
|
|
9
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff81682306
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xac
|
|
10
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff816827da
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x3ea
|
|
11
|
|
vmlinux |
0xffffffff8118f64e
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x24e
|
|
12
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118b645
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x405
|
|
13
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9073
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x433
|
|
14
|
|
vmlinux |
0xffffffff81689f5c
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x17c
|
|
15
|
khugepaged
|
vmlinux |
0xffffffff811e9b07
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x257
|
|
16
|
|
vmlinux |
0xffffffff810b1b20
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x30
|
|
17
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e98b0
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xc70
|
|
18
|
kthread
|
vmlinux |
0xffffffff810b0a4f
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xcf
|
|
19
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x140
|
|
20
|
ret_from_fork
|
vmlinux |
0xffffffff81697518
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x58
|
|
21
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [khugepaged:65]
Modules linked in: ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev bochs_drm ttm drm_kms_helper sg syscopyarea pcspkr sysfillrect shpchp sysimgblt fb_sys_fops i2c_piix4 drm virtio_balloon parport_pc i2c_core parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk virtio_console crct10dif_pclmul ata_piix crct10dif_common crc32c_intel libata serio_raw virtio_pci virtio_ring virtio floppy
CPU: 5 PID: 65 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.21.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff88042c658fb0 ti: ffff88042c6bc000 task.ti: ffff88042c6bc000
RIP: 0010:[<ffffffff811a6f82>] [<ffffffff811a6f82>] isolate_freepages_block+0xa2/0x2c0
RSP: 0018:ffff88042c6bf8b8 EFLAGS: 00010206
RAX: 00000000ffffffff RBX: ffffea000de37000 RCX: ffff88042c6bfaa0
RDX: 0000000000378a00 RSI: ffff88042c6bfaf8 RDI: ffff88042c6bfaa0
RBP: ffff88042c6bf948 R08: 0000000000000000 R09: ffff88043ffd8000
R10: 0000000000100200 R11: 0000000000440000 R12: 0000000000378dc0
R13: 0000000000000000 R14: ffffea000de30000 R15: 00000000000001c0
FS: 0000000000000000(0000) GS:ffff88043fd40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000201005 CR3: 00000000362b8000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffea000de80000 000000000037a000 ffff88042c6bf998 ffff88042c6bfaa0
ffff88042c6bfaa0 ffff88042c6bfae0 ffff88042c6bfaf8 ffff88042c6bfaa0
ffff88043ff86d00 0000000000000000 0000000000378a00 0000000000000003
Call Trace:
[<ffffffff811a73b5>] compaction_alloc+0x215/0x290
[<ffffffff811e4059>] migrate_pages+0x219/0x790
[<ffffffff811a71a0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a71a0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a82b9>] compact_zone+0x2d9/0x430
[<ffffffff811a84ac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8861>] try_to_compact_pages+0x121/0x1a0
[<ffffffff81682306>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff816827da>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118f64e>] ? release_pages+0x24e/0x430
[<ffffffff8118b645>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e9073>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff81689f5c>] ? schedule_timeout+0x17c/0x2c0
[<ffffffff811e9b07>] khugepaged+0x257/0x480
[<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e98b0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697518>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: 48 89 75 a0 48 89 4d 90 45 89 c5 83 f8 80 74 2e 45 84 ed 0f 85 10 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 fe 00 00 00 <8b> 43 18 41 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b
Complete report #139427
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff8118df8f
Build id: 3.10.0-327.el7.x86_64
|
0xaf
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff8118e42f
Build id: 3.10.0-327.el7.x86_64
|
0x1cf
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811c71f9
Build id: 3.10.0-327.el7.x86_64
|
0x1e9
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff8118e260
Build id: 3.10.0-327.el7.x86_64
|
0x380
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff8118f259
Build id: 3.10.0-327.el7.x86_64
|
0x299
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff8118f45c
Build id: 3.10.0-327.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff8118f811
Build id: 3.10.0-327.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff816305c8
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff811734e8
Build id: 3.10.0-327.el7.x86_64
|
0x788
|
|
10
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c9941
Build id: 3.10.0-327.el7.x86_64
|
0x3f1
|
|
11
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
12
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
13
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
14
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
15
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
16
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
17
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#18 stuck for 24s! [khugepaged:717]
Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support intel_powerclamp coretemp kvm_intel kvm ext4 mbcache jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif pcspkr sg sb_edac edac_core lpc_ich i2c_i801 mfd_core ipmi_si ipmi_msghandler mei_me mei shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast syscopyarea sysfillrect sysimgblt drm_kms_helper igb ttm ahci megaraid_sas
libahci ptp drm pps_core dca libata i2c_algo_bit i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 18 PID: 717 Comm: khugepaged Tainted: G L ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: Inspur NF8460M3/NF8460M3, BIOS NF8460M3_1.2.8 05/27/2016
task: ffff881fcfcee780 ti: ffff881fcfe6c000 task.ti: ffff881fcfe6c000
RIP: 0010:[<ffffffff8118df8f>] [<ffffffff8118df8f>] isolate_freepages_block+0xaf/0x380
RSP: 0018:ffff881fcfe6f908 EFLAGS: 00000286
RAX: 00000000ffffffff RBX: 00000001270f0000 RCX: ffff881fcfe6fb58
RDX: 00000000047d6c00 RSI: 00000000047d6a00 RDI: ffff881fcfe6fb00
RBP: ffff881fcfe6f9a0 R08: 0000000000000000 R09: ffff88607ffda000
R10: 000000011f5a8000 R11: 0000000006080000 R12: 0000000000000000
R13: ffff881fcfe6f8f8 R14: ffff883fff934780 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff883fff900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7b88fd8ff4 CR3: 000000000194a000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff881fcfe6f9a0 000000000000008c 0000000006080000 000000000408a600
00000000047d6a00 ffff881fcfe6fb00 0000000006080000 0000000000000000
ffff881fcfe6fb40 ffff881fcfe6fb58 ffff881fcfe6fb00 0000000000000000
Call Trace:
[<ffffffff8118e42f>] compaction_alloc+0x1cf/0x240
[<ffffffff811c71f9>] migrate_pages+0x1e9/0x790
[<ffffffff8118e260>] ? isolate_freepages_block+0x380/0x380
[<ffffffff8118f259>] compact_zone+0x299/0x400
[<ffffffff8118f45c>] compact_zone_order+0x9c/0xf0
[<ffffffff8118f811>] try_to_compact_pages+0x121/0x1a0
[<ffffffff816305c8>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff811734e8>] __alloc_pages_nodemask+0x788/0xb90
[<ffffffff811c9941>] khugepaged_scan_mm_slot+0x3f1/0xca0
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: 89 ee 48 89 4d b0 41 89 c5 eb 1d 90 49 83 c7 01 48 83 c3 40 4d 39 fc 0f 86 07 01 00 00 41 83 c5 01 4d 85 f6 4c 0f 44 f3 8b 43 18 <83> f8 80 75 dc 48 8b 45 b8 0f b6 55 c0 48 8d 75 c8 4c 8b 45 b0
Complete report #133872
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
compaction_alloc
|
vmlinux |
0xffffffff811a72d8
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x128
|
|
2
|
migrate_pages
|
vmlinux |
0xffffffff811e4229
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x219
|
|
3
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a71b0
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x2c0
|
|
4
|
compact_zone
|
vmlinux |
0xffffffff811a82c9
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x2d9
|
|
5
|
compact_zone_order
|
vmlinux |
0xffffffff811a84bc
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x9c
|
|
6
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8871
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x121
|
|
7
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff81682526
Build id: 3.10.0-514.26.2.el7.x86_64
|
0xac
|
|
8
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff816829fa
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x3ea
|
|
9
|
|
vmlinux |
0xffffffff8118f65e
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x24e
|
|
10
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118b655
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x405
|
|
11
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9243
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x433
|
|
12
|
khugepaged
|
vmlinux |
0xffffffff811e9cd7
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x257
|
|
13
|
|
vmlinux |
0xffffffff810b1b20
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x30
|
|
14
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9a80
Build id: 3.10.0-514.26.2.el7.x86_64
|
0xc70
|
|
15
|
kthread
|
vmlinux |
0xffffffff810b0a4f
Build id: 3.10.0-514.26.2.el7.x86_64
|
0xcf
|
|
16
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x140
|
|
17
|
ret_from_fork
|
vmlinux |
0xffffffff81697758
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x58
|
|
18
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [khugepaged:27]
Modules linked in: xt_multiport xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr shpchp sg parport_pc
parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc fuse ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 mptspi scsi_transport_spi mptscsih mptbase ata_generic pata_acpi floppy vmwgfx drm_kms_helper fjes syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm i2c_core ata_piix libata
CPU: 0 PID: 27 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.26.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88017a360fb0 ti: ffff88042b470000 task.ti: ffff88042b470000
RIP: 0010:[<ffffffff811a72d8>] [<ffffffff811a72d8>] compaction_alloc+0x128/0x290
RSP: 0018:ffff88042b473958 EFLAGS: 00010202
RAX: ffff88043ffcbdc0 RBX: ffff88043fbc9e40 RCX: 0000000000000000
RDX: ffff88043ffcb000 RSI: 0000000000000001 RDI: 0000000000000084
RBP: ffff88042b473998 R08: ffff88043fbc9cc0 R09: ffff88043ffcd000
R10: 0000000000104a00 R11: 0000000000440000 R12: 0000000000104a00
R13: 0000000000440000 R14: ffffffff811a7174 R15: ffff88042b473948
FS: 0000000000000000(0000) GS:ffff88043fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fe98235bd52 CR3: 00000004052f2000 CR4: 00000000000407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000104a00 ffff88043ffcd000 0000000000440000 ffffea00041209c0
ffff88042b473ab0 ffffea0004120980 ffff88042b473aa0 0000000000000000
ffff88042b473a38 ffffffff811e4229 ffff88017a360fb0 ffff88017a360fb0
Call Trace:
[<ffffffff811e4229>] migrate_pages+0x219/0x790
[<ffffffff811a71b0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a82c9>] compact_zone+0x2d9/0x430
[<ffffffff811a84bc>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8871>] try_to_compact_pages+0x121/0x1a0
[<ffffffff81682526>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff816829fa>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118f65e>] ? release_pages+0x24e/0x430
[<ffffffff8118b655>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e9243>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff811e9cd7>] khugepaged+0x257/0x480
[<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e9a80>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697758>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: ff ff 07 00 77 d1 4c 89 e2 48 c1 ea 16 48 8b 14 d5 80 d5 f4 81 48 85 d2 74 bd 83 e0 7f 48 c1 e0 05 48 01 d0 74 b1 f6 00 02 74 ac <49> 8b 45 00 a9 00 00 00 80 75 85 48 89 c2 48 c1 e8 29 48 c1 ea
Complete report #89873
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a6882
Build id: 3.10.0-514.el7.x86_64
|
0xa2
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a6cb5
Build id: 3.10.0-514.el7.x86_64
|
0x215
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e3969
Build id: 3.10.0-514.el7.x86_64
|
0x219
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a6aa0
Build id: 3.10.0-514.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a7bb9
Build id: 3.10.0-514.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a7dac
Build id: 3.10.0-514.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8161
Build id: 3.10.0-514.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8168139f
Build id: 3.10.0-514.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff81681873
Build id: 3.10.0-514.el7.x86_64
|
0x3ea
|
|
10
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118af55
Build id: 3.10.0-514.el7.x86_64
|
0x405
|
|
11
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8983
Build id: 3.10.0-514.el7.x86_64
|
0x433
|
|
12
|
|
vmlinux |
0xffffffff8168900d
Build id: 3.10.0-514.el7.x86_64
|
0x17d
|
|
13
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
14
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
15
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
16
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
17
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
18
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
19
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [khugepaged:103]
Modules linked in: tcp_lp binfmt_misc veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371
snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm vmw_balloon ppdev snd_timer pcspkr snd shpchp soundcore vmw_vmci sg i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi serio_raw drm scsi_transport_spi ata_piix mptscsih e1000 libata mptbase i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 103 Comm: khugepaged Tainted: G L ------------ T 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
task: ffff880fff43ce70 ti: ffff880ff95d0000 task.ti: ffff880ff95d0000
RIP: 0010:[<ffffffff811a6882>] [<ffffffff811a6882>] isolate_freepages_block+0xa2/0x2c0
RSP: 0018:ffff880ff95d38b8 EFLAGS: 00010206
RAX: 00000000ffffffff RBX: ffffffff8106c4c8 RCX: ffff880ff95d3aa0
RDX: 0000000000cdb000 RSI: ffff880ff95d3af8 RDI: ffff880ff95d3aa0
RBP: ffff880ff95d3948 R08: 0000000000000000 R09: ffff88103ffd8000
R10: 0000000000fc2000 R11: 0000000001040000 R12: ffff8800771c3d88
R13: ffff880087785488 R14: 0000000000000000 R15: 00007f39c9877000
FS: 0000000000000000(0000) GS:ffff880ffd680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f38cb3dffd8 CR3: 00000000019ba000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000fbf952 ffffea003efe0000 ffff880ff95d3998 0000000100000000
ffff880ff95d3aa0 ffff880ff95d3ae0 ffff880ff95d3af8 ffff880ff95d3aa0
ffff88103ffd8000 0000000000000000 0000000000cdb000 0000000000000001
Call Trace:
[<ffffffff811a6cb5>] compaction_alloc+0x215/0x290
[<ffffffff811e3969>] migrate_pages+0x219/0x790
[<ffffffff811a6aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a7bb9>] compact_zone+0x2d9/0x430
[<ffffffff811a7dac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8161>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8168139f>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff81681873>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e8983>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff8168900d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 48 89 75 a0 48 89 4d 90 45 89 c5 83 f8 80 74 2e 45 84 ed 0f 85 10 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 fe 00 00 00 <8b> 43 18 41 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b
Complete report #19829
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
compaction_alloc
|
vmlinux |
0xffffffff811a6bc8
Build id: 3.10.0-514.el7.x86_64
|
0x128
|
|
2
|
migrate_pages
|
vmlinux |
0xffffffff811e3969
Build id: 3.10.0-514.el7.x86_64
|
0x219
|
|
3
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a6aa0
Build id: 3.10.0-514.el7.x86_64
|
0x2c0
|
|
4
|
compact_zone
|
vmlinux |
0xffffffff811a7bb9
Build id: 3.10.0-514.el7.x86_64
|
0x2d9
|
|
5
|
compact_zone_order
|
vmlinux |
0xffffffff811a7dac
Build id: 3.10.0-514.el7.x86_64
|
0x9c
|
|
6
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8161
Build id: 3.10.0-514.el7.x86_64
|
0x121
|
|
7
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8168139f
Build id: 3.10.0-514.el7.x86_64
|
0xac
|
|
8
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff81681873
Build id: 3.10.0-514.el7.x86_64
|
0x3ea
|
|
9
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118af55
Build id: 3.10.0-514.el7.x86_64
|
0x405
|
|
10
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8983
Build id: 3.10.0-514.el7.x86_64
|
0x433
|
|
11
|
|
vmlinux |
0xffffffff8168900d
Build id: 3.10.0-514.el7.x86_64
|
0x17d
|
|
12
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
13
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
14
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
15
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
16
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
17
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
18
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [khugepaged:44]
Modules linked in: tcp_lp br_netfilter overlay fuse tun ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp coretemp snd_ens1371 snd_rawmidi iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul snd_ac97_codec glue_helper ablk_helper ac97_bus snd_seq cryptd ppdev snd_seq_device snd_pcm snd_timer vmw_balloon snd sg soundcore nfit libnvdimm pcspkr parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif crct10dif_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ttm scsi_transport_spi mptscsih ata_piix e1000 mptbase drm libata i2c_core
fjes dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 44 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8801360b6dd0 ti: ffff880131e48000 task.ti: ffff880131e48000
RIP: 0010:[<ffffffff811a6bc8>] [<ffffffff811a6bc8>] compaction_alloc+0x128/0x290
RSP: 0018:ffff880131e4b958 EFLAGS: 00010202
RAX: ffff88013ffd60c0 RBX: ffff8800a4126400 RCX: 0000000000000000
RDX: ffff88013ffd6000 RSI: 0000000000000001 RDI: 0000000000000054
RBP: ffff880131e4b998 R08: ffff88013fbd5cc0 R09: ffff88013ffd7800
R10: 0000000000001c00 R11: 0000000000100000 R12: 0000000000000000
R13: 0000000000000000 R14: ffffffff811a6a64 R15: ffff880131e4b948
FS: 0000000000000000(0000) GS:ffff880139600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001ae2820 CR3: 00000000019ba000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000001c00 ffff88013ffd7800 0000000000100000 ffffea000006c400
ffff880131e4bab0 ffffea000006c3c0 ffff880131e4baa0 0000000000000000
ffff880131e4ba38 ffffffff811e3969 ffff8801360b6dd0 ffff8801360b6dd0
Call Trace:
[<ffffffff811e3969>] migrate_pages+0x219/0x790
[<ffffffff811a6aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a7bb9>] compact_zone+0x2d9/0x430
[<ffffffff811a7dac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8161>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8168139f>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff81681873>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e8983>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff8168900d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: ff ff 07 00 77 d1 4c 89 e2 48 c1 ea 16 48 8b 14 d5 00 85 f4 81 48 85 d2 74 bd 83 e0 7f 48 c1 e0 05 48 01 d0 74 b1 f6 00 02 74 ac <49> 8b 45 00 a9 00 00 00 80 75 85 48 89 c2 48 c1 e8 29 48 c1 ea
Complete report #18866
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
compaction_alloc
|
vmlinux |
0xffffffff811a6bc8
Build id: 3.10.0-514.el7.x86_64
|
0x128
|
|
2
|
migrate_pages
|
vmlinux |
0xffffffff811e3969
Build id: 3.10.0-514.el7.x86_64
|
0x219
|
|
3
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a6aa0
Build id: 3.10.0-514.el7.x86_64
|
0x2c0
|
|
4
|
compact_zone
|
vmlinux |
0xffffffff811a7bb9
Build id: 3.10.0-514.el7.x86_64
|
0x2d9
|
|
5
|
compact_zone_order
|
vmlinux |
0xffffffff811a7dac
Build id: 3.10.0-514.el7.x86_64
|
0x9c
|
|
6
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8161
Build id: 3.10.0-514.el7.x86_64
|
0x121
|
|
7
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8168139f
Build id: 3.10.0-514.el7.x86_64
|
0xac
|
|
8
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff81681873
Build id: 3.10.0-514.el7.x86_64
|
0x3ea
|
|
9
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118af55
Build id: 3.10.0-514.el7.x86_64
|
0x405
|
|
10
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8983
Build id: 3.10.0-514.el7.x86_64
|
0x433
|
|
11
|
|
vmlinux |
0xffffffff8168900d
Build id: 3.10.0-514.el7.x86_64
|
0x17d
|
|
12
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
13
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
14
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
15
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
16
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
17
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
18
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [khugepaged:28]
Modules linked in: oss_usb(OE) osscore(OE) nfsv3 nfs fscache vmw_vsock_vmci_transport vsock vfat fat intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon nfit pcspkr libnvdimm sg shpchp i2c_piix4 vmw_vmci parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops vmxnet3 ahci libahci ata_piix ttm vmw_pvscsi drm i2c_core libata fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 28 Comm: khugepaged Tainted: G OEL ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff880077931f60 ti: ffff880077974000 task.ti: ffff880077974000
RIP: 0010:[<ffffffff811a6bc8>] [<ffffffff811a6bc8>] compaction_alloc+0x128/0x290
RSP: 0018:ffff880077977958 EFLAGS: 00010202
RAX: ffff88007ffd8080 RBX: 00000000000f3c00 RCX: 0000000000000000
RDX: ffff88007ffd8000 RSI: 0000000000000003 RDI: 0000000000000005
RBP: ffff880077977998 R08: 0000000000000000 R09: ffff88007ffd9800
R10: 0000000000001e00 R11: 0000000000080000 R12: ffff8800779778e8
R13: 0000000000000000 R14: ffffffff811a68ba R15: ffff880077977948
FS: 0000000000000000(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000291fb47 CR3: 000000007c347000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000001e00 0000000000080000 ffff88007ffd9800 ffffea000006bfc0
ffff880077977ab0 ffffea000006bf80 ffff880077977aa0 0000000000000000
ffff880077977a38 ffffffff811e3969 ffff880077931f60 ffff880077931f60
Call Trace:
[<ffffffff811e3969>] migrate_pages+0x219/0x790
[<ffffffff811a6aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a7bb9>] compact_zone+0x2d9/0x430
[<ffffffff811a7dac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8161>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8168139f>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff81681873>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e8983>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff8168900d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: ff ff 07 00 77 d1 4c 89 e2 48 c1 ea 16 48 8b 14 d5 00 85 f4 81 48 85 d2 74 bd 83 e0 7f 48 c1 e0 05 48 01 d0 74 b1 f6 00 02 74 ac <49> 8b 45 00 a9 00 00 00 80 75 85 48 89 c2 48 c1 e8 29 48 c1 ea
Complete report #13803
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a6882
Build id: 3.10.0-514.el7.x86_64
|
0xa2
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a6cb5
Build id: 3.10.0-514.el7.x86_64
|
0x215
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e3969
Build id: 3.10.0-514.el7.x86_64
|
0x219
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a6aa0
Build id: 3.10.0-514.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a7bb9
Build id: 3.10.0-514.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a7dac
Build id: 3.10.0-514.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8161
Build id: 3.10.0-514.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8168139f
Build id: 3.10.0-514.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff81681873
Build id: 3.10.0-514.el7.x86_64
|
0x3ea
|
|
10
|
|
vmlinux |
0xffffffff8118ef5e
Build id: 3.10.0-514.el7.x86_64
|
0x24e
|
|
11
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118af55
Build id: 3.10.0-514.el7.x86_64
|
0x405
|
|
12
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8983
Build id: 3.10.0-514.el7.x86_64
|
0x433
|
|
13
|
|
vmlinux |
0xffffffff8168900d
Build id: 3.10.0-514.el7.x86_64
|
0x17d
|
|
14
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
15
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
16
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
17
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
18
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
19
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
20
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [khugepaged:63]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock intel_powerclamp coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg nfit ppdev vmw_balloon pcspkr libnvdimm parport_pc parport vmw_vmci shpchp i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul sysimgblt crct10dif_common fb_sys_fops crc32c_intel serio_raw
ttm mptspi ata_piix drm scsi_transport_spi mptscsih libata mptbase vmxnet3 i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 63 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff88043f669f60 ti: ffff88042ad18000 task.ti: ffff88042ad18000
RIP: 0010:[<ffffffff811a6882>] [<ffffffff811a6882>] isolate_freepages_block+0xa2/0x2c0
RSP: 0018:ffff88042ad1b8b8 EFLAGS: 00010202
RAX: 00000000ffffffff RBX: 0000000000000001 RCX: ffff88042ad1baa0
RDX: 0000000000208e00 RSI: ffff88042ad1baf8 RDI: ffff88042ad1baa0
RBP: ffff88042ad1b948 R08: 0000000000000000 R09: ffff88043ffcd000
R10: 0000000000100c00 R11: 0000000000440000 R12: 0000000000000003
R13: ffff88043ffcb860 R14: ffff88043fbc9760 R15: ffff88043ffcd000
FS: 0000000000000000(0000) GS:ffff88043fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f0aa81f5e00 CR3: 00000000019ba000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000001 ffff88042ad1b998 ffffffff818e1c27
ffff88042ad1baa0 ffff88042ad1bae0 ffff88042ad1baf8 ffff88042ad1baa0
ffff88043ffcb800 0000000000000000 0000000000208e00 0000000000000003
Call Trace:
[<ffffffff811a6cb5>] compaction_alloc+0x215/0x290
[<ffffffff811e3969>] migrate_pages+0x219/0x790
[<ffffffff811a6aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a7bb9>] compact_zone+0x2d9/0x430
[<ffffffff811a7dac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8161>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8168139f>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff81681873>] __alloc_pages_slowpath+0x3ea/0x725
[<ffffffff8118ef5e>] ? release_pages+0x24e/0x430
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff811e8983>] khugepaged_scan_mm_slot+0x433/0xc70
[<ffffffff8168900d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 48 89 75 a0 48 89 4d 90 45 89 c5 83 f8 80 74 2e 45 84 ed 0f 85 10 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 fe 00 00 00 <8b> 43 18 41 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b
Complete report #9087
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff8118df8c
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
2
|
|
vmlinux |
0xffffffff81013588
Build id: 3.10.0-327.el7.x86_64
|
0xf8
|
|
3
|
compaction_alloc
|
vmlinux |
0xffffffff8118e42f
Build id: 3.10.0-327.el7.x86_64
|
0x1cf
|
|
4
|
migrate_pages
|
vmlinux |
0xffffffff811c71f9
Build id: 3.10.0-327.el7.x86_64
|
0x1e9
|
|
5
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff8118e260
Build id: 3.10.0-327.el7.x86_64
|
0x380
|
|
6
|
compact_zone
|
vmlinux |
0xffffffff8118f259
Build id: 3.10.0-327.el7.x86_64
|
0x299
|
|
7
|
compact_zone_order
|
vmlinux |
0xffffffff8118f45c
Build id: 3.10.0-327.el7.x86_64
|
0x9c
|
|
8
|
try_to_compact_pages
|
vmlinux |
0xffffffff8118f811
Build id: 3.10.0-327.el7.x86_64
|
0x121
|
|
9
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff816305c8
Build id: 3.10.0-327.el7.x86_64
|
0xac
|
|
10
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff811734e8
Build id: 3.10.0-327.el7.x86_64
|
0x788
|
|
11
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c9941
Build id: 3.10.0-327.el7.x86_64
|
0x3f1
|
|
12
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
13
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
14
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
15
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
16
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
17
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
18
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#0 stuck for 23s! [khugepaged:288]
Modules linked in: nls_utf8 isofs bnep bluetooth rfkill fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event kvm_intel kvm crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus pcspkr snd_seq snd_seq_device snd_pcm sg snd_timer snd soundcore
vmw_vmci i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic ata_piix crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper ttm drm i2c_core mptspi scsi_transport_spi mptscsih mptbase e1000 libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 288 Comm: khugepaged Not tainted 3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff88007afcdc00 ti: ffff880075f54000 task.ti: ffff880075f54000
RIP: 0010:[<ffffffff8118df8c>] [<ffffffff8118df8c>] isolate_freepages_block+0xac/0x380
RSP: 0018:ffff880075f57908 EFLAGS: 00010286
RAX: 00000000ffffffff RBX: 0000000000000000 RCX: ffff880075f57b58
RDX: 0000000000062400 RSI: 0000000000062200 RDI: ffff880075f57b00
RBP: ffff880075f579a0 R08: 0000000000000000 R09: ffff88007ffd9800
R10: 0000000001888000 R11: 0000000000080000 R12: 0000000000000000
R13: ffff880075f57860 R14: ffff880075f57860 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff88007c600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fee183da000 CR3: 000000000194a000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff81013588 0000000175f57960 ffff88007afce298 0000000000000000
0000000000062200 ffff880075f57b00 0000000000080000 0000000000000000
ffff880075f57b40 ffff880075f57b58 ffff880075f57b00 ffff88007ffd9800
Call Trace:
[<ffffffff81013588>] ? __switch_to+0xf8/0x4b0
[<ffffffff8118e42f>] compaction_alloc+0x1cf/0x240
[<ffffffff811c71f9>] migrate_pages+0x1e9/0x790
[<ffffffff8118e260>] ? isolate_freepages_block+0x380/0x380
[<ffffffff8118f259>] compact_zone+0x299/0x400
[<ffffffff8118f45c>] compact_zone_order+0x9c/0xf0
[<ffffffff8118f811>] try_to_compact_pages+0x121/0x1a0
[<ffffffff816305c8>] __alloc_pages_direct_compact+0xac/0x196
[<ffffffff811734e8>] __alloc_pages_nodemask+0x788/0xb90
[<ffffffff811c9941>] khugepaged_scan_mm_slot+0x3f1/0xca0
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: c0 00 4d 89 ee 48 89 4d b0 41 89 c5 eb 1d 90 49 83 c7 01 48 83 c3 40 4d 39 fc 0f 86 07 01 00 00 41 83 c5 01 4d 85 f6 4c 0f 44 f3 <8b> 43 18 83 f8 80 75 dc 48 8b 45 b8 0f b6 55 c0 48 8d 75 c8 4c