Complete report #2115
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_page_rep
|
vmlinux |
0xffffffff81325265
Build id: 3.10.0-514.el7.x86_64
|
0x5
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8f71
Build id: 3.10.0-514.el7.x86_64
|
0xa21
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [khugepaged:43]
Modules linked in: fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic iosf_mbi crc32_pclmul snd_hda_intel ghash_clmulni_intel aesni_intel snd_hda_codec lrw gf128mul glue_helper ablk_helper snd_hda_core snd_hwdep snd_seq ppdev snd_seq_device cryptd snd_pcm snd_timer snd pcspkr parport_pc virtio_balloon parport sg i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect
crc32c_intel sysimgblt ata_piix fb_sys_fops serio_raw ttm drm virtio_pci virtio_ring virtio libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 43 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff880239643ec0 ti: ffff88022d178000 task.ti: ffff88022d178000
RIP: 0010:[<ffffffff81325265>] [<ffffffff81325265>] copy_page_rep+0x5/0x10
RSP: 0018:ffff88022d17bd90 EFLAGS: 00010206
RAX: 000000012276d000 RBX: 00007f3200000000 RCX: 0000000000000200
RDX: ffff880000000000 RSI: ffff8801fffca000 RDI: ffff88012276d000
RBP: ffff88022d17be40 R08: 0000000000000024 R09: a8000a03ec000000
R10: 57ffdffc1680fb00 R11: 0000000000000000 R12: ffff88022d17bd40
R13: ffff880187c3b400 R14: 0000000000000001 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff880239c80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000d8cdd001 CR3: 00000000b8f44000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811e8f71 00007f320b400000 00000001cefe9067 00007f320b400000
ffff880034e263c0 ffffea000890ea70 ffff88022d17be60 ffff8802243a92d0
ffff88022d17bfd8 ffff88022d17bfd8 ffff88022d35d7f8 ffff8801ccf0cbd0
Call Trace:
[<ffffffff811e8f71>] ? khugepaged_scan_mm_slot+0xa21/0xc70
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: b7 90 90 90 90 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 66 66 66 90 66 66 66 90
Complete report #845049
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c97e5
Build id: 3.10.0-327.el7.x86_64
|
0x295
|
|
2
|
|
vmlinux |
0xffffffff8163856d
Build id: 3.10.0-327.el7.x86_64
|
0x17d
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
6
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#0 stuck for 22s! [khugepaged:29]
Modules linked in: tcp_lp bnep bluetooth rfkill fuse vboxsf(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev vboxguest(OE) snd_timer pcspkr snd sg soundcore i2c_piix4 parport_pc parport video nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper ttm ahci ata_piix libahci crc32c_intel serio_raw e1000 drm i2c_core libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 29 Comm: khugepaged Tainted: G OEL ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff880108d84500 ti: ffff880035c40000 task.ti: ffff880035c40000
RIP: 0010:[<ffffffff811c97e5>] [<ffffffff811c97e5>] khugepaged_scan_mm_slot+0x295/0xca0
RSP: 0018:ffff880035c43d90 EFLAGS: 00010286
RAX: ffffea0001deba00 RBX: 0000000000000286 RCX: 0000000001deba00
RDX: 0000000077ae8067 RSI: 000033172ab9f000 RDI: 0000000077ae8067
RBP: ffff880035c43e40 R08: ffff880061f810d8 R09: 000000000000b7c0
R10: 0000000000000009 R11: ffff880025ac4000 R12: 0000000000000000
R13: ffff8800d05f3200 R14: ffff88010ee14780 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88010ee00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000261891f70000 CR3: 00000000cb695000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880035c43e40 ffffffff8163856d ffff88010ee14780 0000000000000286
0000000000000000 ffff8800610870a0 ffff880035c43e60 ffff880000000000
000033172aa00000 ffff8800b8642c38 ffffea000096b0f0 ffff8800b8642bc0
Call Trace:
[<ffffffff8163856d>] ? schedule_timeout+0x17d/0x2d0
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: ff 0f 00 00 00 c0 ff ff 49 21 d5 41 f7 c5 01 01 00 00 74 cd 41 f6 c5 02 74 c7 48 8b 7d c8 48 89 de e8 f0 a2 fc ff 48 85 c0 74 b6 <48> 8b 10 48 c1 ea 36 83 04 95 80 59 ee 81 01 48 8b 10 83 e2 20
Complete report #142931
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_page
|
vmlinux |
0xffffffff812e18c9
Build id: 3.10.0-229.el7.x86_64
|
0x49
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811b4a0c
Build id: 3.10.0-229.el7.x86_64
|
0x9ec
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811b4f1f
Build id: 3.10.0-229.el7.x86_64
|
0x25f
|
|
4
|
|
vmlinux |
0xffffffff81098350
Build id: 3.10.0-229.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811b4cc0
Build id: 3.10.0-229.el7.x86_64
|
0xca0
|
|
6
|
kthread
|
vmlinux |
0xffffffff8109739f
Build id: 3.10.0-229.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810972d0
Build id: 3.10.0-229.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff8161497c
Build id: 3.10.0-229.el7.x86_64
|
0x7c
|
|
9
|
|
vmlinux |
0xffffffff810972d0
Build id: 3.10.0-229.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#2 stuck for 23s! [khugepaged:339]
Modules linked in: ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ppdev sg parport_pc parport pcspkr serio_raw virtio_balloon i2c_piix4 uinput ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net virtio_scsi cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper ttm drm ata_piix virtio_pci libata virtio_ring i2c_core virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 339 Comm: khugepaged Not tainted 3.10.0-229.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff880217622220 ti: ffff880217674000 task.ti: ffff880217674000
RIP: 0010:[<ffffffff812e18c9>] [<ffffffff812e18c9>] copy_page+0x49/0xe0
RSP: 0000:ffff880217677d78 EFLAGS: 00010206
RAX: 0000000000000000 RBX: ffff880216a43878 RCX: 000000000000003a
RDX: 0000000000000000 RSI: ffff8804df019000 RDI: ffff880827d0f000
RBP: ffff880217677e40 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffea00137c0640
R13: 00000000209f43c0 R14: ffff880216a405e8 R15: 00007fa6ccf0f000
FS: 0000000000000000(0000) GS:ffff880237040000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000024d98002 CR3: 000000000190a000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880216a43878 ffffea00137c0640 ffffffff811b4a0c ffff88021807f080
0000000216a43067 00007fa6cce00000 0000020700000286 ffff880217677fd8
ffff8800bb951000 ffff880217677e60 ffffea00209f0000 ffff880217677fd8
Call Trace:
[<ffffffff811b4a0c>] ? khugepaged_scan_mm_slot+0x9ec/0xca0
[<ffffffff811b4f1f>] khugepaged+0x25f/0x4a0
[<ffffffff81098350>] ? wake_up_bit+0x30/0x30
[<ffffffff811b4cc0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff8109739f>] kthread+0xcf/0xe0
[<ffffffff810972d0>] ? kthread_create_on_node+0x140/0x140
[<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
[<ffffffff810972d0>] ? kthread_create_on_node+0x140/0x140
Code: 00 00 48 ff c9 48 8b 06 48 8b 5e 08 48 8b 56 10 4c 8b 46 18 4c 8b 4e 20 4c 8b 56 28 4c 8b 5e 30 4c 8b 66 38 0f 18 8e 40 01 00 00 <48> 89 07 48 89 5f 08 48 89 57 10 4c 89 47 18 4c 89 4f 20 4c 89
Complete report #171266
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_page
|
vmlinux |
0xffffffff812ffea3
Build id: 3.10.0-327.el7.x86_64
|
0x23
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c9f4d
Build id: 3.10.0-327.el7.x86_64
|
0x9fd
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
6
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#0 stuck for 27s! [khugepaged:78]
Modules linked in: vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr sg parport_pc parport vmw_vmci shpchp i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel ttm serio_raw mptspi ata_piix scsi_transport_spi drm mptscsih e1000 libata mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 78 Comm: khugepaged Tainted: G W L ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880425998000 ti: ffff880425994000 task.ti: ffff880425994000
RIP: 0010:[<ffffffff812ffea3>] [<ffffffff812ffea3>] copy_page+0x23/0xe0
RSP: 0018:ffff880425997d78 EFLAGS: 00010206
RAX: 0000000222f7c000 RBX: ffff88042f052000 RCX: 000000000000003a
RDX: ffff880000000000 RSI: ffff8803e7036000 RDI: ffff880222f7c000
RBP: ffff880425997e40 R08: 0000000000000040 R09: a8003e3e35000000
R10: 57ffabc1da8f8d40 R11: 0000000000000000 R12: 0000000000000001
R13: ffffea000f8f8d40 R14: ffff88042b6be350 R15: ffff88043ffd8000
FS: 0000000000000000(0000) GS:ffff88043fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f8b251ac000 CR3: 000000000194a000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8801bb54fbe0 00000000088bdf00 ffffffff811c9f4d 00007f51a0000000
00007f51a0000000 00000001bb54f067 ffff880425695dc0 00007f5100000e01
ffff8804294f4ac8 ffff880425997e60 ffff8801bb54f000 ffffea0006ed51b0
Call Trace:
[<ffffffff811c9f4d>] ? khugepaged_scan_mm_slot+0x9fd/0xca0
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: c3 0f 1f 80 00 00 00 00 48 83 ec 10 48 89 1c 24 4c 89 64 24 08 b9 3b 00 00 00 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c9 <48> 8b 06 48 8b 5e 08 48 8b 56 10 4c 8b 46 18 4c 8b 4e 20 4c 8b
Complete report #159271
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page
|
vmlinux |
0xffffffff812ffd02
Build id: 3.10.0-327.el7.x86_64
|
0x12
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811c9eb5
Build id: 3.10.0-327.el7.x86_64
|
0x965
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811ca447
Build id: 3.10.0-327.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810a6ae0
Build id: 3.10.0-327.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811ca1f0
Build id: 3.10.0-327.el7.x86_64
|
0xca0
|
|
6
|
kthread
|
vmlinux |
0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
|
0x140
|
BUG: soft lockup - CPU#3 stuck for 22s! [khugepaged:300]
Modules linked in: bnep bluetooth rfkill fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables vmhgfs(OE) vmw_vsock_vmci_transport vsock ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event coretemp crc32_pclmul ppdev ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg vmw_vmci shpchp i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic
ata_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw pata_acpi vmwgfx e1000 drm_kms_helper ttm mptspi scsi_transport_spi drm mptscsih mptbase i2c_core ata_piix libata
CPU: 3 PID: 300 Comm: khugepaged Tainted: G OEL ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff880422859700 ti: ffff8804228b0000 task.ti: ffff8804228b0000
RIP: 0010:[<ffffffff812ffd02>] [<ffffffff812ffd02>] clear_page+0x12/0x40
RSP: 0018:ffff8804228b3d88 EFLAGS: 00010216
RAX: 0000000000000000 RBX: ffffffff811d05ed RCX: 000000000000003f
RDX: 0000000000017c08 RSI: 0000000000000008 RDI: ffff880170fdd000
RBP: ffff8804228b3e40 R08: 0000000000000030 R09: a8001341f1000000
R10: 57ffd6be13d07c40 R11: 0000000000000000 R12: ffff88042e013000
R13: 0000000000000001 R14: 0000000000000001 R15: ffffea0004d07c40
FS: 0000000000000000(0000) GS:ffff88042e660000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f73bb539000 CR3: 000000000194a000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811c9eb5 0000000017600000 0000000017600000 00000004126f7067
ffff8804225844c0 0000000000000e01 ffff8804278b2938 ffff8804228b3e60
ffff8804126f7000 ffffea00109a2730 ffff880422584538 ffff88042689c5d8
Call Trace:
[<ffffffff811c9eb5>] ? khugepaged_scan_mm_slot+0x965/0xca0
[<ffffffff811ca447>] khugepaged+0x257/0x480
[<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811ca1f0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f 1f 44 00 00 31 c0 b9 40 00 00 00 66 0f 1f 84 00 00 00 00 00 ff c9 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89
Complete report #115336
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_page
|
vmlinux |
0xffffffff81325293
Build id: 3.10.0-514.el7.x86_64
|
0x23
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8f71
Build id: 3.10.0-514.el7.x86_64
|
0xa21
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810bf8d6
Build id: 3.10.0-514.el7.x86_64
|
0x56
|
|
8
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
9
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
10
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [khugepaged:65]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun binfmt_misc ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr ppdev vmw_balloon shpchp parport_pc parport vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables
xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ttm crct10dif_common crc32c_intel mptspi drm ata_piix scsi_transport_spi serio_raw mptscsih libata vmxnet3 i2c_core mptbase floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 65 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88081ac53ec0 ti: ffff88081accc000 task.ti: ffff88081accc000
RIP: 0010:[<ffffffff81325293>] [<ffffffff81325293>] copy_page+0x23/0xe0
RSP: 0018:ffff88081accfd80 EFLAGS: 00010206
RAX: 00000001f7c64000 RBX: ffff88081accfd40 RCX: 000000000000003a
RDX: ffff880000000000 RSI: ffff8801680f3000 RDI: ffff8801f7c64000
RBP: ffff88081accfe40 R08: 0000000000000030 R09: a8000aa0d0000000
R10: 57ffdf5f32a83400 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000001 R14: ffffea0002a83400 R15: ffff880187320d00
FS: 0000000000000000(0000) GS:ffff88083fc40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f937ca67640 CR3: 00000000019ba000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00007f2df3c64000 0000000007df1900 ffffffff811e8f71 00007f2df3c00000
000000028f797067 00007f2df3c00000 ffff880035c1a1b8 ffffea00207260b0
ffff88081accfe60 ffff88081c982cf0 ffff88081accffd8 ffff88081accffd8
Call Trace:
[<ffffffff811e8f71>] ? khugepaged_scan_mm_slot+0xa21/0xc70
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810bf8d6>] ? finish_task_switch+0x56/0x180
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: c3 0f 1f 80 00 00 00 00 48 83 ec 10 48 89 1c 24 4c 89 64 24 08 b9 3b 00 00 00 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c9 <48> 8b 06 48 8b 5e 08 48 8b 56 10 4c 8b 46 18 4c 8b 4e 20 4c 8b
Complete report #98045
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_page
|
vmlinux |
0xffffffff81325439
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x49
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e90d1
Build id: 3.10.0-514.2.2.el7.x86_64
|
0xa21
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9577
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1720
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9320
Build id: 3.10.0-514.2.2.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b064f
Build id: 3.10.0-514.2.2.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810b0580
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81696718
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b0580
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [khugepaged:43]
Modules linked in: rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun binfmt_misc xt_nat xt_mark ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmhgfs(OE) bnep vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 intel_powerclamp coretemp snd_seq_midi snd_seq_midi_event iosf_mbi btusb crc32_pclmul btrtl ghash_clmulni_intel aesni_intel btbcm btintel bluetooth lrw gf128mul glue_helper ablk_helper cryptd
ppdev snd_ens1371 vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus snd_seq sg snd_seq_device pcspkr snd_pcm vmw_vmci rfkill nfit snd_timer snd i2c_piix4 soundcore shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm drm mptspi ata_piix scsi_transport_spi mptscsih libata e1000 mptbase i2c_core fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 43 Comm: khugepaged Tainted: G OE ------------ 3.10.0-514.2.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff880136824e70 ti: ffff8801323c0000 task.ti: ffff8801323c0000
RIP: 0010:[<ffffffff81325439>] [<ffffffff81325439>] copy_page+0x49/0xe0
RSP: 0018:ffff8801323c3d80 EFLAGS: 00010206
RAX: 0000000000000000 RBX: ffff8801323c3d40 RCX: 000000000000003a
RDX: ffffffffffffffff RSI: ffff88009c8a5000 RDI: ffff880079dd4000
RBP: ffff8801323c3e40 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: ffffffffffffffff R12: 0000000000000001
R13: 0000000000000001 R14: ffffea0002722900 R15: ffff8801386c8a40
FS: 0000000000000000(0000) GS:ffff880139640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000485075 CR3: 00000000019ba000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00007f753dfd4000 0000000001e77500 ffffffff811e90d1 00007f753de00000
00000000a3a2b067 00007f753de00000 ffff880137b5ef78 ffffea0004cf50b0
ffff8801323c3e60 ffff880133d42f78 ffff8801323c3fd8 ffff8801323c3fd8
Call Trace:
[<ffffffff811e90d1>] ? khugepaged_scan_mm_slot+0xa21/0xc70
[<ffffffff811e9577>] khugepaged+0x257/0x480
[<ffffffff810b1720>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e9320>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b064f>] kthread+0xcf/0xe0
[<ffffffff810b0580>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696718>] ret_from_fork+0x58/0x90
[<ffffffff810b0580>] ? kthread_create_on_node+0x140/0x140
Code: 00 00 48 ff c9 48 8b 06 48 8b 5e 08 48 8b 56 10 4c 8b 46 18 4c 8b 4e 20 4c 8b 56 28 4c 8b 5e 30 4c 8b 66 38 0f 18 8e 40 01 00 00 <48> 89 07 48 89 5f 08 48 89 57 10 4c 89 47 18 4c 89 4f 20 4c 89
Complete report #45903
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page
|
vmlinux |
0xffffffff81325922
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x12
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e95c5
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x985
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9b07
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1b20
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e98b0
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b0a4f
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81697518
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [khugepaged:73]
Modules linked in: cfg80211 rfkill xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc fuse ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_lsm_pinned_10807(E) intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg vmw_balloon pcspkr vmw_vmci shpchp i2c_piix4 nfit parport_pc libnvdimm parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea mptspi
sysfillrect ata_piix sysimgblt scsi_transport_spi fb_sys_fops crct10dif_pclmul ttm crct10dif_common drm crc32c_intel mptscsih libata serio_raw mptbase i2c_core vmxnet3 fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 73 Comm: khugepaged Tainted: P EL ------------ 3.10.0-514.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff88089c07de20 ti: ffff880897160000 task.ti: ffff880897160000
RIP: 0010:[<ffffffff81325922>] [<ffffffff81325922>] clear_page+0x12/0x40
RSP: 0018:ffff880897163d90 EFLAGS: 00010216
RAX: 0000000000000000 RBX: 57ff798e00000000 RCX: 000000000000003f
RDX: 000000000001a0e0 RSI: 0000000000000008 RDI: ffff8806aa183000
RBP: ffff880897163e40 R08: 0000000000000048 R09: a800707155000000
R10: 57ff798ec71c5540 R11: 0000000000000000 R12: ffff880897163d40
R13: ffff88017fc76400 R14: 0000000000000001 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff88089b640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f525322947e CR3: 0000000825364000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811e95c5 000000c423c00000 00000007307ec067 000000c423c00000
ffff88088dfc0000 ffffea002245d870 ffff880897163e60 ffff8808917618f0
ffff880897163fd8 ffff880897163fd8 ffff880896648078 ffff880896a33a28
Call Trace:
[<ffffffff811e95c5>] ? khugepaged_scan_mm_slot+0x985/0xc70
[<ffffffff811e9b07>] khugepaged+0x257/0x480
[<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e98b0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697518>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f 1f 44 00 00 31 c0 b9 40 00 00 00 66 0f 1f 84 00 00 00 00 00 ff c9 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89
Complete report #45628
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_page
|
vmlinux |
0xffffffff81325ac3
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x23
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9661
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xa21
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9b07
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1b20
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e98b0
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b0a4f
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81697518
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [khugepaged:73]
Modules linked in: fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cfg80211 rfkill vmw_vsock_vmci_transport vsock falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_lsm_pinned_10807(E) intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg nfit vmw_balloon parport_pc pcspkr parport libnvdimm shpchp i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common
crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi serio_raw ttm scsi_transport_spi ata_piix drm mptscsih vmxnet3 libata mptbase i2c_core fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 73 Comm: khugepaged Tainted: P EL ------------ 3.10.0-514.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff88089c6aedd0 ti: ffff880897190000 task.ti: ffff880897190000
RIP: 0010:[<ffffffff81325ac3>] [<ffffffff81325ac3>] copy_page+0x23/0xe0
RSP: 0018:ffff880897193d80 EFLAGS: 00010206
RAX: 000000008a90f000 RBX: ffff880897193d40 RCX: 000000000000003a
RDX: ffff880000000000 RSI: ffff8806f43fd000 RDI: ffff88008a90f000
RBP: ffff880897193e40 R08: 0000000000000048 R09: a8006f43fc000000
R10: 57ff7abc1fd0ff00 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000001 R14: ffffea001bd0ff00 R15: ffff880178c43fc0
FS: 0000000000000000(0000) GS:ffff88089b6c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe299676885 CR3: 00000000019be000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00000005c270f000 00000000022a43c0 ffffffff811e9661 00000005c2600000
00000006ed7f4067 00000005c2600000 ffff880896cfa2a8 ffffea001d719af0
ffff880897193e60 ffff88075c66b098 ffff880897193fd8 ffff880897193fd8
Call Trace:
[<ffffffff811e9661>] ? khugepaged_scan_mm_slot+0xa21/0xc70
[<ffffffff811e9b07>] khugepaged+0x257/0x480
[<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e98b0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697518>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: c3 0f 1f 80 00 00 00 00 48 83 ec 10 48 89 1c 24 4c 89 64 24 08 b9 3b 00 00 00 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c9 <48> 8b 06 48 8b 5e 08 48 8b 56 10 4c 8b 46 18 4c 8b 4e 20 4c 8b
Complete report #17561
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9001
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x1f1
|
|
2
|
|
vmlinux |
0xffffffff8168a17c
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x17c
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9cd7
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1b20
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9a80
Build id: 3.10.0-514.26.2.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b0a4f
Build id: 3.10.0-514.26.2.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81697758
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.26.2.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [xfsaild/dm-0:425]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [khugepaged:43]
Modules linked in: nfnetlink_queue nfnetlink_log bluetooth rfkill ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg pcspkr vmw_balloon parport_pc vmw_vmci parport shpchp i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi mptspi
scsi_transport_spi crct10dif_pclmul crct10dif_common crc32c_intel mptscsih vmwgfx serio_raw drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptbase ata_piix vmxnet3 i2c_core libata floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 43 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.26.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88017cededd0 ti: ffff880234010000 task.ti: ffff880234010000
RIP: 0010:[<ffffffff811e9001>] [<ffffffff811e9001>] khugepaged_scan_mm_slot+0x1f1/0xc70
RSP: 0018:ffff880234013d98 EFLAGS: 00010246
RAX: 00000000ea7eea7e RBX: 0000000000000286 RCX: 000000022d289000
RDX: 000000000000ea7e RSI: ffffea0000000000 RDI: ffffea0008b4a270
RBP: ffff880234013e40 R08: ffff880000000000 R09: 0000000000000001
R10: 0000000000000000 R11: ffff88022d28a000 R12: 0000000000000000
R13: ffff8800bb9dd780 R14: ffff88023fd16c40 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88023fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcf8444e990 CR3: 00000000b91c3000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff8168a17c 00000002337f6067 0000000000000286 ffff88017cc6e078
dead000000000200 ffff880234013e60 ffff880100001000 ffff880000000000
00007f36d8400000 ffff88023430de38 ffffea0008b4a270 0000000000000000
Call Trace:
[<ffffffff8168a17c>] ? schedule_timeout+0x17c/0x2c0
[<ffffffff811e9cd7>] khugepaged+0x257/0x480
[<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e9a80>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697758>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: 00 4d 8d 9d 00 10 00 00 4d 39 dd 73 4d 49 89 de 45 31 ff 48 89 5d 98 c7 45 b0 00 00 00 00 4c 89 e3 4c 89 5d d0 4d 89 f4 45 89 fe <49> 8b 55 00 48 f7 c2 9f ff ff ff 0f 85 86 00 00 00 48 f7 43 50
Complete report #13241
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page
|
vmlinux |
0xffffffff813250f2
Build id: 3.10.0-514.el7.x86_64
|
0x12
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e8ed5
Build id: 3.10.0-514.el7.x86_64
|
0x985
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9417
Build id: 3.10.0-514.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1600
Build id: 3.10.0-514.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e91c0
Build id: 3.10.0-514.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [khugepaged:64]
Modules linked in: tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun xt_AUDIT nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_set ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set_hash_ip ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul snd_ens1371
glue_helper snd_rawmidi snd_ac97_codec ac97_bus ablk_helper snd_seq snd_seq_device cryptd snd_pcm btusb btrtl btbcm nfit snd_timer snd ppdev soundcore btintel bluetooth sg pcspkr rfkill vmw_balloon libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm mptspi drm scsi_transport_spi ata_piix mptscsih e1000 libata mptbase i2c_core fjes
CPU: 7 PID: 64 Comm: khugepaged Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff880232724e70 ti: ffff8802321c8000 task.ti: ffff8802321c8000
RIP: 0010:[<ffffffff813250f2>] [<ffffffff813250f2>] clear_page+0x12/0x40
RSP: 0018:ffff8802321cbd90 EFLAGS: 00010216
RAX: 0000000000000000 RBX: f0f0f0f000000000 RCX: 000000000000003f
RDX: 000000000001a078 RSI: 0000000000000008 RDI: ffff8801a5df2000
RBP: ffff8802321cbe40 R08: 0000000000000030 R09: a8000a6b6f000000
R10: 57ffdf94939adbc0 R11: 0000000000000000 R12: ffff8802321cbd40
R13: ffff880187c76400 R14: 0000000000000001 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff8802357c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6e66bab188 CR3: 00000002272d2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811e8ed5 00007fe05ca00000 00000000b32fe067 00007fe05ca00000
ffff880184459aa0 ffffea0008b593b0 ffff8802321cbe60 ffff88022d64e728
ffff8802321cbfd8 ffff8802321cbfd8 ffff880216338078 ffff8800355a5e60
Call Trace:
[<ffffffff811e8ed5>] ? khugepaged_scan_mm_slot+0x985/0xc70
[<ffffffff811e9417>] khugepaged+0x257/0x480
[<ffffffff810b1600>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e91c0>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f 1f 44 00 00 31 c0 b9 40 00 00 00 66 0f 1f 84 00 00 00 00 00 ff c9 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89
Complete report #5928
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_page
|
vmlinux |
0xffffffff813259b3
Build id: 3.10.0-514.16.1.el7.x86_64
|
0x23
|
|
2
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e95e1
Build id: 3.10.0-514.16.1.el7.x86_64
|
0xa21
|
|
3
|
khugepaged
|
vmlinux |
0xffffffff811e9a87
Build id: 3.10.0-514.16.1.el7.x86_64
|
0x257
|
|
4
|
|
vmlinux |
0xffffffff810b1b20
Build id: 3.10.0-514.16.1.el7.x86_64
|
0x30
|
|
5
|
?
khugepaged_scan_mm_slot
|
vmlinux |
0xffffffff811e9830
Build id: 3.10.0-514.16.1.el7.x86_64
|
0xc70
|
|
6
|
kthread
|
vmlinux |
0xffffffff810b0a4f
Build id: 3.10.0-514.16.1.el7.x86_64
|
0xcf
|
|
7
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.16.1.el7.x86_64
|
0x140
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff816970d8
Build id: 3.10.0-514.16.1.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b0980
Build id: 3.10.0-514.16.1.el7.x86_64
|
0x140
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [khugepaged:42]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg vmw_balloon pcspkr parport_pc parport vmw_vmci shpchp i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom ata_generic vmwgfx crct10dif_pclmul crct10dif_common
drm_kms_helper syscopyarea sysfillrect ahci sysimgblt crc32c_intel fb_sys_fops pata_acpi vmxnet3 libahci serio_raw ttm mptspi scsi_transport_spi drm mptscsih mptbase i2c_core ata_piix libata fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 42 Comm: khugepaged Not tainted 3.10.0-514.16.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88042e86edd0 ti: ffff88042ac74000 task.ti: ffff88042ac74000
RIP: 0010:[<ffffffff813259b3>] [<ffffffff813259b3>] copy_page+0x23/0xe0
RSP: 0018:ffff88042ac77d80 EFLAGS: 00010206
RAX: 00000002b6f14000 RBX: ffff88042ac77d40 RCX: 000000000000003a
RDX: ffff880000000000 RSI: ffff8801046d9000 RDI: ffff8802b6f14000
RBP: ffff88042ac77e40 R08: 0000000000000030 R09: a800145d68000000
R10: 57ffd5a29d175a00 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000001 R14: ffffea0005175a00 R15: ffff88017e95d680
FS: 0000000000000000(0000) GS:ffff88042d600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f43f03ceb00 CR3: 0000000424bd3000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00007f9f72114000 000000000adbc500 ffffffff811e95e1 00007f9f72000000
00000003533d2067 00007f9f72000000 ffff880427fca960 ffffea00109faa30
ffff88042ac77e60 ffff880427ea8c80 ffff88042ac77fd8 ffff88042ac77fd8
Call Trace:
[<ffffffff811e95e1>] ? khugepaged_scan_mm_slot+0xa21/0xc70
[<ffffffff811e9a87>] khugepaged+0x257/0x480
[<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff811e9830>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff816970d8>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: c3 0f 1f 80 00 00 00 00 48 83 ec 10 48 89 1c 24 4c 89 64 24 08 b9 3b 00 00 00 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c9 <48> 8b 06 48 8b 5e 08 48 8b 56 10 4c 8b 46 18 4c 8b 4e 20 4c 8b