Complete report #1416980
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffff8bdab557
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffff8bbd36bf
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8bbd3fa4
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x184
|
|
4
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffff8bbca37c
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x34c
|
|
5
|
|
vmlinux |
0xffffffff8bbcaf41
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x1d1
|
|
6
|
|
vmlinux |
0xffffffff8bbd8c1c
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x10c
|
|
7
|
alloc_pages_vma
|
vmlinux |
0xffffffff8bc29419
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xa9
|
|
8
|
handle_mm_fault
|
vmlinux |
0xffffffff8bc0233f
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xb9f
|
|
9
|
__do_page_fault
|
vmlinux |
0xffffffff8c1bf7d3
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x213
|
|
10
|
trace_do_page_fault
|
vmlinux |
0xffffffff8c1bfbb6
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x56
|
|
11
|
do_async_page_fault
|
vmlinux |
0xffffffff8c1bf112
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x22
|
|
12
|
async_page_fault
|
vmlinux |
0xffffffff8c1bb7e8
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#30 stuck for 27s! [python3:27430]
Modules linked in: tcp_diag inet_diag xt_nat xt_mark ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ext4 mbcache jbd2 kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops virtio_net ttm net_failover virtio_console failover ahci drm ata_piix libahci libata crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 30 PID: 27430 Comm: python3 Kdump: loaded Tainted: G L ------------ 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
task: ffff8e217400c200 ti: ffff8e1b57234000 task.ti: ffff8e1b57234000
RIP: 0010:[<ffffffff8bdab557>] [<ffffffff8bdab557>] clear_page_c+0x7/0x10
RSP: 0000:ffff8e1b57237b08 EFLAGS: 00000246
RAX: 0000000000000000 RBX: fffffc590934d740 RCX: 0000000000000200
RDX: ffffffff8c4902a8 RSI: 0000000000000000 RDI: ffff8e1eb24a0000
RBP: ffff8e1b57237c18 R08: ffffffff8c490297 R09: fffffc5911c92840
R10: 0000000000004b29 R11: ffff8e1c34ff7440 R12: ffffffff8bd687d9
R13: ffff8e1b57237a80 R14: 0000000000000000 R15: ffff8e2254188000
FS: 00007fdf04eda740(0000) GS:ffff8e225f580000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000307ab068 CR3: 0000000117f46000 CR4: 0000000000340fe0
Call Trace:
[<ffffffff8bbd36bf>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff8bbd3fa4>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff8bbca37c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffff8bbcaf41>] ? generic_file_aio_read+0x1d1/0x2a0
[<ffffffff8bbd8c1c>] ? pagevec_lru_move_fn+0x10c/0x130
[<ffffffff8bc29419>] alloc_pages_vma+0xa9/0x210
[<ffffffff8bc0233f>] handle_mm_fault+0xb9f/0x1190
[<ffffffff8c1bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff8c1bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff8c1bf112>] do_async_page_fault+0x22/0x100
[<ffffffff8c1bb7e8>] async_page_fault+0x28/0x30
Code: c2 ff ff 48 89 c2 31 c0 4c 29 ea 39 da 89 d1 7f bc 85 d2 7f 95 89 d0 eb b4 e8 56 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 92 2f
Complete report #1395978
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffffbcbab4a7
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffbc9d35ff
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffbc9d3ee4
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x184
|
|
4
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffffbc9ca2bc
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x34c
|
|
5
|
|
vmlinux |
0xffffffffbc9cae81
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x1d1
|
|
6
|
alloc_pages_vma
|
vmlinux |
0xffffffffbca29359
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0xa9
|
|
7
|
handle_mm_fault
|
vmlinux |
0xffffffffbca0227f
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0xb9f
|
|
8
|
__do_page_fault
|
vmlinux |
0xffffffffbcfbf7d3
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x213
|
|
9
|
trace_do_page_fault
|
vmlinux |
0xffffffffbcfbfbb6
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x56
|
|
10
|
do_async_page_fault
|
vmlinux |
0xffffffffbcfbf112
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x22
|
|
11
|
async_page_fault
|
vmlinux |
0xffffffffbcfbb7e8
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 29s! [setroubleshootd:3977]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq ppdev snd_seq_device snd_pcm snd_timer snd pcspkr virtio_rng soundcore sg virtio_balloon joydev parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
crct10dif_common ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_net net_failover failover drm ata_piix libata crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 3977 Comm: setroubleshootd Kdump: loaded Not tainted 3.10.0-1160.95.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8b93b972c200 ti: ffff8b9343edc000 task.ti: ffff8b9343edc000
RIP: 0010:[<ffffffffbcbab4a7>] [<ffffffffbcbab4a7>] clear_page_c+0x7/0x10
RSP: 0000:ffff8b9343edfb08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffff373810bd940 RCX: 0000000000000200
RDX: ffffffffbd28feb0 RSI: 0000000000000000 RDI: ffff8b9382f64000
RBP: ffff8b9343edfc18 R08: ffffffffbd28fe9f R09: fffff373810bd940
R10: 0000000000003688 R11: ffff8b93746531b0 R12: ffff8b93b972c200
R13: fffff373810bd900 R14: fffff373810bd900 R15: ffff8b93bffd7800
FS: 00007f7f8db44740(0000) GS:ffff8b93bfd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000c3c90001 CR3: 0000000003e9e000 CR4: 00000000000006e0
Call Trace:
[<ffffffffbc9d35ff>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffffbc9d3ee4>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffffbc9ca2bc>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffbc9cae81>] ? generic_file_aio_read+0x1d1/0x2a0
[<ffffffffbca29359>] alloc_pages_vma+0xa9/0x210
[<ffffffffbca0227f>] handle_mm_fault+0xb9f/0x1190
[<ffffffffbcfbf7d3>] __do_page_fault+0x213/0x510
[<ffffffffbcfbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffbcfbf112>] do_async_page_fault+0x22/0x100
[<ffffffffbcfbb7e8>] async_page_fault+0x28/0x30
Code: c2 ff ff 48 89 c2 31 c0 4c 29 ea 39 da 89 d1 7f bc 85 d2 7f 95 89 d0 eb b4 e8 06 41 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab e9 91 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 82 2f
Complete report #1386238
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffff8a3ab4a7
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffff8a1d35ff
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8a1d3ee4
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x184
|
|
4
|
|
xfs |
0xffffffffc0380ed4
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x124
|
|
5
|
|
xfs |
0xffffffffc037200e
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x8e
|
|
6
|
|
vmlinux |
0xffffffff8a1d8b5c
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x10c
|
|
7
|
alloc_pages_vma
|
vmlinux |
0xffffffff8a229359
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0xa9
|
|
8
|
handle_mm_fault
|
vmlinux |
0xffffffff8a20227f
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0xb9f
|
|
9
|
__do_page_fault
|
vmlinux |
0xffffffff8a7bf7d3
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x213
|
|
10
|
trace_do_page_fault
|
vmlinux |
0xffffffff8a7bfbb6
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x56
|
|
11
|
do_async_page_fault
|
vmlinux |
0xffffffff8a7bf112
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x22
|
|
12
|
async_page_fault
|
vmlinux |
0xffffffff8a7bb7e8
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [setroubleshootd:19184]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq virtio_rng ppdev snd_seq_device pcspkr snd_pcm joydev snd_timer virtio_balloon snd soundcore i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi
virtio_console virtio_blk qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm 8139too crc32c_intel ata_piix serio_raw virtio_pci libata virtio_ring virtio 8139cp mii floppy drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 19184 Comm: setroubleshootd Kdump: loaded Tainted: G L ------------ 3.10.0-1160.95.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9aa8395db180 ti: ffff9aa7c66a8000 task.ti: ffff9aa7c66a8000
RIP: 0010:[<ffffffff8a3ab4a7>] [<ffffffff8a3ab4a7>] clear_page_c+0x7/0x10
RSP: 0000:ffff9aa7c66abb08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffff016c060fc00 RCX: 0000000000000200
RDX: ffffffff8aa8feb0 RSI: 0000000000000000 RDI: ffff9aa7d83ef000
RBP: ffff9aa7c66abc18 R08: ffffffff8aa8fe9f R09: fffff016c060fc00
R10: 0000000000003688 R11: ffff9aa83af7d6c0 R12: ffff9aa8395db180
R13: fffff016c060fbc0 R14: fffff016c060fbc0 R15: ffff9aa83ffce800
FS: 00007fe0ebde1740(0000) GS:ffff9aa83fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000b3c90001 CR3: 000000001866a000 CR4: 00000000000006e0
Call Trace:
[<ffffffff8a1d35ff>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff8a1d3ee4>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffffc0380ed4>] ? xfs_iunlock+0x124/0x130 [xfs]
[<ffffffffc037200e>] ? __xfs_filemap_fault+0x8e/0x1d0 [xfs]
[<ffffffff8a1d8b5c>] ? pagevec_lru_move_fn+0x10c/0x130
[<ffffffff8a229359>] alloc_pages_vma+0xa9/0x210
[<ffffffff8a20227f>] handle_mm_fault+0xb9f/0x1190
[<ffffffff8a7bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff8a7bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff8a7bf112>] do_async_page_fault+0x22/0x100
[<ffffffff8a7bb7e8>] async_page_fault+0x28/0x30
Code: c2 ff ff 48 89 c2 31 c0 4c 29 ea 39 da 89 d1 7f bc 85 d2 7f 95 89 d0 eb b4 e8 06 41 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab e9 91 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 82 2f
Complete report #1147276
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffffadbab557
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffad9d36bf
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffad9d3fa4
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x184
|
|
4
|
|
vmlinux |
0xffffffffada7a840
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xb0
|
|
5
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffffad9ca37c
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x34c
|
|
6
|
alloc_pages_vma
|
vmlinux |
0xffffffffada29419
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xa9
|
|
7
|
handle_mm_fault
|
vmlinux |
0xffffffffada0233f
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xb9f
|
|
8
|
__do_page_fault
|
vmlinux |
0xffffffffadfbf7d3
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x213
|
|
9
|
trace_do_page_fault
|
vmlinux |
0xffffffffadfbfbb6
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x56
|
|
10
|
do_async_page_fault
|
vmlinux |
0xffffffffadfbf112
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x22
|
|
11
|
async_page_fault
|
vmlinux |
0xffffffffadfbb7e8
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 27s! [dialer.pl:17903]
Modules linked in: bluetooth rfkill ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev virtio_balloon i2c_piix4 ip_tables ext4 mbcache jbd2 virtio_net net_failover failover virtio_console virtio_blk ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring floppy virtio drm_panel_orientation_quirks
CPU: 1 PID: 17903 Comm: dialer.pl Not tainted 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
task: ffff92df72f59080 ti: ffff92df63358000 task.ti: ffff92df63358000
RIP: 0010:[<ffffffffadbab557>] [<ffffffffadbab557>] clear_page_c+0x7/0x10
RSP: 0000:ffff92df6335bb08 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffe47182500340 RCX: 0000000000000200
RDX: ffffffffae2902a8 RSI: 0000000000000000 RDI: ffff92df54000000
RBP: ffff92df6335bc18 R08: ffffffffae290297 R09: ffffe47182500040
R10: 0000000000004232 R11: ffff92df6475c1b0 R12: ffff92df6335bae0
R13: 0000000000000296 R14: ffffffffffffffff R15: 0000000000000000
FS: 00007f08d8df9740(0000) GS:ffff92dfffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000003327018 CR3: 00000000b2cba000 CR4: 00000000003407e0
Call Trace:
[<ffffffffad9d36bf>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffffad9d3fa4>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffffada7a840>] ? touch_atime+0xb0/0x220
[<ffffffffad9ca37c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffada29419>] alloc_pages_vma+0xa9/0x210
[<ffffffffada0233f>] handle_mm_fault+0xb9f/0x1190
[<ffffffffadfbf7d3>] __do_page_fault+0x213/0x510
[<ffffffffadfbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffadfbf112>] do_async_page_fault+0x22/0x100
[<ffffffffadfbb7e8>] async_page_fault+0x28/0x30
Code: c2 ff ff 48 89 c2 31 c0 4c 29 ea 39 da 89 d1 7f bc 85 d2 7f 95 89 d0 eb b4 e8 56 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 92 2f
Complete report #882071
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffffa9595337
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffa93c860f
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffa93c8ee4
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x184
|
|
4
|
|
vmlinux |
0xffffffffa9429376
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x1d6
|
|
5
|
alloc_pages_vma
|
vmlinux |
0xffffffffa941cc19
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0xa9
|
|
6
|
handle_mm_fault
|
vmlinux |
0xffffffffa93f6817
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0xcb7
|
|
7
|
__do_page_fault
|
vmlinux |
0xffffffffa9990653
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x213
|
|
8
|
trace_do_page_fault
|
vmlinux |
0xffffffffa9990a26
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x56
|
|
9
|
do_async_page_fault
|
vmlinux |
0xffffffffa998ffa2
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x22
|
|
10
|
async_page_fault
|
vmlinux |
0xffffffffa998c7a8
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [klnagent:3454]
Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip6_tables ip_tables bridge stp llc ip_set sunrpc snd_hda_codec_generic snd_hda_intel ppdev snd_hda_codec crc32_pclmul snd_hda_core ghash_clmulni_intel aesni_intel snd_hwdep snd_seq lrw gf128mul snd_seq_device glue_helper snd_pcm ablk_helper pcspkr cryptd snd_timer sg snd joydev soundcore virtio_balloon parport_pc i2c_piix4 parport xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_console qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common 8139too crc32c_intel serio_raw virtio_pci ata_piix virtio_ring virtio libata 8139cp mii drm_panel_orientation_quirks floppy dm_mirror
dm_region_hash dm_log dm_mod fuse [last unloaded: ip_tables]
CPU: 0 PID: 3454 Comm: klnagent Kdump: loaded Tainted: G L ------------ 3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff92a3c587a100 ti: ffff92a397ac4000 task.ti: ffff92a397ac4000
RIP: 0010:[<ffffffffa9595337>] [<ffffffffa9595337>] clear_page_c+0x7/0x10
RSP: 0000:ffff92a397ac7b08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffc52881086800 RCX: 0000000000000200
RDX: ffffffffa9c8ed40 RSI: 0000000000000000 RDI: ffff92a3c219f000
RBP: ffff92a397ac7c18 R08: ffffffffa9c8ed2f R09: ffffc52881086800
R10: 000000000000415a R11: ffff92a39f0fe1b0 R12: ffff92a3c587a100
R13: ffffc528810867c0 R14: ffffc528810867c0 R15: ffff92a4bffd9800
FS: 00007f73c250b700(0000) GS:ffff92a4bfc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000046000000 CR3: 000000001f74c000 CR4: 00000000000006f0
Call Trace:
[<ffffffffa93c860f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffa93c8ee4>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffffa9429376>] ? kmem_cache_alloc_trace+0x1d6/0x200
[<ffffffffa941cc19>] alloc_pages_vma+0xa9/0x200
[<ffffffffa93f6817>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffa9990653>] __do_page_fault+0x213/0x500
[<ffffffffa9990a26>] trace_do_page_fault+0x56/0x150
[<ffffffffa998ffa2>] do_async_page_fault+0x22/0xf0
[<ffffffffa998c7a8>] async_page_fault+0x28/0x30
Code: 4c 29 ea 39 da 89 d1 7f c4 85 d2 7f 9d 89 d0 eb bc 0f 1f 00 e8 8b 5c d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f
Complete report #877563
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffff91bab4a7
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffff919d35ff
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff919d3ee4
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x184
|
|
4
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffff919ca2bc
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x34c
|
|
5
|
|
vmlinux |
0xffffffff919d8b5c
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x10c
|
|
6
|
alloc_pages_vma
|
vmlinux |
0xffffffff91a29359
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0xa9
|
|
7
|
handle_mm_fault
|
vmlinux |
0xffffffff91a0227f
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0xb9f
|
|
8
|
__do_page_fault
|
vmlinux |
0xffffffff91fbf7d3
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x213
|
|
9
|
trace_do_page_fault
|
vmlinux |
0xffffffff91fbfbb6
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x56
|
|
10
|
do_async_page_fault
|
vmlinux |
0xffffffff91fbf112
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x22
|
|
11
|
async_page_fault
|
vmlinux |
0xffffffff91fbb7e8
Build id: 3.10.0-1160.95.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [check_mysql_hea:31268]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm ppdev snd_timer snd pcspkr parport_pc soundcore virtio_rng parport i2c_piix4 joydev virtio_balloon nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi
qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix ttm virtio_console virtio_blk virtio_net net_failover failover drm libata crc32c_intel serio_raw virtio_pci virtio_ring drm_panel_orientation_quirks virtio floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 31268 Comm: check_mysql_hea Kdump: loaded Not tainted 3.10.0-1160.95.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff942779bd9080 ti: ffff9426c5838000 task.ti: ffff9426c5838000
RIP: 0010:[<ffffffff91bab4a7>] [<ffffffff91bab4a7>] clear_page_c+0x7/0x10
RSP: 0000:ffff9426c583bb08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffffac2c112c540 RCX: 0000000000000200
RDX: ffffffff9228feb0 RSI: 0000000000000000 RDI: ffff942704b14000
RBP: ffff9426c583bc18 R08: ffffffff9228fe9f R09: fffffac2c112c540
R10: 00000000000036b2 R11: ffff9426c1729948 R12: ffff942779bd9080
R13: fffffac2c112c500 R14: fffffac2c112c500 R15: ffff94277ffd0800
FS: 00007f9529f19740(0000) GS:ffff94277fb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000097c90001 CR3: 0000000005d16000 CR4: 00000000000006e0
Call Trace:
[<ffffffff919d35ff>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff919d3ee4>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff919ca2bc>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffff919d8b5c>] ? pagevec_lru_move_fn+0x10c/0x130
[<ffffffff91a29359>] alloc_pages_vma+0xa9/0x210
[<ffffffff91a0227f>] handle_mm_fault+0xb9f/0x1190
[<ffffffff91fbf7d3>] __do_page_fault+0x213/0x510
[<ffffffff91fbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff91fbf112>] do_async_page_fault+0x22/0x100
[<ffffffff91fbb7e8>] async_page_fault+0x28/0x30
Code: c2 ff ff 48 89 c2 31 c0 4c 29 ea 39 da 89 d1 7f bc 85 d2 7f 95 89 d0 eb b4 e8 06 41 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab e9 91 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 82 2f
Complete report #861377
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffff8d7ab517
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffff8d5d366f
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8d5d3f54
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x184
|
|
4
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffff8d5ca32c
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x34c
|
|
5
|
alloc_pages_vma
|
vmlinux |
0xffffffff8d6293c9
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0xa9
|
|
6
|
handle_mm_fault
|
vmlinux |
0xffffffff8d6022ef
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0xb9f
|
|
7
|
__do_page_fault
|
vmlinux |
0xffffffff8dbbf7d3
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x213
|
|
8
|
trace_do_page_fault
|
vmlinux |
0xffffffff8dbbfbb6
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x56
|
|
9
|
do_async_page_fault
|
vmlinux |
0xffffffff8dbbf112
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x22
|
|
10
|
async_page_fault
|
vmlinux |
0xffffffff8dbbb7e8
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [BgIOThr~Pool #1:3987]
Modules linked in: binfmt_misc rte_kni(OE) uio_pci_generic uio tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev parport_pc parport kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev floppy sg i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi virtio_net net_failover failover qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 3987 Comm: BgIOThr~Pool #1 Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.99.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
task: ffff95306ba45280 ti: ffff95315014c000 task.ti: ffff95315014c000
RIP: 0010:[<ffffffff8d7ab517>] [<ffffffff8d7ab517>] clear_page_c+0x7/0x10
RSP: 0000:ffff95315014fb08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff95306cbb2b20 RCX: 0000000000000200
RDX: ffffffff8de8ffa8 RSI: 0000000000000000 RDI: ffff95305965a000
RBP: ffff95315014fc18 R08: ffffffff8de8ff97 R09: fffff751826596c0
R10: 000000000000339a R11: ffff9531748b9d80 R12: 00000000000000d0
R13: ffff9531748b9d80 R14: ffff9531636309f0 R15: 0000000000000000
FS: 00007f592837c700(0000) GS:ffff95317fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5927d78000 CR3: 00000000bbf2a000 CR4: 0000000000760ff0
PKRU: 55555554
Call Trace:
[<ffffffff8d5d366f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff8d5d3f54>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff8d5ca32c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffff8d6293c9>] alloc_pages_vma+0xa9/0x210
[<ffffffff8d6022ef>] handle_mm_fault+0xb9f/0x1190
[<ffffffff8dbbf7d3>] __do_page_fault+0x213/0x510
[<ffffffff8dbbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff8dbbf112>] do_async_page_fault+0x22/0x100
[<ffffffff8dbbb7e8>] async_page_fault+0x28/0x30
Code: c2 ff ff 48 89 c2 31 c0 4c 29 ea 39 da 89 d1 7f bc 85 d2 7f 95 89 d0 eb b4 e8 96 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 92 2f
Complete report #748806
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffffb51ab3c7
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffb4fd35ff
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x36f
|
|
3
|
|
vmlinux |
0xffffffffb54682c4
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x34
|
|
4
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffb4fd3ee4
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x184
|
|
5
|
|
vmlinux |
0xffffffffb507a780
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0xb0
|
|
6
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffffb4fca2bc
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x34c
|
|
7
|
alloc_pages_vma
|
vmlinux |
0xffffffffb5029359
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0xa9
|
|
8
|
handle_mm_fault
|
vmlinux |
0xffffffffb500227f
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0xb9f
|
|
9
|
__do_page_fault
|
vmlinux |
0xffffffffb55bf7d3
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x213
|
|
10
|
trace_do_page_fault
|
vmlinux |
0xffffffffb55bfbb6
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x56
|
|
11
|
do_async_page_fault
|
vmlinux |
0xffffffffb55bf112
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x22
|
|
12
|
async_page_fault
|
vmlinux |
0xffffffffb55bb7e8
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 26s! [perl:7455]
Modules linked in: nfsv3 nfs_acl nfs lockd grace fscache ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel iTCO_wdt iTCO_vendor_support aesni_intel lrw gf128mul glue_helper ablk_helper cryptd lpc_ich sg pcspkr joydev i2c_i801 virtio_rng virtio_balloon ip_tables ext4 mbcache jbd2 sd_mod sr_mod cdrom crc_t10dif
crct10dif_generic virtio_gpu drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_scsi drm ahci virtio_console virtio_net net_failover libahci failover drm_panel_orientation_quirks libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio fuse
CPU: 2 PID: 7455 Comm: perl Kdump: loaded Tainted: G L ------------ 3.10.0-1160.90.1.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.16.1-1.el9 04/01/2014
task: ffff9260768a8000 ti: ffff926081e40000 task.ti: ffff926081e40000
RIP: 0010:[<ffffffffb51ab3c7>] [<ffffffffb51ab3c7>] clear_page_c+0x7/0x10
RSP: 0000:ffff926081e43b08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffffb51bd2fd RCX: 0000000000000200
RDX: ffffffffb588fe10 RSI: 0000000000000000 RDI: ffff92614128f000
RBP: ffff926081e43c18 R08: ffffffffb588fdff R09: fffffddc4404a400
R10: 0000000000003c6b R11: ffff9260b5ef6af8 R12: ffff92627ffd9800
R13: fffffddc40d89100 R14: ffff9262745db300 R15: ffff9262745db86c
FS: 00007f2eee488740(0000) GS:ffff92627fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000019b33a8 CR3: 00000000361e8000 CR4: 0000000000020fe0
Call Trace:
[<ffffffffb4fd35ff>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffffb54682c4>] ? consume_skb+0x34/0x90
[<ffffffffb4fd3ee4>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffffb507a780>] ? touch_atime+0xb0/0x220
[<ffffffffb4fca2bc>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffb5029359>] alloc_pages_vma+0xa9/0x210
[<ffffffffb500227f>] handle_mm_fault+0xb9f/0x1190
[<ffffffffb55bf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb55bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffb55bf112>] do_async_page_fault+0x22/0x100
[<ffffffffb55bb7e8>] async_page_fault+0x28/0x30
Code: c2 ff ff 48 89 c2 31 c0 4c 29 ea 39 da 89 d1 7f bc 85 d2 7f 95 89 d0 eb b4 e8 e6 41 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab e9 b1 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 a2 2f
Complete report #601593
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffffa9395187
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffa91c866f
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x36f
|
|
3
|
|
vmlinux |
0xffffffffa9794a26
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x56
|
|
4
|
|
vmlinux |
0xffffffffa9793fa2
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x22
|
|
5
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffa91c8f44
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x184
|
|
6
|
|
vmlinux |
0xffffffffa926c350
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xb0
|
|
7
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffffa91bf66c
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x34c
|
|
8
|
alloc_pages_vma
|
vmlinux |
0xffffffffa921cc49
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xa9
|
|
9
|
handle_mm_fault
|
vmlinux |
0xffffffffa91f6867
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xcb7
|
|
10
|
__do_page_fault
|
vmlinux |
0xffffffffa9794653
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x213
|
|
11
|
trace_do_page_fault
|
vmlinux |
0xffffffffa9794a26
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x56
|
|
12
|
do_async_page_fault
|
vmlinux |
0xffffffffa9793fa2
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x22
|
|
13
|
async_page_fault
|
vmlinux |
0xffffffffa97907a8
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 21s! [yum-cron:31904]
Modules linked in: tcp_diag udp_diag inet_diag iptable_mangle ipt_REJECT nf_reject_ipv4 xt_helper nf_conntrack_ipv4 nf_defrag_ipv4 xt_multiport xt_conntrack nf_conntrack libcrc32c iptable_filter ppdev iosf_mbi joydev parport_pc sg parport i2c_piix4 virtio_balloon crc32_pclmul pcspkr ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ip_tables ext4 mbcache jbd2 sr_mod cdrom virtio_net net_failover virtio_console virtio_blk ata_generic failover pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci serio_raw virtio_ring floppy drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 31904 Comm: yum-cron Kdump: loaded Tainted: G L ------------ 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8addc0ff6300 ti: ffff8add9785c000 task.ti: ffff8add9785c000
RIP: 0010:[<ffffffffa9395187>] [<ffffffffa9395187>] clear_page_c+0x7/0x10
RSP: 0000:ffff8add9785fb08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffd8bb49d1d7c0 RCX: 0000000000000200
RDX: ffffffffa9a8f4c8 RSI: 0000000000000000 RDI: ffff8adef475e000
RBP: ffff8add9785fc18 R08: ffffffffa9a8f4b7 R09: ffffd8bb49d1d7c0
R10: 0000000000003a9d R11: ffff8adeb0c39cb0 R12: ffff8addc0ff6300
R13: ffffd8bb49d1d780 R14: ffffd8bb49d1d780 R15: ffff8adf3ffd4000
FS: 00007f26e55bd740(0000) GS:ffff8adf354c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000c8a3000b CR3: 000000029f482000 CR4: 00000000000406e0
Call Trace:
[<ffffffffa91c866f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffa9794a26>] ? trace_do_page_fault+0x56/0x150
[<ffffffffa9793fa2>] ? do_async_page_fault+0x22/0xf0
[<ffffffffa91c8f44>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffffa926c350>] ? touch_atime+0xb0/0x220
[<ffffffffa91bf66c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffa921cc49>] alloc_pages_vma+0xa9/0x200
[<ffffffffa91f6867>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffa9794653>] __do_page_fault+0x213/0x500
[<ffffffffa9794a26>] trace_do_page_fault+0x56/0x150
[<ffffffffa9793fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffa97907a8>] async_page_fault+0x28/0x30
Code: 4c 29 ea 39 da 89 d1 7f c4 85 d2 7f 9d 89 d0 eb bc 0f 1f 00 e8 eb 5f d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f
Complete report #514222
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffffb9b95397
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffb99c862f
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffb99c8f04
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x184
|
|
4
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffffb99bf60c
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x34c
|
|
5
|
alloc_pages_vma
|
vmlinux |
0xffffffffb9a1cc49
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0xa9
|
|
6
|
handle_mm_fault
|
vmlinux |
0xffffffffb99f6837
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0xcb7
|
|
7
|
__do_page_fault
|
vmlinux |
0xffffffffb9f94653
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x213
|
|
8
|
trace_do_page_fault
|
vmlinux |
0xffffffffb9f94a26
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x56
|
|
9
|
do_async_page_fault
|
vmlinux |
0xffffffffb9f93fa2
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x22
|
|
10
|
async_page_fault
|
vmlinux |
0xffffffffb9f907a8
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [python3:20300]
Modules linked in: devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console net_failover
virtio_blk failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw drm_panel_orientation_quirks virtio floppy fuse
CPU: 2 PID: 20300 Comm: python3 Kdump: loaded Tainted: G L ------------ 3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
task: ffff8d16829ca100 ti: ffff8d1681adc000 task.ti: ffff8d1681adc000
RIP: 0010:[<ffffffffb9b95397>] [<ffffffffb9b95397>] clear_page_c+0x7/0x10
RSP: 0000:ffff8d1681adfb08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffffb9ba682d RCX: 0000000000000200
RDX: ffffffffba28f2a0 RSI: 0000000000000000 RDI: ffff8d167f415000
RBP: ffff8d1681adfc18 R08: ffffffffba28f28f R09: fffff20687fd0580
R10: ffff8d1cbfb836e0 R11: ffff8d1c84b3d6c8 R12: 00000000000000d0
R13: ffff8d18a2cf3290 R14: ffff8d1680c48078 R15: 0000000000000000
FS: 00007f41f1bef740(0000) GS:ffff8d1c9f280000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f41ea0c5024 CR3: 0000000200d2c000 CR4: 0000000000060fe0
Call Trace:
[<ffffffffb99c862f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffb99c8f04>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffffb99bf60c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffb9a1cc49>] alloc_pages_vma+0xa9/0x200
[<ffffffffb99f6837>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffb9f94653>] __do_page_fault+0x213/0x500
[<ffffffffb9f94a26>] trace_do_page_fault+0x56/0x150
[<ffffffffb9f93fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb9f907a8>] async_page_fault+0x28/0x30
Code: 4c 29 ea 39 da 89 d1 7f c4 85 d2 7f 9d 89 d0 eb bc 0f 1f 00 e8 4b 5c d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f
Complete report #23284
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c
|
vmlinux |
0xffffffffa5d94767
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffa5bc822f
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x36f
|
|
3
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffa5bc8ae6
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x166
|
|
4
|
alloc_pages_vma
|
vmlinux |
0xffffffffa5c1c2a9
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0xa9
|
|
5
|
handle_mm_fault
|
vmlinux |
0xffffffffa5bf5f07
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0xcb7
|
|
6
|
__do_page_fault
|
vmlinux |
0xffffffffa618f653
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x213
|
|
7
|
trace_do_page_fault
|
vmlinux |
0xffffffffa618fa26
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x56
|
|
8
|
do_async_page_fault
|
vmlinux |
0xffffffffa618efa2
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x22
|
|
9
|
async_page_fault
|
vmlinux |
0xffffffffa618b7a8
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [nagios:1377]
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [yum-cron:9517]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc edac_mce_amd crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd parport_pc sg joydev virtio_balloon parport i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops virtio_blk ttm virtio_console drm ata_piix
e1000 crct10dif_pclmul crct10dif_common libata crc32c_intel virtio_pci virtio_ring serio_raw virtio floppy drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 9517 Comm: yum-cron Tainted: G L ------------ 3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
task: ffff9c6c8fc6d280 ti: ffff9c6bd1c34000 task.ti: ffff9c6bd1c34000
RIP: 0010:[<ffffffffa5d94767>] [<ffffffffa5d94767>] clear_page_c+0x7/0x10
RSP: 0000:ffff9c6bd1c37b90 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffec4785830bc0 RCX: 0000000000000200
RDX: ffffffffa648e578 RSI: 0000000000000000 RDI: ffff9c6be0c33000
RBP: ffff9c6bd1c37ca0 R08: ffffffffa648e567 R09: ffffec4785830d00
R10: 0000000000002ddc R11: ffff9c6b8fcf8e58 R12: ffff9c6c8fc6d280
R13: ffffec4785830b80 R14: ffffec4785830b80 R15: ffff9c6c99fd8000
FS: 00007fe760b51740(0000) GS:ffff9c6c99d80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe71655d024 CR3: 0000000101e80000 CR4: 00000000000407e0
Call Trace:
[<ffffffffa5bc822f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffa5bc8ae6>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffa5c1c2a9>] alloc_pages_vma+0xa9/0x200
[<ffffffffa5bf5f07>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffa618f653>] __do_page_fault+0x213/0x500
[<ffffffffa618fa26>] trace_do_page_fault+0x56/0x150
[<ffffffffa618efa2>] do_async_page_fault+0x22/0xf0
[<ffffffffa618b7a8>] async_page_fault+0x28/0x30
Code: 4c 29 ea 39 da 89 d1 7f c4 85 d2 7f 9d 89 d0 eb bc 0f 1f 00 e8 2b 68 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f
Complete report #1079171
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c_e
|
vmlinux |
0xffffffffb51ab527
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffffb4fd366f
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x36f
|
|
3
|
|
vmlinux |
0xffffffffb51a3bc2
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x22
|
|
4
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffffb4fd3f54
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x184
|
|
5
|
|
vmlinux |
0xffffffffb507a7f0
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0xb0
|
|
6
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffffb4fca32c
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x34c
|
|
7
|
alloc_pages_vma
|
vmlinux |
0xffffffffb50293c9
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0xa9
|
|
8
|
handle_mm_fault
|
vmlinux |
0xffffffffb50022ef
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0xb9f
|
|
9
|
__do_page_fault
|
vmlinux |
0xffffffffb55bf7d3
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x213
|
|
10
|
trace_do_page_fault
|
vmlinux |
0xffffffffb55bfbb6
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x56
|
|
11
|
do_async_page_fault
|
vmlinux |
0xffffffffb55bf112
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x22
|
|
12
|
async_page_fault
|
vmlinux |
0xffffffffb55bb7e8
Build id: 3.10.0-1160.105.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 33s! [php-fpm:16371]
Modules linked in: binfmt_misc xfs iosf_mbi snd_hda_codec_generic crc32_pclmul ghash_clmulni_intel snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep aesni_intel libcrc32c snd_seq lrw ppdev gf128mul snd_seq_device glue_helper ablk_helper snd_pcm cryptd snd_timer virtio_rng joydev virtio_balloon pcspkr snd soundcore i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_net virtio_console virtio_blk net_failover failover drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 16371 Comm: php-fpm Kdump: loaded Not tainted 3.10.0-1160.105.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff882c1798b180 ti: ffff882d02c74000 task.ti: ffff882d02c74000
RIP: 0010:[<ffffffffb51ab527>] [<ffffffffb51ab527>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff882d02c77b08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffff8b6c2129280 RCX: 0000000000001000
RDX: ffffffffb5890218 RSI: 0000000000000000 RDI: ffff882c84a49000
RBP: ffff882d02c77c18 R08: ffffffffb5890207 R09: fffff8b6c2129280
R10: 0000000000004104 R11: ffff882c8ac9ea20 R12: ffff882c1798b180
R13: fffff8b6c2129240 R14: fffff8b6c2129240 R15: ffff882d3ffd6800
FS: 00007fd5e7074880(0000) GS:ffff882d3fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000013c90001 CR3: 00000001329e4000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb4fd366f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffffb51a3bc2>] ? radix_tree_lookup_slot+0x22/0x50
[<ffffffffb4fd3f54>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffffb507a7f0>] ? touch_atime+0xb0/0x220
[<ffffffffb4fca32c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffb50293c9>] alloc_pages_vma+0xa9/0x210
[<ffffffffb50022ef>] handle_mm_fault+0xb9f/0x1190
[<ffffffffb55bf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb55bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffb55bf112>] do_async_page_fault+0x22/0x100
[<ffffffffb55bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 96 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f
Complete report #837696
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page_c_e
|
vmlinux |
0xffffffff9b595187
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x7
|
|
2
|
|
vmlinux |
0xffffffff9b3c864f
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x36f
|
|
3
|
|
vmlinux |
0xffffffff9b993fed
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x6d
|
|
4
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff9b3c8f24
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x184
|
|
5
|
|
vmlinux |
0xffffffff9b46c340
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0xb0
|
|
6
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffff9b3bf64c
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x34c
|
|
7
|
alloc_pages_vma
|
vmlinux |
0xffffffff9b41cc49
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0xa9
|
|
8
|
handle_mm_fault
|
vmlinux |
0xffffffff9b3f6837
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0xcb7
|
|
9
|
__do_page_fault
|
vmlinux |
0xffffffff9b994653
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x213
|
|
10
|
trace_do_page_fault
|
vmlinux |
0xffffffff9b994a26
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x56
|
|
11
|
do_async_page_fault
|
vmlinux |
0xffffffff9b993fa2
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x22
|
|
12
|
async_page_fault
|
vmlinux |
0xffffffff9b9907a8
Build id: 3.10.0-1160.66.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 25s! [php:44646]
Modules linked in: snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core xfs snd_hwdep snd_seq iosf_mbi snd_seq_device crc32_pclmul ghash_clmulni_intel snd_pcm aesni_intel lrw gf128mul snd_timer snd ppdev glue_helper libcrc32c ablk_helper parport_pc cryptd parport soundcore virtio_balloon virtio_rng joydev pcspkr i2c_piix4 ip_tables ext4 mbcache jbd2 qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_generic pata_acpi virtio_net virtio_console net_failover drm virtio_blk failover ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 44646 Comm: php Kdump: loaded Tainted: G L ------------ 3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff990e9218c200 ti: ffff990e925ec000 task.ti: ffff990e925ec000
RIP: 0010:[<ffffffff9b595187>] [<ffffffff9b595187>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff990e925efb08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffefb0c14cc8c0 RCX: 0000000000001000
RDX: ffffffff9bc8f2e8 RSI: 0000000000000000 RDI: ffff990e93322000
RBP: ffff990e925efc18 R08: ffffffff9bc8f2d7 R09: ffffefb0c14cc8c0
R10: 0000000000004105 R11: ffff990ea2db6948 R12: ffff990e9218c200
R13: ffffefb0c14cc880 R14: ffffefb0c14cc880 R15: ffff990f7ffd6800
FS: 00007f2c870ce880(0000) GS:ffff990f7fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000025920002 CR3: 000000005209c000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9b3c864f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffff9b993fed>] ? do_async_page_fault+0x6d/0xf0
[<ffffffff9b3c8f24>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffff9b46c340>] ? touch_atime+0xb0/0x220
[<ffffffff9b3bf64c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffff9b41cc49>] alloc_pages_vma+0xa9/0x200
[<ffffffff9b3f6837>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff9b994653>] __do_page_fault+0x213/0x500
[<ffffffff9b994a26>] trace_do_page_fault+0x56/0x150
[<ffffffff9b993fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9b9907a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 fb 5f d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f
Complete report #779361
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
clear_page
|
vmlinux |
0xffffffff94baaec2
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x12
|
|
2
|
|
vmlinux |
0xffffffff949d349f
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x36f
|
|
3
|
|
vmlinux |
0xffffffff94fbf112
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x22
|
|
4
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff949d3d84
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x184
|
|
5
|
|
vmlinux |
0xffffffff94a7a250
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0xb0
|
|
6
|
?
do_generic_file_read.constprop.52
|
vmlinux |
0xffffffff949ca15c
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x34c
|
|
7
|
alloc_pages_vma
|
vmlinux |
0xffffffff94a29009
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0xa9
|
|
8
|
handle_mm_fault
|
vmlinux |
0xffffffff94a02257
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0xcb7
|
|
9
|
__do_page_fault
|
vmlinux |
0xffffffff94fbf7d3
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x213
|
|
10
|
trace_do_page_fault
|
vmlinux |
0xffffffff94fbfbb6
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x56
|
|
11
|
do_async_page_fault
|
vmlinux |
0xffffffff94fbf112
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x22
|
|
12
|
async_page_fault
|
vmlinux |
0xffffffff94fbb7e8
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#29 stuck for 24s! [postgres:19516]
Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod ppdev parport_pc parport sg i2c_piix4 joydev pcspkr ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net net_failover virtio_scsi virtio_console failover bochs_drm drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix virtio_pci virtio_ring libata virtio serio_raw drm_panel_orientation_quirks floppy fuse
CPU: 29 PID: 19516 Comm: postgres Kdump: loaded Not tainted 3.10.0-1160.81.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org 04/01/2014
task: ffff8fb631216300 ti: ffff8fb60c0b0000 task.ti: ffff8fb60c0b0000
RIP: 0010:[<ffffffff94baaec2>] [<ffffffff94baaec2>] clear_page+0x12/0x40
RSP: 0000:ffff8fb60c0b3b08 EFLAGS: 00010216
RAX: 0000000000000000 RBX: ffffd5fe198d9fc0 RCX: 000000000000003f
RDX: ffffffff9528fca8 RSI: 0000000000000000 RDI: ffff8fb423660000
RBP: ffff8fb60c0b3c18 R08: ffffffff9528fc97 R09: ffffd5fe198d9840
R10: ffff8fbcffb49c80 R11: ffff8fb616af2bd0 R12: ffff8fb60d8da3f0
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 00007fd38fd4a880(0000) GS:ffff8fbcc3340000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd38fc73000 CR3: 0000000834b00000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff949d349f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff94fbf112>] ? do_async_page_fault+0x22/0x100
[<ffffffff949d3d84>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff94a7a250>] ? touch_atime+0xb0/0x220
[<ffffffff949ca15c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffff94a29009>] alloc_pages_vma+0xa9/0x210
[<ffffffff94a02257>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff94fbf7d3>] __do_page_fault+0x213/0x510
[<ffffffff94fbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff94fbf112>] do_async_page_fault+0x22/0x100
[<ffffffff94fbb7e8>] async_page_fault+0x28/0x30
Code: f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 f3 aa e9 92 2f 00 00 66 90 31 c0 b9 40 00 00 00 66 0f 1f 84 00 00 00 00 00 ff c9 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89