Info

Permalink
Function
clear_page_c_e
First occurrence
2020-10-26
Last occurrence
2022-11-05
State
NEW
Components
kernel
Quality
-19

Statistics

Operating system Count
CentOS 7 321
Architecture Count
x86_64 321
Related packages Count
kernel 321
 0:3.10.0-1160.15.2.el7 122
 0:3.10.0-1062.18.1.el7 74
 0:3.10.0-693.5.2.el7 40
 0:3.10.0-1160.102.1.el7 32
 0:3.10.0-1127.10.1.el7 9
 0:3.10.0-1160.el7 8
 0:3.10.0-1127.19.1.el7 7
 0:3.10.0-1160.76.1.el7 7
 0:3.10.0-514.21.1.el7 6
 0:3.10.0-1160.49.1.el7 5
 0:3.10.0-1160.36.2.el7 4
 0:3.10.0-1160.62.1.el7 2
 0:3.10.0-1160.45.1.el7 1
 0:3.10.0-1160.81.1.el7 1
 0:3.10.0-1160.88.1.el7 1
 0:3.10.0-1160.6.1.el7 1
 0:3.10.0-1127.13.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1365131
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffaa193917
Build id: 3.10.0-1127.13.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffa9fc863f
Build id: 3.10.0-1127.13.1.el7.x86_64
0x36f
3
? sock_recvmsg
vmlinux 0xffffffffaa4334f5
Build id: 3.10.0-1127.13.1.el7.x86_64
0xc5
4
? netlink_sendmsg
vmlinux 0xffffffffaa490ec8
Build id: 3.10.0-1127.13.1.el7.x86_64
0x148
5
__alloc_pages_nodemask
vmlinux 0xffffffffa9fc8ed6
Build id: 3.10.0-1127.13.1.el7.x86_64
0x166
6
alloc_pages_vma
vmlinux 0xffffffffaa01c309
Build id: 3.10.0-1127.13.1.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffffa9ff5ff7
Build id: 3.10.0-1127.13.1.el7.x86_64
0xcb7
8
__do_page_fault
vmlinux 0xffffffffaa58d653
Build id: 3.10.0-1127.13.1.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffffaa58da26
Build id: 3.10.0-1127.13.1.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffffaa58cfa2
Build id: 3.10.0-1127.13.1.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffffaa5897a8
Build id: 3.10.0-1127.13.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [sendmail:16467]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod ext4 mbcache jbd2 iosf_mbi snd_hda_codec_generic snd_hda_intel snd_hda_codec crc32_pclmul ghash_clmulni_intel virtio_balloon aesni_intel snd_hda_core snd_hwdep snd_seq pcspkr ppdev lrw i2c_piix4 gf128mul joydev snd_seq_device snd_pcm snd_timer
glue_helper snd ablk_helper cryptd parport_pc parport soundcore sg ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_blk virtio_net net_failover drm virtio_console floppy failover ata_piix crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw libata virtio drm_panel_orientation_quirks
CPU: 2 PID: 16467 Comm: sendmail Kdump: loaded Not tainted 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9370144820e0 ti: ffff937077d44000 task.ti: ffff937077d44000
RIP: 0010:[<ffffffffaa193917>]  [<ffffffffaa193917>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff937077d47b90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 00000000197500b4 RCX: 0000000000001000
RDX: ffffffffaa88d4f0 RSI: 0000000000000000 RDI: ffff936fc1231000
RBP: ffff937077d47ca0 R08: ffffffffaa88d4df R09: fffff02bc0048c80
R10: 0000000000004105 R11: ffff93700dc28ca8 R12: ffff937077d47b00
R13: ffffffffa9ed3820 R14: 0000000000000000 R15: ffff936f00000000
FS:  00007fb7a88ac880(0000) GS:ffff9370ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055fc03b3c0e8 CR3: 0000000137c58000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa9fc863f>] ? get_page_from_freelist+0x36f/0xaa0
[<ffffffffaa4334f5>] ? sock_recvmsg+0xc5/0x100
[<ffffffffaa490ec8>] ? netlink_sendmsg+0x148/0x420
[<ffffffffa9fc8ed6>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffaa01c309>] alloc_pages_vma+0xa9/0x200
[<ffffffffa9ff5ff7>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffaa58d653>] __do_page_fault+0x213/0x500
[<ffffffffaa58da26>] trace_do_page_fault+0x56/0x150
[<ffffffffaa58cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffffaa5897a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 7b 81 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #1012569
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff9ddab527
Build id: 3.10.0-1160.102.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff9dbd366f
Build id: 3.10.0-1160.102.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffff9dbd3f54
Build id: 3.10.0-1160.102.1.el7.x86_64
0x184
4
? __schedule
vmlinux 0xffffffff9e1b7920
Build id: 3.10.0-1160.102.1.el7.x86_64
0x320
5
alloc_pages_vma
vmlinux 0xffffffff9dc293c9
Build id: 3.10.0-1160.102.1.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffff9dc022ef
Build id: 3.10.0-1160.102.1.el7.x86_64
0xb9f
7
__do_page_fault
vmlinux 0xffffffff9e1bf7d3
Build id: 3.10.0-1160.102.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff9e1bfbb6
Build id: 3.10.0-1160.102.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff9e1bf112
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff9e1bb7e8
Build id: 3.10.0-1160.102.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#63 stuck for 25s! [spectre:89696]
Modules linked in: tcp_lp tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon i2c_piix4 auth_rpcgss sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_scsi virtio_net net_failover failover virtio_console ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul
crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 63 PID: 89696 Comm: spectre Kdump: loaded Tainted: G             L ------------   3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014
task: ffff9e2c47dfc200 ti: ffff9d6dd6020000 task.ti: ffff9d6dd6020000
RIP: 0010:[<ffffffff9ddab527>]  [<ffffffff9ddab527>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9d6dd6023b08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffff9ddbd43d RCX: 0000000000001000
RDX: ffffffff9e48ffa8 RSI: 0000000000000000 RDI: ffff9d70b4ab2000
RBP: ffff9d6dd6023c18 R08: ffffffff9e48ff97 R09: ffffdd895dd2acc0
R10: 000000000000dae8 R11: ffff9f0a1c9d4948 R12: 0000000000006400
R13: ffffffff9dbd0e7a R14: ffff9d6dd6023ad0 R15: 000000000000001e
FS:  00007f9678970700(0000) GS:ffff9f0a259c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000035d8dff0 CR3: 00000016afdac000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff9dbd366f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff9dbd3f54>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff9e1b7920>] ? __schedule+0x320/0x680
[<ffffffff9dc293c9>] alloc_pages_vma+0xa9/0x210
[<ffffffff9dc022ef>] handle_mm_fault+0xb9f/0x1190
[<ffffffff9e1bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff9e1bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff9e1bf112>] do_async_page_fault+0x22/0x100
[<ffffffff9e1bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 96 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #1011803
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff9ddab527
Build id: 3.10.0-1160.102.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff9dbd366f
Build id: 3.10.0-1160.102.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffff9dbd3f54
Build id: 3.10.0-1160.102.1.el7.x86_64
0x184
4
? __find_get_page
vmlinux 0xffffffff9dbc85ae
Build id: 3.10.0-1160.102.1.el7.x86_64
0x1e
5
? filemap_fault
vmlinux 0xffffffff9dbcbc5d
Build id: 3.10.0-1160.102.1.el7.x86_64
0x17d
6
alloc_pages_vma
vmlinux 0xffffffff9dc293c9
Build id: 3.10.0-1160.102.1.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffff9dc022ef
Build id: 3.10.0-1160.102.1.el7.x86_64
0xb9f
8
__do_page_fault
vmlinux 0xffffffff9e1bf7d3
Build id: 3.10.0-1160.102.1.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffff9e1bfbb6
Build id: 3.10.0-1160.102.1.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff9e1bf112
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffff9e1bb7e8
Build id: 3.10.0-1160.102.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [spectre:100384]
Modules linked in: tcp_lp tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon i2c_piix4 auth_rpcgss sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_scsi virtio_net net_failover failover virtio_console ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul
crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 100384 Comm: spectre Kdump: loaded Tainted: G             L ------------   3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014
task: ffff9e2cbf6f5280 ti: ffff9e2b8d138000 task.ti: ffff9e2b8d138000
RIP: 0010:[<ffffffff9ddab527>]  [<ffffffff9ddab527>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9e2b8d13bb08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffff9ddbd43d RCX: 0000000000001000
RDX: ffffffff9e48ffa8 RSI: 0000000000000000 RDI: ffff9ea7514d2000
RBP: ffff9e2b8d13bc18 R08: ffffffff9e48ff97 R09: ffffdd8e384534c0
R10: 000000000000dae8 R11: ffff9d73a3baa6c0 R12: 0000000000000000
R13: ffffffff9dbd0e7a R14: ffff9e2b8d13bad0 R15: 000000000000001e
FS:  00007f97596f13c0(0000) GS:ffff9f0a24a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000007ed48000 CR3: 000001169750a000 CR4: 0000000000760ff0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff9dbd366f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff9dbd3f54>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff9dbc85ae>] ? __find_get_page+0x1e/0xa0
[<ffffffff9dbcbc5d>] ? filemap_fault+0x17d/0x420
[<ffffffff9dc293c9>] alloc_pages_vma+0xa9/0x210
[<ffffffff9dc022ef>] handle_mm_fault+0xb9f/0x1190
[<ffffffff9e1bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff9e1bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff9e1bf112>] do_async_page_fault+0x22/0x100
[<ffffffff9e1bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 96 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #1011526
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff9ddab527
Build id: 3.10.0-1160.102.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff9dbd366f
Build id: 3.10.0-1160.102.1.el7.x86_64
0x36f
3
? security_inode_permission
vmlinux 0xffffffff9dd1aef2
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
4
__alloc_pages_nodemask
vmlinux 0xffffffff9dbd3f54
Build id: 3.10.0-1160.102.1.el7.x86_64
0x184
5
? mntput
vmlinux 0xffffffff9dc80cf4
Build id: 3.10.0-1160.102.1.el7.x86_64
0x24
6
alloc_pages_vma
vmlinux 0xffffffff9dc293c9
Build id: 3.10.0-1160.102.1.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffff9dc022ef
Build id: 3.10.0-1160.102.1.el7.x86_64
0xb9f
8
__do_page_fault
vmlinux 0xffffffff9e1bf7d3
Build id: 3.10.0-1160.102.1.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffff9e1bfbb6
Build id: 3.10.0-1160.102.1.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff9e1bf112
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffff9e1bb7e8
Build id: 3.10.0-1160.102.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [strmout:2925]
Modules linked in: tcp_lp tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon i2c_piix4 auth_rpcgss sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_scsi virtio_net net_failover failover virtio_console ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul
crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 7 PID: 2925 Comm: strmout Kdump: loaded Tainted: G             L ------------   3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014
task: ffff9d2ee608b180 ti: ffff9da412b74000 task.ti: ffff9da412b74000
RIP: 0010:[<ffffffff9ddab527>]  [<ffffffff9ddab527>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9da412b77b08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffff9ddbd43d RCX: 0000000000001000
RDX: ffffffff9e48ffa8 RSI: 0000000000000000 RDI: ffff9da2dde9f000
RBP: ffff9da412b77c18 R08: ffffffff9e48ff97 R09: ffffdd8a2677a800
R10: 000000000000dae8 R11: ffff9e3b8b58c510 R12: ffffffff9dda87ab
R13: ffff9da412b77b08 R14: ffff9d71abab221f R15: ffff9d712bab2220
FS:  00007fb3baa23980(0000) GS:ffff9f0a24bc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000048643018 CR3: 000001d09a2ea000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff9dbd366f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff9dd1aef2>] ? security_inode_permission+0x22/0x30
[<ffffffff9dbd3f54>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff9dc80cf4>] ? mntput+0x24/0x40
[<ffffffff9dc293c9>] alloc_pages_vma+0xa9/0x210
[<ffffffff9dc022ef>] handle_mm_fault+0xb9f/0x1190
[<ffffffff9e1bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff9e1bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff9e1bf112>] do_async_page_fault+0x22/0x100
[<ffffffff9e1bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 96 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #1010855
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff86d94737
Build id: 3.10.0-1160.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff86bc81ef
Build id: 3.10.0-1160.el7.x86_64
0x36f
3
? __slab_free
vmlinux 0xffffffff86c25e0d
Build id: 3.10.0-1160.el7.x86_64
0x9d
4
__alloc_pages_nodemask
vmlinux 0xffffffff86bc8aa6
Build id: 3.10.0-1160.el7.x86_64
0x166
5
alloc_pages_vma
vmlinux 0xffffffff86c1c269
Build id: 3.10.0-1160.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffff86bf5ec7
Build id: 3.10.0-1160.el7.x86_64
0xcb7
7
__do_page_fault
vmlinux 0xffffffff8718e653
Build id: 3.10.0-1160.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff8718ea26
Build id: 3.10.0-1160.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff8718dfa2
Build id: 3.10.0-1160.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff8718a7a8
Build id: 3.10.0-1160.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [zabbix_agent2:6336]
Modules linked in: xt_comment udp_diag xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) tun tcp_diag inet_diag snd_hda_codec_generic nfit snd_hda_intel libnvdimm snd_hda_codec snd_hda_core iosf_mbi snd_hwdep crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel ppdev snd_pcm parport_pc aesni_intel parport lrw snd_timer gf128mul glue_helper ablk_helper cryptd snd pcspkr joydev virtio_balloon soundcore i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_blk drm ata_piix 8139too libata crct10dif_pclmul crct10dif_common
8139cp serio_raw virtio_pci crc32c_intel virtio_ring mii virtio floppy drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 6336 Comm: zabbix_agent2 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9751ce862100 ti: ffff97501f764000 task.ti: ffff97501f764000
RIP: 0010:[<ffffffff86d94737>]  [<ffffffff86d94737>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff97501f767b90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffff9b48852f080 RCX: 0000000000001000
RDX: ffffffff8748e488 RSI: 0000000000000000 RDI: ffff975214bc1000
RBP: ffff97501f767ca0 R08: ffffffff8748e477 R09: fffff9b48852f080
R10: 0000000000003ac5 R11: ffff975198b75d88 R12: ffff9751ce862100
R13: fffff9b48852f040 R14: fffff9b48852f040 R15: ffff9752bffd8000
FS:  00007fea96ffd700(0000) GS:ffff9752bfdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000547f0007 CR3: 0000000066d74000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff86bc81ef>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffff86c25e0d>] ? __slab_free+0x9d/0x290
[<ffffffff86bc8aa6>] __alloc_pages_nodemask+0x166/0x450
[<ffffffff86c1c269>] alloc_pages_vma+0xa9/0x200
[<ffffffff86bf5ec7>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff8718e653>] __do_page_fault+0x213/0x500
[<ffffffff8718ea26>] trace_do_page_fault+0x56/0x150
[<ffffffff8718dfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8718a7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 6b 68 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #1010501
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff9ddab527
Build id: 3.10.0-1160.102.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff9dbd366f
Build id: 3.10.0-1160.102.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffff9dbd3f54
Build id: 3.10.0-1160.102.1.el7.x86_64
0x184
4
? flush_tlb_page
vmlinux 0xffffffff9da82144
Build id: 3.10.0-1160.102.1.el7.x86_64
0x54
5
? lru_cache_add
vmlinux 0xffffffff9dbd923e
Build id: 3.10.0-1160.102.1.el7.x86_64
0xe
6
alloc_pages_vma
vmlinux 0xffffffff9dc293c9
Build id: 3.10.0-1160.102.1.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffff9dc022ef
Build id: 3.10.0-1160.102.1.el7.x86_64
0xb9f
8
__do_page_fault
vmlinux 0xffffffff9e1bf7d3
Build id: 3.10.0-1160.102.1.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffff9e1bfbb6
Build id: 3.10.0-1160.102.1.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff9e1bf112
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffff9e1bb7e8
Build id: 3.10.0-1160.102.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#59 stuck for 21s! [spectre:81488]
Modules linked in: tcp_lp tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon i2c_piix4 auth_rpcgss sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_scsi virtio_net net_failover failover virtio_console ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul
crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 59 PID: 81488 Comm: spectre Kdump: loaded Tainted: G             L ------------   3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014
task: ffff9df49699c200 ti: ffff9dc0bb020000 task.ti: ffff9dc0bb020000
RIP: 0010:[<ffffffff9ddab527>]  [<ffffffff9ddab527>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9dc0bb023b08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffff9ddbd43d RCX: 0000000000001000
RDX: ffffffff9e48ffa8 RSI: 0000000000000000 RDI: ffff9e2e42993000
RBP: ffff9dc0bb023c18 R08: ffffffff9e48ff97 R09: ffffdd8c540a6500
R10: 000000000000dae8 R11: ffff9d4f771eb7a0 R12: 0000000000000000
R13: ffffffff9dbd0e7a R14: ffff9dc0bb023ad0 R15: 000000000000001e
FS:  00007f08439ef700(0000) GS:ffff9f0a258c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000001d14fa000 CR3: 000000949afbc000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff9dbd366f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff9dbd3f54>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff9da82144>] ? flush_tlb_page+0x54/0xa0
[<ffffffff9dbd923e>] ? lru_cache_add+0xe/0x20
[<ffffffff9dc293c9>] alloc_pages_vma+0xa9/0x210
[<ffffffff9dc022ef>] handle_mm_fault+0xb9f/0x1190
[<ffffffff9e1bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff9e1bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff9e1bf112>] do_async_page_fault+0x22/0x100
[<ffffffff9e1bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 96 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #921709
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffb69ab527
Build id: 3.10.0-1160.102.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffb67d366f
Build id: 3.10.0-1160.102.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffffb67d3f54
Build id: 3.10.0-1160.102.1.el7.x86_64
0x184
4
? unmap_page_range
vmlinux 0xffffffffb67fd484
Build id: 3.10.0-1160.102.1.el7.x86_64
0xa94
5
alloc_pages_vma
vmlinux 0xffffffffb68293c9
Build id: 3.10.0-1160.102.1.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffffb68022ef
Build id: 3.10.0-1160.102.1.el7.x86_64
0xb9f
7
__do_page_fault
vmlinux 0xffffffffb6dbf7d3
Build id: 3.10.0-1160.102.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffffb6dbfbb6
Build id: 3.10.0-1160.102.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffffb6dbf112
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffffb6dbb7e8
Build id: 3.10.0-1160.102.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [pegasus:77237]
Modules linked in: tcp_diag inet_diag tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr virtio_balloon i2c_piix4 auth_rpcgss sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_scsi virtio_net net_failover failover ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul libata
virtio_pci crct10dif_common crc32c_intel virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 14 PID: 77237 Comm: pegasus Kdump: loaded Tainted: G             L ------------   3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014
task: ffff9ade3a3fc200 ti: ffff9b3a6c7c4000 task.ti: ffff9b3a6c7c4000
RIP: 0010:[<ffffffffb69ab527>]  [<ffffffffb69ab527>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9b3a6c7c7b08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffffb69bd43d RCX: 0000000000001000
RDX: ffffffffb708ffa8 RSI: 0000000000000000 RDI: ffff9a87b53bd000
RBP: ffff9b3a6c7c7c18 R08: ffffffffb708ff97 R09: fffff23378d4ef80
R10: 000000000000dae8 R11: ffff9c30c3661950 R12: 0000000100020200
R13: ffffffffb67d0e7a R14: ffff9b3a6c7c7ad0 R15: 000000000000001e
FS:  00002af6aa0fb140(0000) GS:ffff9c3a64d80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000429e9060 CR3: 000001daea302000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb67d366f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffffb67d3f54>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffffb67fd484>] ? unmap_page_range+0xa94/0xc80
[<ffffffffb68293c9>] alloc_pages_vma+0xa9/0x210
[<ffffffffb68022ef>] handle_mm_fault+0xb9f/0x1190
[<ffffffffb6dbf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb6dbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffb6dbf112>] do_async_page_fault+0x22/0x100
[<ffffffffb6dbb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 96 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #895206
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff9eb94777
Build id: 3.10.0-1160.6.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff9e9c822f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffff9e9c8ae6
Build id: 3.10.0-1160.6.1.el7.x86_64
0x166
4
alloc_pages_vma
vmlinux 0xffffffff9ea1c2a9
Build id: 3.10.0-1160.6.1.el7.x86_64
0xa9
5
handle_mm_fault
vmlinux 0xffffffff9e9f5f07
Build id: 3.10.0-1160.6.1.el7.x86_64
0xcb7
6
__do_page_fault
vmlinux 0xffffffff9ef8f653
Build id: 3.10.0-1160.6.1.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffff9ef8fa26
Build id: 3.10.0-1160.6.1.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffff9ef8efa2
Build id: 3.10.0-1160.6.1.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffff9ef8b7a8
Build id: 3.10.0-1160.6.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 34s! [QThread:13084]
Modules linked in: tcp_diag inet_diag binfmt_misc tun nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc nfit libnvdimm iosf_mbi crc32_pclmul sg ghash_clmulni_intel ppdev aesni_intel virtio_balloon parport_pc lrw parport gf128mul joydev i2c_piix4 glue_helper pcspkr ablk_helper cryptd ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic
pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata 8139too crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci serio_raw 8139cp virtio_ring floppy virtio mii drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 13084 Comm: QThread Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8cb735ec5280 ti: ffff8cb82258c000 task.ti: ffff8cb82258c000
RIP: 0010:[<ffffffff9eb94777>]  [<ffffffff9eb94777>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8cb82258fb90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffdaca847062c0 RCX: 0000000000001000
RDX: ffffffff9f28e578 RSI: 0000000000000000 RDI: ffff8cb81c18a000
RBP: ffff8cb82258fca0 R08: ffffffff9f28e567 R09: ffffdaca847062c0
R10: 00000000000043b5 R11: ffff8cbaed56ca20 R12: ffff8cb735ec5280
R13: ffffdaca84706280 R14: ffffdaca84706280 R15: ffff8cbb3ffda000
FS:  00007f1954ff9700(0000) GS:ffff8cbb3fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000002c920002 CR3: 0000000115b78000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9e9c822f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffff9e9c8ae6>] __alloc_pages_nodemask+0x166/0x450
[<ffffffff9ea1c2a9>] alloc_pages_vma+0xa9/0x200
[<ffffffff9e9f5f07>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff9ef8f653>] __do_page_fault+0x213/0x500
[<ffffffff9ef8fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff9ef8efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9ef8b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 2b 68 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #808313
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff89994737
Build id: 3.10.0-1160.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff897c81ef
Build id: 3.10.0-1160.el7.x86_64
0x36f
3
__alloc_pages_slowpath
vmlinux 0xffffffff89d7c47f
Build id: 3.10.0-1160.el7.x86_64
0x2dd
4
__alloc_pages_nodemask
vmlinux 0xffffffff897c8d76
Build id: 3.10.0-1160.el7.x86_64
0x436
5
alloc_pages_vma
vmlinux 0xffffffff8981c269
Build id: 3.10.0-1160.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffff897f5ec7
Build id: 3.10.0-1160.el7.x86_64
0xcb7
7
__do_page_fault
vmlinux 0xffffffff89d8e653
Build id: 3.10.0-1160.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff89d8ea26
Build id: 3.10.0-1160.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff89d8dfa2
Build id: 3.10.0-1160.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff89d8a7a8
Build id: 3.10.0-1160.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 25s! [rpm:29654]
Modules linked in: tcp_diag inet_diag tun nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc nfit libnvdimm iosf_mbi crc32_pclmul sg ppdev ghash_clmulni_intel parport_pc aesni_intel virtio_balloon parport i2c_piix4 pcspkr joydev lrw gf128mul glue_helper ablk_helper cryptd ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi
cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm 8139too libata crct10dif_pclmul crct10dif_common floppy virtio_pci crc32c_intel virtio_ring 8139cp serio_raw virtio mii drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 29654 Comm: rpm Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff88a447071080 ti: ffff88a44cdf0000 task.ti: ffff88a44cdf0000
RIP: 0010:[<ffffffff89994737>]  [<ffffffff89994737>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff88a44cdf3aa0  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffeb1d8932d680 RCX: 0000000000001000
RDX: ffffffff8a08e488 RSI: 0000000000000000 RDI: ffff88a58cb59000
RBP: ffff88a44cdf3bb0 R08: ffffffff8a08e477 R09: ffffeb1d8932d680
R10: 0000000000003225 R11: fffffffffffffff8 R12: ffff88a447071080
R13: ffffeb1d8932d640 R14: ffffeb1d8932d640 R15: ffff88a67ffda000
FS:  00007f90e823d840(0000) GS:ffff88a67fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000c1c90001 CR3: 00000001071e2000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff897c81ef>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffff89d7c47f>] __alloc_pages_slowpath+0x2dd/0x729
[<ffffffff897c8d76>] __alloc_pages_nodemask+0x436/0x450
[<ffffffff8981c269>] alloc_pages_vma+0xa9/0x200
[<ffffffff897f5ec7>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff89d8e653>] __do_page_fault+0x213/0x500
[<ffffffff89d8ea26>] trace_do_page_fault+0x56/0x150
[<ffffffff89d8dfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff89d8a7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 6b 68 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #601199
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff917aaea7
Build id: 3.10.0-1160.81.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff915d349f
Build id: 3.10.0-1160.81.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffff915d3d84
Build id: 3.10.0-1160.81.1.el7.x86_64
0x184
4
? do_generic_file_read.constprop.52
vmlinux 0xffffffff915ca15c
Build id: 3.10.0-1160.81.1.el7.x86_64
0x34c
5
alloc_pages_vma
vmlinux 0xffffffff91629009
Build id: 3.10.0-1160.81.1.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffff91602257
Build id: 3.10.0-1160.81.1.el7.x86_64
0xcb7
7
__do_page_fault
vmlinux 0xffffffff91bbf7d3
Build id: 3.10.0-1160.81.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff91bbfbb6
Build id: 3.10.0-1160.81.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff91bbf112
Build id: 3.10.0-1160.81.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff91bbb7e8
Build id: 3.10.0-1160.81.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 43s! [yumBackend.py:3270]
Modules linked in: tcp_lp uinput raid1 xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iosf_mbi kvm_intel ext4 snd_hda_codec_generic kvm irqbypass crc32_pclmul mbcache jbd2 snd_hda_intel snd_hda_codec iTCO_wdt iTCO_vendor_support ghash_clmulni_intel snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper
cryptd sg snd_timer lpc_ich joydev i2c_i801 pcspkr snd soundcore virtio_balloon ip_tables xfs libcrc32c sr_mod cdrom qxl virtio_net drm_kms_helper net_failover failover virtio_console virtio_blk ahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libahci crct10dif_pclmul libata crct10dif_common crc32c_intel drm serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 3270 Comm: yumBackend.py Kdump: loaded Not tainted 3.10.0-1160.81.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.1-2.fc36 04/01/2014
task: ffff916836fa6300 ti: ffff91680c210000 task.ti: ffff91680c210000
RIP: 0010:[<ffffffff917aaea7>]  [<ffffffff917aaea7>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff91680c213b08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffff12a804c86c0 RCX: 0000000000001000
RDX: ffffffff91e8fca8 RSI: 0000000000000000 RDI: ffff9167d3200000
RBP: ffff91680c213c18 R08: ffffffff91e8fc97 R09: fffff12a804c8040
R10: 0000000000003688 R11: ffff916809739000 R12: 00000000000000d0
R13: ffff916804fa8438 R14: ffff916814e931f0 R15: 0000000000000000
FS:  00007f7208846740(0000) GS:ffff91683fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f71abe0c024 CR3: 000000003d3c0000 CR4: 0000000000160ff0
Call Trace:
[<ffffffff915d349f>] ? get_page_from_freelist+0x36f/0xad0
[<ffffffff915d3d84>] __alloc_pages_nodemask+0x184/0xbf0
[<ffffffff915ca15c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffff91629009>] alloc_pages_vma+0xa9/0x210
[<ffffffff91602257>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff91bbf7d3>] __do_page_fault+0x213/0x510
[<ffffffff91bbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff91bbf112>] do_async_page_fault+0x22/0x100
[<ffffffff91bbb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 c6 46 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #600224
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff84595397
Build id: 3.10.0-1160.49.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff843c862f
Build id: 3.10.0-1160.49.1.el7.x86_64
0x36f
3
? __radix_tree_lookup
vmlinux 0xffffffff8458dc44
Build id: 3.10.0-1160.49.1.el7.x86_64
0x84
4
__alloc_pages_nodemask
vmlinux 0xffffffff843c8f04
Build id: 3.10.0-1160.49.1.el7.x86_64
0x184
5
? filemap_fault
vmlinux 0xffffffff843c0ead
Build id: 3.10.0-1160.49.1.el7.x86_64
0x17d
6
alloc_pages_vma
vmlinux 0xffffffff8441cc49
Build id: 3.10.0-1160.49.1.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffff843f6837
Build id: 3.10.0-1160.49.1.el7.x86_64
0xcb7
8
__do_page_fault
vmlinux 0xffffffff84990653
Build id: 3.10.0-1160.49.1.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffff84990a26
Build id: 3.10.0-1160.49.1.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff8498ffa2
Build id: 3.10.0-1160.49.1.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffff8498c7a8
Build id: 3.10.0-1160.49.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 39s! [abrt-action-gen:7550]
Modules linked in: uinput xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod nfit libnvdimm snd_hda_codec_generic iosf_mbi snd_hda_intel crc32_pclmul ghash_clmulni_intel aesni_intel ppdev snd_hda_codec lrw gf128mul glue_helper snd_hda_core snd_hwdep snd_seq ablk_helper snd_seq_device cryptd snd_pcm pcspkr
snd_timer joydev snd virtio_rng virtio_balloon soundcore sg parport_pc parport i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm virtio_net ata_piix net_failover virtio_console virtio_blk failover crct10dif_pclmul crct10dif_common libata crc32c_intel floppy serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks fuse
CPU: 1 PID: 7550 Comm: abrt-action-gen Kdump: loaded Tainted: G             L ------------   3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff915cf87de300 ti: ffff915e5407c000 task.ti: ffff915e5407c000
RIP: 0010:[<ffffffff84595397>]  [<ffffffff84595397>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff915e5407fb08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffe2a306740080 RCX: 0000000000001000
RDX: ffffffff84c8ed70 RSI: 0000000000000000 RDI: ffff915ddd001000
RBP: ffff915e5407fc18 R08: ffffffff84c8ed5f R09: ffffe2a306740080
R10: 00000000000034aa R11: ffff915caf510d80 R12: ffff915cf87de300
R13: ffffe2a306740040 R14: ffffe2a306740040 R15: ffff915e803d8000
FS:  00007f6e181ce740(0000) GS:ffff915e80100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000005dc90001 CR3: 00000002141d2000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff843c862f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffff8458dc44>] ? __radix_tree_lookup+0x84/0xf0
[<ffffffff843c8f04>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffff843c0ead>] ? filemap_fault+0x17d/0x420
[<ffffffff8441cc49>] alloc_pages_vma+0xa9/0x200
[<ffffffff843f6837>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff84990653>] __do_page_fault+0x213/0x500
[<ffffffff84990a26>] trace_do_page_fault+0x56/0x150
[<ffffffff8498ffa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8498c7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 5b 5c d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #597327
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffb0b93957
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffb09c868f
Build id: 3.10.0-1127.19.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffffb09c8f26
Build id: 3.10.0-1127.19.1.el7.x86_64
0x166
4
alloc_pages_vma
vmlinux 0xffffffffb0a1c339
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa9
5
handle_mm_fault
vmlinux 0xffffffffb09f6017
Build id: 3.10.0-1127.19.1.el7.x86_64
0xcb7
6
? unmap_region
vmlinux 0xffffffffb09f9234
Build id: 3.10.0-1127.19.1.el7.x86_64
0xf4
7
__do_page_fault
vmlinux 0xffffffffb0f8d653
Build id: 3.10.0-1127.19.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffffb0f8da26
Build id: 3.10.0-1127.19.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffffb0f8cfa2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffffb0f897a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [C1 CompilerThre:2937]
Modules linked in: uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc snd_hda_codec_generic snd_hda_intel snd_hda_codec iosf_mbi snd_hda_core kvm_intel snd_hwdep kvm snd_seq snd_seq_device snd_pcm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper snd_timer cryptd snd joydev pcspkr parport_pc i2c_piix4 soundcore sg virtio_balloon parport ip_tables xfs libcrc32c qxl drm_kms_helper sr_mod cdrom syscopyarea sysfillrect virtio_net sysimgblt ata_generic pata_acpi fb_sys_fops net_failover virtio_console ttm failover
virtio_blk drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci serio_raw virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 2937 Comm: C1 CompilerThre Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff915a5c13e2a0 ti: ffff915a0d774000 task.ti: ffff915a0d774000
RIP: 0010:[<ffffffffb0b93957>]  [<ffffffffb0b93957>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff915a0d777b90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffffd6d81547280 RCX: 0000000000001000
RDX: ffffffffb128d5a0 RSI: 0000000000000000 RDI: ffff915a71345000
RBP: ffff915a0d777ca0 R08: ffffffffb128d58f R09: fffffd6d81c4d180
R10: 0000000000004104 R11: ffff915a10c85e58 R12: ffff915a5c13e2a0
R13: fffffd6d81547240 R14: fffffd6d81547240 R15: ffff915b3ffd7800
FS:  00007f10d9bfb700(0000) GS:ffff915b3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f10b4040000 CR3: 000000003e35c000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb09c868f>] ? get_page_from_freelist+0x36f/0xaa0
[<ffffffffb09c8f26>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffb0a1c339>] alloc_pages_vma+0xa9/0x200
[<ffffffffb09f6017>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffb09f9234>] ? unmap_region+0xf4/0x140
[<ffffffffb0f8d653>] __do_page_fault+0x213/0x500
[<ffffffffb0f8da26>] trace_do_page_fault+0x56/0x150
[<ffffffffb0f8cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb0f897a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 3b 81 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #545927
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffbbd95197
Build id: 3.10.0-1160.76.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffbbbc866f
Build id: 3.10.0-1160.76.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffffbbbc8f44
Build id: 3.10.0-1160.76.1.el7.x86_64
0x184
4
? do_generic_file_read.constprop.52
vmlinux 0xffffffffbbbbf66c
Build id: 3.10.0-1160.76.1.el7.x86_64
0x34c
5
? generic_file_aio_read
vmlinux 0xffffffffbbbc01d9
Build id: 3.10.0-1160.76.1.el7.x86_64
0x1c9
6
alloc_pages_vma
vmlinux 0xffffffffbbc1cc49
Build id: 3.10.0-1160.76.1.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffffbbbf6867
Build id: 3.10.0-1160.76.1.el7.x86_64
0xcb7
8
? __check_object_size
vmlinux 0xffffffffbbc4b16a
Build id: 3.10.0-1160.76.1.el7.x86_64
0x1ca
9
__do_page_fault
vmlinux 0xffffffffbc194653
Build id: 3.10.0-1160.76.1.el7.x86_64
0x213
10
trace_do_page_fault
vmlinux 0xffffffffbc194a26
Build id: 3.10.0-1160.76.1.el7.x86_64
0x56
11
do_async_page_fault
vmlinux 0xffffffffbc193fa2
Build id: 3.10.0-1160.76.1.el7.x86_64
0x22
12
async_page_fault
vmlinux 0xffffffffbc1907a8
Build id: 3.10.0-1160.76.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 28s! [abrt-action-ure:19561]
Modules linked in: tcp_lp uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink arc4 md4 nls_utf8 cifs ccm dns_resolver sunrpc ext4 mbcache jbd2 snd_hda_codec_generic snd_hda_intel iosf_mbi snd_hda_codec crc32_pclmul snd_hda_core ghash_clmulni_intel snd_hwdep aesni_intel snd_seq lrw snd_seq_device snd_pcm gf128mul snd_timer snd soundcore glue_helper ablk_helper virtio_balloon joydev ppdev cryptd i2c_piix4 parport_pc parport pcspkr ip_tables xfs libcrc32c ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm virtio_net
net_failover libata virtio_console virtio_blk failover crct10dif_pclmul serio_raw crct10dif_common crc32c_intel virtio_pci virtio_ring floppy virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 19561 Comm: abrt-action-ure Kdump: loaded Tainted: G             L ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9dfb8597b180 ti: ffff9df887f98000 task.ti: ffff9df887f98000
RIP: 0010:[<ffffffffbbd95197>]  [<ffffffffbbd95197>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9df887f9bb08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: fffffa52884a4540 RCX: 0000000000001000
RDX: ffffffffbc48f4c8 RSI: 0000000000000000 RDI: ffff9dfa92914000
RBP: ffff9df887f9bc18 R08: ffffffffbc48f4b7 R09: fffffa52884a4540
R10: 0000000000003eae R11: ffff9df887f89290 R12: ffff9dfb8597b180
R13: fffffa52884a4500 R14: fffffa52884a4500 R15: ffff9dfbbffd2000
FS:  00007f7a7378a740(0000) GS:ffff9dfbbfcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000002e5b0003 CR3: 0000000007f1a000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffbbbc866f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffbbbc8f44>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffffbbbbf66c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffbbbc01d9>] ? generic_file_aio_read+0x1c9/0x290
[<ffffffffbbc1cc49>] alloc_pages_vma+0xa9/0x200
[<ffffffffbbbf6867>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffbbc4b16a>] ? __check_object_size+0x1ca/0x250
[<ffffffffbc194653>] __do_page_fault+0x213/0x500
[<ffffffffbc194a26>] trace_do_page_fault+0x56/0x150
[<ffffffffbc193fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffbc1907a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 eb 5f d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #533866
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff82394c17
Build id: 3.10.0-1160.15.2.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff821c835f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffff821c8c16
Build id: 3.10.0-1160.15.2.el7.x86_64
0x166
4
alloc_pages_vma
vmlinux 0xffffffff8221c449
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa9
5
handle_mm_fault
vmlinux 0xffffffff821f6047
Build id: 3.10.0-1160.15.2.el7.x86_64
0xcb7
6
__do_page_fault
vmlinux 0xffffffff8278f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffff8278fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffff8278efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffff8278b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#40 stuck for 21s! [java:55034]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14203(E) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_pinned_14108(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm qxl snd_timer ttm drm_kms_helper iTCO_wdt syscopyarea
crc32_pclmul ghash_clmulni_intel aesni_intel lrw sysfillrect gf128mul sysimgblt glue_helper ppdev fb_sys_fops ablk_helper snd iTCO_vendor_support joydev cryptd parport_pc drm virtio_rng pcspkr i2c_i801 lpc_ich soundcore virtio_balloon parport drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net net_failover virtio_console virtio_blk failover crct10dif_pclmul ahci virtio_pci libahci crct10dif_common serio_raw crc32c_intel libata virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 40 PID: 55034 Comm: java Kdump: loaded Tainted: P        W   EL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff8b1fb1c94200 ti: ffff8b1e22bac000 task.ti: ffff8b1e22bac000
RIP: 0010:[<ffffffff82394c17>]  [<ffffffff82394c17>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8b1e22bafb90  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 000000000000001e RCX: 0000000000001000
RDX: ffffffff82a8e578 RSI: 0000000000000000 RDI: ffff8c5d10a40000
RBP: ffff8b1e22bafca0 R08: ffffffff82a8e567 R09: fffff70d38429040
R10: 000000000000dda2 R11: ffff8b118abafcb0 R12: ffff8b1e22bafb18
R13: 0000000000000000 R14: ffff8d037ffd8000 R15: 0000000000000009
FS:  00007f71d0be2700(0000) GS:ffff8cfbac000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000005bc9b6ff0 CR3: 00000005f73be000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffff821c835f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffff821c8c16>] __alloc_pages_nodemask+0x166/0x450
[<ffffffff8221c449>] alloc_pages_vma+0xa9/0x200
[<ffffffff821f6047>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff8278f653>] __do_page_fault+0x213/0x500
[<ffffffff8278fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff8278efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8278b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 1b 63 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 66 66 66 90 66 66 66 90 66 66 
Complete report #531637
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffa9795177
Build id: 3.10.0-1160.62.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffa95c863f
Build id: 3.10.0-1160.62.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffffa95c8f14
Build id: 3.10.0-1160.62.1.el7.x86_64
0x184
4
? unmap_region
vmlinux 0xffffffffa95f9a44
Build id: 3.10.0-1160.62.1.el7.x86_64
0xf4
5
alloc_pages_vma
vmlinux 0xffffffffa961cc39
Build id: 3.10.0-1160.62.1.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffffa95f6827
Build id: 3.10.0-1160.62.1.el7.x86_64
0xcb7
7
__do_page_fault
vmlinux 0xffffffffa9b94653
Build id: 3.10.0-1160.62.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffffa9b94a26
Build id: 3.10.0-1160.62.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffffa9b93fa2
Build id: 3.10.0-1160.62.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffffa9b907a8
Build id: 3.10.0-1160.62.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 60s! [oracle:28170]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod sg iosf_mbi crc32_pclmul virtio_balloon ghash_clmulni_intel aesni_intel i2c_piix4 lrw gf128mul glue_helper pcspkr joydev ablk_helper cryptd binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod cdrom
sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover virtio_console virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common virtio_pci libata virtio_ring crc32c_intel drm_panel_orientation_quirks serio_raw virtio floppy fuse
CPU: 0 PID: 28170 Comm: oracle Tainted: G             L ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffff9ec6b72c4200 ti: ffff9ec6401fc000 task.ti: ffff9ec6401fc000
RIP: 0010:[<ffffffffa9795177>]  [<ffffffffa9795177>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9ec6401ffb08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffe84c02dfea80 RCX: 0000000000001000
RDX: ffffffffa9e8f2e8 RSI: 0000000000000000 RDI: ffff9ec58ff41000
RBP: ffff9ec6401ffc18 R08: ffffffffa9e8f2d7 R09: ffffe84c013fd080
R10: 0000000000003ca2 R11: ffff9ec6403c0870 R12: ffff9ec6b72c4200
R13: ffffe84c02dfea40 R14: ffffe84c02dfea40 R15: ffff9ec8fffd9800
FS:  00007f78df752740(0000) GS:ffff9ec8ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f78d9284e60 CR3: 00000000616ba000 CR4: 0000000000360ff0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa95c863f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffa95c8f14>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffffa95f9a44>] ? unmap_region+0xf4/0x140
[<ffffffffa961cc39>] alloc_pages_vma+0xa9/0x200
[<ffffffffa95f6827>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffa9b94653>] __do_page_fault+0x213/0x500
[<ffffffffa9b94a26>] trace_do_page_fault+0x56/0x150
[<ffffffffa9b93fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffa9b907a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 0b 60 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #489738
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffb8d94c17
Build id: 3.10.0-1160.15.2.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffb8bc835f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x36f
3
? do_generic_file_read.constprop.52
vmlinux 0xffffffffb8bbf33c
Build id: 3.10.0-1160.15.2.el7.x86_64
0x34c
4
? generic_file_aio_read
vmlinux 0xffffffffb8bbfea9
Build id: 3.10.0-1160.15.2.el7.x86_64
0x1c9
5
__alloc_pages_nodemask
vmlinux 0xffffffffb8bc8c16
Build id: 3.10.0-1160.15.2.el7.x86_64
0x166
6
alloc_pages_vma
vmlinux 0xffffffffb8c1c449
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffffb8bf6047
Build id: 3.10.0-1160.15.2.el7.x86_64
0xcb7
8
__do_page_fault
vmlinux 0xffffffffb918f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffffb918fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffffb918efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffffb918b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#128 stuck for 21s! [sh:60991]
NMI watchdog: BUG: soft lockup - CPU#235 stuck for 22s! [samtools:61084]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13707(E) falcon_lsm_pinned_13601(E) veth ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter overlay(T) tcp_lp rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_pinned_13501(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec
snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer qxl ttm snd drm_kms_helper ppdev syscopyarea sysfillrect iTCO_wdt iTCO_vendor_support crc32_pclmul parport_pc ghash_clmulni_intel aesni_intel sysimgblt lrw gf128mul pcspkr fb_sys_fops glue_helper ablk_helper soundcore lpc_ich cryptd joydev parport i2c_i801 virtio_balloon virtio_rng drm drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net virtio_console virtio_blk net_failover failover ahci libahci crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel libata virtio_ring serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 235 PID: 61084 Comm: samtools Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff8f881f7a9080 ti: ffff8f93201d8000 task.ti: ffff8f93201d8000
RIP: 0010:[<ffffffffb8d94c17>]  [<ffffffffb8d94c17>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8f93201dbb90  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 000000000000001e RCX: 0000000000001000
RDX: ffffffffb948e578 RSI: 0000000000000000 RDI: ffff9086c4a9d000
RBP: ffff8f93201dbca0 R08: ffffffffb948e567 R09: ffffdbb85e12a780
R10: 000000000000dda2 R11: ffff8f9292738510 R12: ffff8f93201dbb18
R13: 0000000000000000 R14: ffffffffb91870f2 R15: 00000000000001c1
FS:  00007f962a067740(0000) GS:ffff916bef0c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001e35518 CR3: 000000115aa2e000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffffb8bc835f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffb8bbf33c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffb8bbfea9>] ? generic_file_aio_read+0x1c9/0x290
[<ffffffffb8bc8c16>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffb8c1c449>] alloc_pages_vma+0xa9/0x200
[<ffffffffb8bf6047>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffb918f653>] __do_page_fault+0x213/0x500
[<ffffffffb918fa26>] trace_do_page_fault+0x56/0x150
[<ffffffffb918efa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb918b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 1b 63 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 66 66 66 90 66 66 66 90 66 66 
Complete report #479088
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffb8d94c17
Build id: 3.10.0-1160.15.2.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffb8bc835f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x36f
3
? do_generic_file_read.constprop.52
vmlinux 0xffffffffb8bbf33c
Build id: 3.10.0-1160.15.2.el7.x86_64
0x34c
4
? generic_file_aio_read
vmlinux 0xffffffffb8bbfea9
Build id: 3.10.0-1160.15.2.el7.x86_64
0x1c9
5
__alloc_pages_nodemask
vmlinux 0xffffffffb8bc8c16
Build id: 3.10.0-1160.15.2.el7.x86_64
0x166
6
? nfs_end_io_read
nfs 0xffffffffc07af475
Build id: 3.10.0-1160.15.2.el7.x86_64
0x15
7
? do_sync_read
vmlinux 0xffffffffb8c4d163
Build id: 3.10.0-1160.15.2.el7.x86_64
0x93
8
alloc_pages_vma
vmlinux 0xffffffffb8c1c449
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa9
9
handle_mm_fault
vmlinux 0xffffffffb8bf6047
Build id: 3.10.0-1160.15.2.el7.x86_64
0xcb7
10
__do_page_fault
vmlinux 0xffffffffb918f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
11
trace_do_page_fault
vmlinux 0xffffffffb918fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
12
do_async_page_fault
vmlinux 0xffffffffb918efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
13
async_page_fault
vmlinux 0xffffffffb918b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#150 stuck for 21s! [samtools:137837]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13707(E) falcon_lsm_pinned_13601(E) veth ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter overlay(T) tcp_lp rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_pinned_13501(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec
snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer qxl ttm snd drm_kms_helper ppdev syscopyarea sysfillrect iTCO_wdt iTCO_vendor_support crc32_pclmul parport_pc ghash_clmulni_intel aesni_intel sysimgblt lrw gf128mul pcspkr fb_sys_fops glue_helper ablk_helper soundcore lpc_ich cryptd joydev parport i2c_i801 virtio_balloon virtio_rng drm drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net virtio_console virtio_blk net_failover failover ahci libahci crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel libata virtio_ring serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 150 PID: 137837 Comm: samtools Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff8f9212162100 ti: ffff8f915b7c0000 task.ti: ffff8f915b7c0000
RIP: 0010:[<ffffffffb8d94c17>]  [<ffffffffb8d94c17>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8f915b7c3b90  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8f915b7c3fd8 RCX: 0000000000001000
RDX: ffffffffb948e578 RSI: 0000000000000000 RDI: ffff8fdfdb86c000
RBP: ffff8f915b7c3ca0 R08: ffffffffb948e567 R09: ffffdbb5c26e1b40
R10: 000000000000dda2 R11: ffff8f885d7e8510 R12: ffff8f915b7c3b18
R13: ffff8f915b7c3bb8 R14: ffffffffb91870f2 R15: ffffdbb7000001c1
FS:  00007fee1e336740(0000) GS:ffff916bedb80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000002779048 CR3: 00000007cb3be000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffffb8bc835f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffb8bbf33c>] ? do_generic_file_read.constprop.52+0x34c/0x510
[<ffffffffb8bbfea9>] ? generic_file_aio_read+0x1c9/0x290
[<ffffffffb8bc8c16>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffc07af475>] ? nfs_end_io_read+0x15/0x20 [nfs]
[<ffffffffb8c4d163>] ? do_sync_read+0x93/0xe0
[<ffffffffb8c1c449>] alloc_pages_vma+0xa9/0x200
[<ffffffffb8bf6047>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffb918f653>] __do_page_fault+0x213/0x500
[<ffffffffb918fa26>] trace_do_page_fault+0x56/0x150
[<ffffffffb918efa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb918b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 1b 63 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 66 66 66 90 66 66 66 90 66 66 
Complete report #444917
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffb8d94c17
Build id: 3.10.0-1160.15.2.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffb8bc835f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffffb8bc8c16
Build id: 3.10.0-1160.15.2.el7.x86_64
0x166
4
alloc_pages_vma
vmlinux 0xffffffffb8c1c449
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa9
5
handle_mm_fault
vmlinux 0xffffffffb8bf6047
Build id: 3.10.0-1160.15.2.el7.x86_64
0xcb7
6
__do_page_fault
vmlinux 0xffffffffb918f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffffb918fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffffb918efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffffb918b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#95 stuck for 21s! [samtools:136361]
Modules linked in: veth ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter overlay(T) tcp_lp rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13501(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device
snd_pcm snd_timer qxl ttm snd drm_kms_helper ppdev syscopyarea sysfillrect iTCO_wdt iTCO_vendor_support crc32_pclmul parport_pc ghash_clmulni_intel aesni_intel sysimgblt lrw gf128mul pcspkr fb_sys_fops glue_helper ablk_helper soundcore lpc_ich cryptd joydev parport i2c_i801 virtio_balloon virtio_rng drm drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net virtio_console virtio_blk net_failover failover ahci libahci crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel libata virtio_ring serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 95 PID: 136361 Comm: samtools Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff8f8ba03e4200 ti: ffff8f8197bb0000 task.ti: ffff8f8197bb0000
RIP: 0010:[<ffffffffb8d94c17>]  [<ffffffffb8d94c17>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8f8197bb3b90  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 000000000000001e RCX: 0000000000001000
RDX: ffffffffb948e578 RSI: 0000000000000000 RDI: ffff90ed8f5ce000
RBP: ffff8f8197bb3ca0 R08: ffffffffb948e567 R09: ffffdbb9f93d73c0
R10: 000000000000dda2 R11: ffff8f90c3302360 R12: ffff8f8197bb3b18
R13: 0000000000000000 R14: ffff9173bffd7000 R15: 0000000000000009
FS:  00007fc4f2611740(0000) GS:ffff916becdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc4edfe1000 CR3: 00000001a2798000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffffb8bc835f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffb8bc8c16>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffb8c1c449>] alloc_pages_vma+0xa9/0x200
[<ffffffffb8bf6047>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffb918f653>] __do_page_fault+0x213/0x500
[<ffffffffb918fa26>] trace_do_page_fault+0x56/0x150
[<ffffffffb918efa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb918b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 1b 63 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 66 66 66 90 66 66 66 90 66 66 
Complete report #285508
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffb4995347
Build id: 3.10.0-1160.36.2.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffb47c860f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffffb47c8ee4
Build id: 3.10.0-1160.36.2.el7.x86_64
0x184
4
? flush_tlb_page
vmlinux 0xffffffffb467e3d8
Build id: 3.10.0-1160.36.2.el7.x86_64
0x38
5
? lru_cache_add
vmlinux 0xffffffffb47ce09e
Build id: 3.10.0-1160.36.2.el7.x86_64
0xe
6
alloc_pages_vma
vmlinux 0xffffffffb481cc19
Build id: 3.10.0-1160.36.2.el7.x86_64
0xa9
7
handle_mm_fault
vmlinux 0xffffffffb47f6817
Build id: 3.10.0-1160.36.2.el7.x86_64
0xcb7
8
__do_page_fault
vmlinux 0xffffffffb4d90653
Build id: 3.10.0-1160.36.2.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffffb4d90a26
Build id: 3.10.0-1160.36.2.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffffb4d8ffa2
Build id: 3.10.0-1160.36.2.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffffb4d8c7a8
Build id: 3.10.0-1160.36.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [sh:5613]
Modules linked in: iptable_nat nf_nat_ipv4 nf_nat ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_multiport xt_conntrack nf_conntrack iptable_filter sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd parport_pc joydev parport pcspkr virtio_balloon i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi virtio_net net_failover virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common drm_panel_orientation_quirks crc32c_intel virtio_pci floppy virtio_ring serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: fidelis]
CPU: 0 PID: 5613 Comm: sh Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: OpenStack Foundation OpenStack Nova, BIOS 1.10.2-1.1~u16.04+mcp2 04/01/2014
task: ffff9aaa935cd280 ti: ffff9aa4a1620000 task.ti: ffff9aa4a1620000
RIP: 0010:[<ffffffffb4995347>]  [<ffffffffb4995347>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9aa4a1623b08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000246 RCX: 0000000000001000
RDX: ffffffffb508ed40 RSI: 0000000000000000 RDI: ffff9aa4ca7d4000
RBP: ffff9aa4a1623c18 R08: ffffffffb508ed2f R09: fffff8fb8729f540
R10: 0000000000004b35 R11: ffff9aab1a45a438 R12: 0000000000000000
R13: ffff9aa39f621a00 R14: 00000000000001c1 R15: 0000000000000000
FS:  00007f6c54e3f740(0000) GS:ffff9aab1f200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001cae008 CR3: 0000000141c5a000 CR4: 00000000003606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb47c860f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffb47c8ee4>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffffb467e3d8>] ? flush_tlb_page+0x38/0xa0
[<ffffffffb47ce09e>] ? lru_cache_add+0xe/0x10
[<ffffffffb481cc19>] alloc_pages_vma+0xa9/0x200
[<ffffffffb47f6817>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffb4d90653>] __do_page_fault+0x213/0x500
[<ffffffffb4d90a26>] trace_do_page_fault+0x56/0x150
[<ffffffffb4d8ffa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb4d8c7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 8b 5c d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #270876
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff82b95397
Build id: 3.10.0-1160.45.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff829c862f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffff829c8f04
Build id: 3.10.0-1160.45.1.el7.x86_64
0x184
4
alloc_pages_vma
vmlinux 0xffffffff82a1cc49
Build id: 3.10.0-1160.45.1.el7.x86_64
0xa9
5
handle_mm_fault
vmlinux 0xffffffff829f6837
Build id: 3.10.0-1160.45.1.el7.x86_64
0xcb7
6
? switch_mm_irqs_off
vmlinux 0xffffffff8287dff5
Build id: 3.10.0-1160.45.1.el7.x86_64
0x115
7
__do_page_fault
vmlinux 0xffffffff82f90653
Build id: 3.10.0-1160.45.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff82f90a26
Build id: 3.10.0-1160.45.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff82f8ffa2
Build id: 3.10.0-1160.45.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff82f8c7a8
Build id: 3.10.0-1160.45.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#17 stuck for 22s! [worker:12032]
Modules linked in: xt_multiport rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sunrpc snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer ppdev iosf_mbi snd crc32_pclmul soundcore ghash_clmulni_intel sg aesni_intel virtio_rng lrw gf128mul parport_pc virtio_balloon joydev glue_helper parport ablk_helper i2c_piix4 cryptd pcspkr binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover qxl virtio_console virtio_blk failover drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul libata crct10dif_common floppy virtio_pci crc32c_intel
virtio_ring serio_raw drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 17 PID: 12032 Comm: worker Kdump: loaded Tainted: G             L ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff88c3ffda5280 ti: ffff88e9a9894000 task.ti: ffff88e9a9894000
RIP: 0010:[<ffffffff82b95397>]  [<ffffffff82b95397>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff88e9a9897b08  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffd38062565e80 RCX: 0000000000001000
RDX: ffffffff8328ed70 RSI: 0000000000000000 RDI: ffff88e31597a000
RBP: ffff88e9a9897c18 R08: ffffffff8328ed5f R09: ffffd38062565ec0
R10: 0000000000005197 R11: ffff88c8ab039950 R12: ffff88c3ffda5280
R13: ffffd38062565e40 R14: ffffd38062565e40 R15: ffff88eabffda000
FS:  00002b341975aec0(0000) GS:ffff88e9bf440000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002b34a8eb2000 CR3: 000000379e3e6000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff829c862f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffff829c8f04>] __alloc_pages_nodemask+0x184/0xbe0
[<ffffffff82a1cc49>] alloc_pages_vma+0xa9/0x200
[<ffffffff829f6837>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff8287dff5>] ? switch_mm_irqs_off+0x115/0x290
[<ffffffff82f90653>] __do_page_fault+0x213/0x500
[<ffffffff82f90a26>] trace_do_page_fault+0x56/0x150
[<ffffffff82f8ffa2>] do_async_page_fault+0x22/0xf0
[<ffffffff82f8c7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 5b 5c d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #224947
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffbb194c17
Build id: 3.10.0-1160.15.2.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffbafc835f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x36f
3
? flush_tlb_page
vmlinux 0xffffffffbae7e9e8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x38
4
__alloc_pages_nodemask
vmlinux 0xffffffffbafc8c16
Build id: 3.10.0-1160.15.2.el7.x86_64
0x166
5
alloc_pages_vma
vmlinux 0xffffffffbb01c449
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffffbaff6047
Build id: 3.10.0-1160.15.2.el7.x86_64
0xcb7
7
__do_page_fault
vmlinux 0xffffffffbb58f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffffbb58fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffffbb58efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffffbb58b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 29s! [/usr/bin/spamd :9914]
Modules linked in: ip_set nfnetlink nf_log_ipv6 nf_log_ipv4 nf_log_common nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_LOG xt_limit iptable_nat nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_nat_ipv6 nf_nat iptable_mangle iptable_raw ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables nf_conntrack_ftp ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_multiport xt_conntrack nf_conntrack iptable_filter ext4 mbcache jbd2 ppdev sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper sg cryptd parport_pc joydev pcspkr virtio_balloon parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc fuse ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover
virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 9914 Comm: /usr/bin/spamd  Kdump: loaded Tainted: G             L ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
task: ffff952d63dea100 ti: ffff952d7ce08000 task.ti: ffff952d7ce08000
RIP: 0010:[<ffffffffbb194c17>]  [<ffffffffbb194c17>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff952d7ce0bb90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000030 RCX: 0000000000001000
RDX: ffffffffbb88e578 RSI: 0000000000000000 RDI: ffff95233595f000
RBP: ffff952d7ce0bca0 R08: ffffffffbb88e567 R09: fffffe7a80d65800
R10: 000000000000bd70 R11: ffff952455f90000 R12: ffff952fbffda000
R13: 0000000000000000 R14: fffffe7aac1b4a00 R15: ffff952f000001c1
FS:  00007fcfaadb5740(0000) GS:ffff952f8d300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000470e348 CR3: 00000009c0d62000 CR4: 0000000000360fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffbafc835f>] ? get_page_from_freelist+0x36f/0xac0
[<ffffffffbae7e9e8>] ? flush_tlb_page+0x38/0xa0
[<ffffffffbafc8c16>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffbb01c449>] alloc_pages_vma+0xa9/0x200
[<ffffffffbaff6047>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffbb58f653>] __do_page_fault+0x213/0x500
[<ffffffffbb58fa26>] trace_do_page_fault+0x56/0x150
[<ffffffffbb58efa2>] do_async_page_fault+0x22/0xf0
[<ffffffffbb58b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 1b 63 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #210696
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffffa9d901b7
Build id: 3.10.0-1062.18.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffffa9bc7acf
Build id: 3.10.0-1062.18.1.el7.x86_64
0x36f
3
__alloc_pages_nodemask
vmlinux 0xffffffffa9bc8366
Build id: 3.10.0-1062.18.1.el7.x86_64
0x166
4
alloc_pages_vma
vmlinux 0xffffffffa9c1a0e9
Build id: 3.10.0-1062.18.1.el7.x86_64
0xa9
5
handle_mm_fault
vmlinux 0xffffffffa9bf3e67
Build id: 3.10.0-1062.18.1.el7.x86_64
0xcb7
6
__do_page_fault
vmlinux 0xffffffffaa188653
Build id: 3.10.0-1062.18.1.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffffaa188a26
Build id: 3.10.0-1062.18.1.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffffaa187fa2
Build id: 3.10.0-1062.18.1.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffffaa1847a8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [rpm:3937]
Modules linked in: xt_hashlimit ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables bridge stp llc ip_set nfnetlink sunrpc snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq iosf_mbi crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel snd_timer ppdev snd lrw gf128mul glue_helper soundcore ablk_helper cryptd sg pcspkr joydev parport_pc virtio_balloon parport i2c_piix4 xfs libcrc32c sr_mod cdrom
qxl drm_kms_helper ata_generic pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops virtio_blk ttm virtio_net virtio_console drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 3937 Comm: rpm Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8c5c4abd5230 ti: ffff8c5c9456c000 task.ti: ffff8c5c9456c000
RIP: 0010:[<ffffffffa9d901b7>]  [<ffffffffa9d901b7>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8c5c9456fb90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffcfb8006aea40 RCX: 0000000000001000
RDX: ffffffffaa48b210 RSI: 0000000000000000 RDI: ffff8c5c5aba8000
RBP: ffff8c5c9456fca0 R08: ffffffffaa48b1ff R09: ffffcfb8006aea40
R10: 0000000000003688 R11: ffff8c5c75c7f798 R12: ffff8c5c4abd5230
R13: ffffcfb8006aea00 R14: ffffcfb8006aea00 R15: ffff8c5cbffd0800
FS:  00007ff517a65840(0000) GS:ffff8c5cbfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000f9c90001 CR3: 000000002042e000 CR4: 00000000001406e0
Call Trace:
[<ffffffffa9bc7acf>] ? get_page_from_freelist+0x36f/0xaa0
[<ffffffffa9bc8366>] __alloc_pages_nodemask+0x166/0x450
[<ffffffffa9c1a0e9>] alloc_pages_vma+0xa9/0x200
[<ffffffffa9bf3e67>] handle_mm_fault+0xcb7/0xfb0
[<ffffffffaa188653>] __do_page_fault+0x213/0x500
[<ffffffffaa188a26>] trace_do_page_fault+0x56/0x150
[<ffffffffaa187fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffaa1847a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 7b b6 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #14570
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff859901b7
Build id: 3.10.0-1062.18.1.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff857c7acf
Build id: 3.10.0-1062.18.1.el7.x86_64
0x36f
3
? async_page_fault
vmlinux 0xffffffff85d847a8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x28
4
__alloc_pages_nodemask
vmlinux 0xffffffff857c8366
Build id: 3.10.0-1062.18.1.el7.x86_64
0x166
5
alloc_pages_vma
vmlinux 0xffffffff8581a0e9
Build id: 3.10.0-1062.18.1.el7.x86_64
0xa9
6
handle_mm_fault
vmlinux 0xffffffff857f3e67
Build id: 3.10.0-1062.18.1.el7.x86_64
0xcb7
7
__do_page_fault
vmlinux 0xffffffff85d88653
Build id: 3.10.0-1062.18.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff85d88a26
Build id: 3.10.0-1062.18.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff85d87fa2
Build id: 3.10.0-1062.18.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff85d847a8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [yumBackend.py:8549]
Modules linked in: tcp_lp uinput fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink arc4 md4 nls_utf8 cifs ccm dns_resolver sunrpc ext4 mbcache jbd2 ppdev snd_hda_codec_generic snd_hda_intel iosf_mbi crc32_pclmul snd_hda_codec ghash_clmulni_intel snd_hda_core aesni_intel lrw snd_hwdep gf128mul snd_seq glue_helper snd_seq_device ablk_helper cryptd snd_pcm snd_timer snd pcspkr soundcore parport_pc joydev virtio_balloon parport i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper ata_piix crct10dif_pclmul
crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm libata drm serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 8549 Comm: yumBackend.py Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9f4cd855b150 ti: ffff9f4ac3f24000 task.ti: ffff9f4ac3f24000
RIP: 0010:[<ffffffff859901b7>]  [<ffffffff859901b7>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff9f4ac3f27b90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffeff28551ae00 RCX: 0000000000001000
RDX: ffffffff8608b210 RSI: 0000000000000000 RDI: ffff9f4c146b7000
RBP: ffff9f4ac3f27ca0 R08: ffffffff8608b1ff R09: ffffeff28551ae00
R10: 0000000000003476 R11: ffff9f4cf4ff4288 R12: ffff9f4cd855b150
R13: ffffeff28551adc0 R14: ffffeff28551adc0 R15: ffff9f4cfffda000
FS:  00007f6d9a609740(0000) GS:ffff9f4cffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000046c90001 CR3: 0000000152b9e000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff857c7acf>] ? get_page_from_freelist+0x36f/0xaa0
[<ffffffff85d847a8>] ? async_page_fault+0x28/0x30
[<ffffffff857c8366>] __alloc_pages_nodemask+0x166/0x450
[<ffffffff8581a0e9>] alloc_pages_vma+0xa9/0x200
[<ffffffff857f3e67>] handle_mm_fault+0xcb7/0xfb0
[<ffffffff85d88653>] __do_page_fault+0x213/0x500
[<ffffffff85d88a26>] trace_do_page_fault+0x56/0x150
[<ffffffff85d87fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff85d847a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 7b b6 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
Complete report #11468
Frame # Function Binary Source or offset Line
1
clear_page_c_e
vmlinux 0xffffffff8132ff97
Build id: 3.10.0-693.5.2.el7.x86_64
0x7
2
? get_page_from_freelist
vmlinux 0xffffffff8118c4ca
Build id: 3.10.0-693.5.2.el7.x86_64
0x4fa
3
__alloc_pages_nodemask
vmlinux 0xffffffff8118cb26
Build id: 3.10.0-693.5.2.el7.x86_64
0x176
4
alloc_pages_vma
vmlinux 0xffffffff811d40a5
Build id: 3.10.0-693.5.2.el7.x86_64
0xb5
5
handle_mm_fault
vmlinux 0xffffffff811b2350
Build id: 3.10.0-693.5.2.el7.x86_64
0xb60
6
__do_page_fault
vmlinux 0xffffffff816b0074
Build id: 3.10.0-693.5.2.el7.x86_64
0x154
7
? __switch_to
vmlinux 0xffffffff810295da
Build id: 3.10.0-693.5.2.el7.x86_64
0x15a
8
trace_do_page_fault
vmlinux 0xffffffff816b0456
Build id: 3.10.0-693.5.2.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff816afaea
Build id: 3.10.0-693.5.2.el7.x86_64
0x1a
10
async_page_fault
vmlinux 0xffffffff816ac5f8
Build id: 3.10.0-693.5.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [httpd:3348]
Modules linked in: fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_hda_codec_generic snd_hda_intel snd_hda_codec ppdev snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd soundcore parport_pc joydev pcspkr parport i2c_piix4 virtio_balloon nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi qxl drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm ahci virtio_net virtio_console virtio_blk drm libahci ata_piix libata crct10dif_pclmul crct10dif_common serio_raw virtio_pci crc32c_intel virtio_ring virtio i2c_core floppy
CPU: 2 PID: 3348 Comm: httpd Tainted: G             L ------------   3.10.0-693.5.2.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8800a06a1fa0 ti: ffff88006af30000 task.ti: ffff88006af30000
RIP: 0010:[<ffffffff8132ff97>]  [<ffffffff8132ff97>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff88006af33bd0  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88006af33fd8 RCX: 0000000000001000
RDX: ffffffff81911198 RSI: 0000000000000000 RDI: ffff88005d2bb000
RBP: ffff88006af33cd8 R08: ffffffff81911187 R09: ffffea000174af00
R10: ffffea000174af00 R11: 0000000000000000 R12: ffffea000174af00
R13: ffffea000174aec0 R14: ffff88006af30000 R15: ffffea000174aec0
FS:  00007f53cadd1880(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000cd920002 CR3: 000000006aefb000 CR4: 00000000001406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff8118c4ca 0000000000000081 ffff88013ffd9008 000000006af33bf8
0000000000000002 ffff88006af33c20 ffff88013fd1a170 ffff88013fd1a1a0
ffff88013ffd7868 0000000000000000 0000000300000044 ffff88013ffd9000
Call Trace:
[<ffffffff8118c4ca>] ? get_page_from_freelist+0x4fa/0x9e0
[<ffffffff8118cb26>] __alloc_pages_nodemask+0x176/0x420
[<ffffffff811d40a5>] alloc_pages_vma+0xb5/0x200
[<ffffffff811b2350>] handle_mm_fault+0xb60/0xfa0
[<ffffffff816b0074>] __do_page_fault+0x154/0x450
[<ffffffff810295da>] ? __switch_to+0x15a/0x510
[<ffffffff816b0456>] trace_do_page_fault+0x56/0x150
[<ffffffff816afaea>] do_async_page_fault+0x1a/0xd0
[<ffffffff816ac5f8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 bb 77 d5 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f