Info

Permalink
Function
generic_exec_single
First occurrence
2020-09-23
Last occurrence
2025-03-23
State
NEW
Components
kernel
Quality
-38

Statistics

Operating system Count
CentOS 7 277
Architecture Count
x86_64 277
Related packages Count
kernel 277
 0:3.10.0-1160.el7 34
 0:3.10.0-1160.71.1.el7 28
 0:3.10.0-1160.59.1.el7 20
 0:3.10.0-1160.15.2.el7 20
 0:3.10.0-957.el7 18
 0:3.10.0-1160.62.1.el7 16
 0:3.10.0-1127.13.1.el7 14
 0:3.10.0-1160.99.1.el7 13
 0:3.10.0-1160.88.1.el7 10
 0:3.10.0-1062.el7 9
 0:3.10.0-1160.49.1.el7 8
 0:3.10.0-1160.81.1.el7 8
 0:3.10.0-1160.92.1.el7 7
 0:3.10.0-1127.19.1.el7 5
 0:3.10.0-1160.25.1.el7 5
 0:3.10.0-1160.76.1.el7 5
 0:3.10.0-1160.80.1.el7 5
 0:3.10.0-1062.4.1.el7 5
 0:3.10.0-1127.8.2.el7 4
 0:3.10.0-1160.119.1.el7 4
 0:3.10.0-1127.18.2.el7 4
 0:3.10.0-1160.108.1.el7 4
 0:3.10.0-1160.11.1.el7 4
 0:3.10.0-1160.105.1.el7 3
 0:3.10.0-1160.24.1.el7 3
 0:3.10.0-1160.6.1.el7 3
 0:3.10.0-1062.9.1.el7 3
 0:3.10.0-957.21.3.el7 2
 0:3.10.0-1062.18.1.el7 2
 0:3.10.0-1160.118.1.el7 2
 0:3.10.0-862.el7 2
 0:3.10.0-1160.102.1.el7 1
 0:3.10.0-1160.41.1.el7 1
 0:3.10.0-1160.66.1.el7 1
 0:3.10.0-1160.95.1.el7 1
 0:3.10.0-1160.53.1.el7 1
 0:3.10.0-1160.31.1.el7 1
 0:3.10.0-862.14.4.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1388668
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff86b16562
Build id: 3.10.0-1160.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff86a7e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff86a7e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff86a7e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff86b1667f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff86d85de5
Build id: 3.10.0-1160.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff86b16c2b
Build id: 3.10.0-1160.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff86a7e868
Build id: 3.10.0-1160.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff86a7ea04
Build id: 3.10.0-1160.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff86c07728
Build id: 3.10.0-1160.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff86bef295
Build id: 3.10.0-1160.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff86bf16bb
Build id: 3.10.0-1160.el7.x86_64 		0xfb
13
handle_mm_fault
vmlinux 0xffffffff86bf5d32
Build id: 3.10.0-1160.el7.x86_64 		0xb22
14
__do_page_fault
vmlinux 0xffffffff8718e653
Build id: 3.10.0-1160.el7.x86_64 		0x213
15
trace_do_page_fault
vmlinux 0xffffffff8718ea26
Build id: 3.10.0-1160.el7.x86_64 		0x56
16
do_async_page_fault
vmlinux 0xffffffff8718dfa2
Build id: 3.10.0-1160.el7.x86_64 		0x22
17
async_page_fault
vmlinux 0xffffffff8718a7a8
Build id: 3.10.0-1160.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [pritunl:10869]
Modules linked in: xt_comment udp_diag xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) tun tcp_diag inet_diag snd_hda_codec_generic nfit snd_hda_intel libnvdimm snd_hda_codec snd_hda_core iosf_mbi snd_hwdep crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel ppdev snd_pcm parport_pc aesni_intel parport lrw snd_timer gf128mul glue_helper ablk_helper cryptd snd pcspkr joydev virtio_balloon soundcore i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_blk drm ata_piix 8139too libata crct10dif_pclmul crct10dif_common
8139cp serio_raw virtio_pci crc32c_intel virtio_ring mii virtio floppy drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 10869 Comm: pritunl Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff97500329e300 ti: ffff975104540000 task.ti: ffff975104540000
RIP: 0010:[<ffffffff86b16562>]  [<ffffffff86b16562>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff975104543b30  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff975104543b00 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffff975104543b78 R08: ffffffff87214400 R09: fffff9b48a63bd40
R10: 0000000000003ac5 R11: ffff975103021a28 R12: 0000000000000002
R13: ffff9752bffd9008 R14: 0000000000000000 R15: 0000000386a7e868
FS:  00007f873adeb700(0000) GS:ffff9752bfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000005aa65002 CR3: 0000000219df8000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff86a7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff86a7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff86a7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff86b1667f>] smp_call_function_single+0x5f/0xa0
[<ffffffff86d85de5>] ? cpumask_next_and+0x35/0x50
[<ffffffff86b16c2b>] smp_call_function_many+0x22b/0x270
[<ffffffff86a7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff86a7ea04>] flush_tlb_page+0x54/0xa0
[<ffffffff86c07728>] ptep_clear_flush+0x68/0xa0
[<ffffffff86bef295>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff86bf16bb>] do_wp_page+0xfb/0x720
[<ffffffff86bf5d32>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8718e653>] __do_page_fault+0x213/0x500
[<ffffffff8718ea26>] trace_do_page_fault+0x56/0x150
[<ffffffff8718dfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8718a7a8>] async_page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 13 75 87 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1388378
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa4316802
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa427e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa427e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa427e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa431691f
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa4585fe5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa4316ecb
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa427e868
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffa427ea04
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffa4407908
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffa43ef475
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffa43f189b
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xfb
13
handle_mm_fault
vmlinux 0xffffffffa43f5f12
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb22
14
? async_page_fault
vmlinux 0xffffffffa498a7a8
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x28
15
__do_page_fault
vmlinux 0xffffffffa498e653
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x213
16
trace_do_page_fault
vmlinux 0xffffffffa498ea26
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x56
17
do_async_page_fault
vmlinux 0xffffffffa498dfa2
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x22
18
async_page_fault
vmlinux 0xffffffffa498a7a8
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [f2b/observer:1693]
Modules linked in: xt_multiport xt_set ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ip_set_hash_ip ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod aesni_intel lrw sg virtio_balloon i2c_piix4 gf128mul glue_helper ablk_helper cryptd pcspkr joydev sch_fq_codel br_netfilter bridge stp llc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic virtio_scsi pata_acpi virtio_net net_failover failover
virtio_console bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crc32c_intel serio_raw virtio_pci virtio_ring drm_panel_orientation_quirks virtio floppy fuse
CPU: 15 PID: 1693 Comm: f2b/observer Kdump: loaded Not tainted 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
task: ffff9d3ef53a8000 ti: ffff9d3efbbc8000 task.ti: ffff9d3efbbc8000
RIP: 0010:[<ffffffffa4316802>]  [<ffffffffa4316802>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9d3efbbcbb30  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffff9d3efbbcbb00 RCX: 0000000000000030
RDX: 000000000000ffff RSI: 0000000000000010 RDI: 0000000000000286
RBP: ffff9d3efbbcbb78 R08: ffffffffa4a137c0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
R13: ffff9d6549f42008 R14: 0000000000000000 R15: 00000003a427e868
FS:  00007f91fb210700(0000) GS:ffff9d64a91c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f91fb9162b0 CR3: 00000001f87de000 CR4: 00000000000006e0
Call Trace:
[<ffffffffa427e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa427e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa427e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa431691f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa4585fe5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa4316ecb>] smp_call_function_many+0x22b/0x270
[<ffffffffa427e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa427ea04>] flush_tlb_page+0x54/0xa0
[<ffffffffa4407908>] ptep_clear_flush+0x68/0xa0
[<ffffffffa43ef475>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffa43f189b>] do_wp_page+0xfb/0x720
[<ffffffffa43f5f12>] handle_mm_fault+0xb22/0xfb0
[<ffffffffa498a7a8>] ? async_page_fault+0x28/0x30
[<ffffffffa498e653>] __do_page_fault+0x213/0x500
[<ffffffffa498ea26>] trace_do_page_fault+0x56/0x150
[<ffffffffa498dfa2>] do_async_page_fault+0x22/0xf0
[<ffffffffa498a7a8>] async_page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 13 f5 a4 48 89 df e8 57 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1357337
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff84f16b62
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff84e7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff84e7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff84e7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff84f16c7f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff85186865
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff84f1722b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff84e7e258
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff84e7e3f4
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff85008128
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff84fefc55
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff84ff207b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff84fbe49b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff84feee99
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffff84ff66f2
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xb22
16
? switch_mm_irqs_off
vmlinux 0xffffffff84e7dfe9
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x109
17
? switch_mm_irqs_off
vmlinux 0xffffffff84e7dff5
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x115
18
? switch_mm_irqs_off
vmlinux 0xffffffff84e7dfe9
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x109
19
? switch_mm_irqs_off
vmlinux 0xffffffff84e7dff5
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x115
20
__do_page_fault
vmlinux 0xffffffff85594653
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x213
21
do_page_fault
vmlinux 0xffffffff85594975
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x35
22
page_fault
vmlinux 0xffffffff85590778
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#21 stuck for 22s! [setroubleshootd:60616]
Modules linked in: tcp_lp tun devlink bridge stp llc ip_set vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev vmw_balloon iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi aesni_intel snd_ac97_codec lrw gf128mul ac97_bus glue_helper ablk_helper snd_seq cryptd snd_seq_device snd_pcm joydev pcspkr snd_timer sg snd vmw_vmci soundcore i2c_piix4 parport_pc parport xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul drm crct10dif_common nfit mptspi crc32c_intel scsi_transport_spi libata mptscsih libnvdimm e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
[last unloaded: ip_tables]
CPU: 21 PID: 60616 Comm: setroubleshootd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9cbaadc09080 ti: ffff9cb82d3b8000 task.ti: ffff9cb82d3b8000
RIP: 0010:[<ffffffff84f16b62>]  [<ffffffff84f16b62>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9cb82d3bbb60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9cb82d3bbb30 RCX: 0000000000000011
RDX: ffffffff85615fd0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9cb82d3bbba8 R08: ffffffff85615fc8 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000001f
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007ff0999c5740(0000) GS:ffff9cbafdb40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff099916398 CR3: 0000001f7fa68000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff84e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff84e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff84e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff84f16c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffff85186865>] ? cpumask_next_and+0x35/0x50
[<ffffffff84f1722b>] smp_call_function_many+0x22b/0x270
[<ffffffff84e7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff84e7e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffff85008128>] ptep_clear_flush+0x68/0xa0
[<ffffffff84fefc55>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff84ff207b>] do_wp_page+0xfb/0x720
[<ffffffff84fbe49b>] ? unlock_page+0x2b/0x30
[<ffffffff84feee99>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff84ff66f2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff84e7dfe9>] ? switch_mm_irqs_off+0x109/0x290
[<ffffffff84e7dff5>] ? switch_mm_irqs_off+0x115/0x290
[<ffffffff84e7dfe9>] ? switch_mm_irqs_off+0x109/0x290
[<ffffffff84e7dff5>] ? switch_mm_irqs_off+0x115/0x290
[<ffffffff85594653>] __do_page_fault+0x213/0x500
[<ffffffff85594975>] do_page_fault+0x35/0x90
[<ffffffff85590778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 60 18 b5 85 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1194682
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8bd14a22
Build id: 3.10.0-1062.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8bc7d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff8bc7d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff8bc7d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff8bd14b3f
Build id: 3.10.0-1062.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8bf7f475
Build id: 3.10.0-1062.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8bd150eb
Build id: 3.10.0-1062.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8bc7d398
Build id: 3.10.0-1062.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8bc7d534
Build id: 3.10.0-1062.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff8be03b18
Build id: 3.10.0-1062.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff8bdeb6f5
Build id: 3.10.0-1062.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff8bdedb1b
Build id: 3.10.0-1062.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff8bdbbffb
Build id: 3.10.0-1062.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff8bdea939
Build id: 3.10.0-1062.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff8bdef4a5
Build id: 3.10.0-1062.el7.x86_64 		0x3d5
16
handle_mm_fault
vmlinux 0xffffffff8bdf200d
Build id: 3.10.0-1062.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffff8c386633
Build id: 3.10.0-1062.el7.x86_64 		0x213
18
do_page_fault
vmlinux 0xffffffff8c386955
Build id: 3.10.0-1062.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffff8c382768
Build id: 3.10.0-1062.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [containerd-shim:7606]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_statistic veth nf_tables vxlan ip6_udp_tunnel udp_tunnel xt_physdev xt_nat ipt_REJECT nf_reject_ipv4 xt_comment xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod nfit libnvdimm ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper pcspkr joydev cryptd sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy
CPU: 1 PID: 7606 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9cfbab758000 ti: ffff9cfba8d10000 task.ti: ffff9cfba8d10000
RIP: 0010:[<ffffffff8bd14a22>]  [<ffffffff8bd14a22>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9cfba8d13ad0  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffff9cfba8d13aa0 RCX: 0000000000000030
RDX: 000000000000ffff RSI: 0000000000000010 RDI: 0000000000000286
RBP: ffff9cfba8d13b20 R08: ffffffff8c413500 R09: fffff933045f7280
R10: 0000000000031e4e R11: 00003ffffffff000 R12: 000000000000033b
R13: 0000000000000002 R14: ffff9cf9fffdb008 R15: 0000000000000000
FS:  000000c0000f1210(0000) GS:ffff9cf9f7c80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1748618650 CR3: 00000001dbe2c000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8bc7d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8bc7d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8bc7d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8bd14b3f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8bf7f475>] ? cpumask_next_and+0x35/0x50
[<ffffffff8bd150eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8bc7d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8bc7d534>] flush_tlb_page+0x54/0xa0
[<ffffffff8be03b18>] ptep_clear_flush+0x68/0xa0
[<ffffffff8bdeb6f5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff8bdedb1b>] do_wp_page+0xfb/0x720
[<ffffffff8bdbbffb>] ? unlock_page+0x2b/0x30
[<ffffffff8bdea939>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff8bdef4a5>] handle_pte_fault+0x3d5/0xe20
[<ffffffff8bdf200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8c386633>] __do_page_fault+0x213/0x500
[<ffffffff8c386955>] do_page_fault+0x35/0x90
[<ffffffff8c382768>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 d7 94 8c 48 89 df e8 d7 1a 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1091017
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9f1167a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x100
2
? leave_mm
vmlinux 0xffffffff9f07e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff9f07e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff9f07e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff9f1168bf
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9f3862c5
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9f116e6b
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9f07e868
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9f07ea04
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff9f207908
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff9f1ef415
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff9f1f183b
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff9f1be15b
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff9f1ee659
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffff9f1f5eb2
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xb22
16
? __check_object_size
vmlinux 0xffffffff9f24a8da
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x1ca
17
__do_page_fault
vmlinux 0xffffffff9f78f653
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x213
18
do_page_fault
vmlinux 0xffffffff9f78f975
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffff9f78b778
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [abrt-action-sav:94204]
Modules linked in: binfmt_misc resguard_linux(OE) secmodel_linux(OE) syshook_linux(OE) secmodel3(OE) xt_set ip_set_hash_net xt_NFLOG nfnetlink_log ip_set xt_CHECKSUM iptable_raw ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock nf_conntrack_netlink nfnetlink nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev joydev vmw_balloon pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel mptspi libata scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 94204 Comm: abrt-action-sav Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8d2cc0ec1080 ti: ffff8d27855ac000 task.ti: ffff8d27855ac000
RIP: 0010:[<ffffffff9f1167a0>]  [<ffffffff9f1167a0>] generic_exec_single+0x100/0x1c0
RSP: 0000:ffff8d27855afb60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8d27855afb30 RCX: 0000000000000001
RDX: ffffffff9f8137d0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8d27855afba8 R08: ffffffff9f8137c8 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
R13: ffff8d34bff9f008 R14: 0000000000000000 R15: 000000039f3862c5
FS:  00007f499a4f5740(0000) GS:ffff8d347d6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4999ebae88 CR3: 00000001e565a000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9f07e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9f07e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9f07e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9f1168bf>] smp_call_function_single+0x5f/0xa0
[<ffffffff9f3862c5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9f116e6b>] smp_call_function_many+0x22b/0x270
[<ffffffff9f07e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9f07ea04>] flush_tlb_page+0x54/0xa0
[<ffffffff9f207908>] ptep_clear_flush+0x68/0xa0
[<ffffffff9f1ef415>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff9f1f183b>] do_wp_page+0xfb/0x720
[<ffffffff9f1be15b>] ? unlock_page+0x2b/0x30
[<ffffffff9f1ee659>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff9f1f5eb2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff9f24a8da>] ? __check_object_size+0x1ca/0x250
[<ffffffff9f78f653>] __do_page_fault+0x213/0x500
[<ffffffff9f78f975>] do_page_fault+0x35/0x90
[<ffffffff9f78b778>] page_fault+0x28/0x30
Code: 00 48 89 de 48 03 14 c5 e0 13 d5 9f 48 89 df e8 97 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 <f3> 90 f6 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28
Complete report #1069419
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb310aa2e
Build id: 3.10.0-862.14.4.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffffb3079c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffb3079c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffb3079c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffb310ab3f
Build id: 3.10.0-862.14.4.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb334edb5
Build id: 3.10.0-862.14.4.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb310b0eb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb3079e58
Build id: 3.10.0-862.14.4.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffb3079ff4
Build id: 3.10.0-862.14.4.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffb31dc0f7
Build id: 3.10.0-862.14.4.el7.x86_64 		0x67
11
wp_page_copy.isra.70
vmlinux 0xffffffffb31c49cb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffffb31c6c3b
Build id: 3.10.0-862.14.4.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffb31970cb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x2b
14
? do_read_fault.isra.60
vmlinux 0xffffffffb31c3c89
Build id: 3.10.0-862.14.4.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffffb31c85cd
Build id: 3.10.0-862.14.4.el7.x86_64 		0x37d
16
handle_mm_fault
vmlinux 0xffffffffb31caefd
Build id: 3.10.0-862.14.4.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffffb3720547
Build id: 3.10.0-862.14.4.el7.x86_64 		0x197
18
do_page_fault
vmlinux 0xffffffffb37208d5
Build id: 3.10.0-862.14.4.el7.x86_64 		0x35
19
? error_swapgs
vmlinux 0xffffffffb371ca96
Build id: 3.10.0-862.14.4.el7.x86_64 		0xa7
20
page_fault
vmlinux 0xffffffffb371c758
Build id: 3.10.0-862.14.4.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [httpd:6142]
Modules linked in: binfmt_misc xt_time xt_conntrack kcare(OE) ebtable_filter ebtables devlink ipt_REJECT nf_reject_ipv4 xt_REDIRECT nf_nat_redirect iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack tcp_diag xt_multiport inet_diag iptable_filter ip6table_filter xt_owner ip6_tables loop vmw_vsock_vmci_transport vsock xfs libcrc32c ppdev vmw_balloon edac_mce_amd pcspkr joydev i2c_piix4 vmw_vmci sg shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod sr_mod cdrom crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih libata
mptbase serio_raw vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 13 PID: 6142 Comm: httpd Kdump: loaded Tainted: G           OEL ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff95e394a55ee0 ti: ffff95e3f2814000 task.ti: ffff95e3f2814000
RIP: 0010:[<ffffffffb310aa2e>]  [<ffffffffb310aa2e>] generic_exec_single+0xfe/0x1b0
RSP: 0000:ffff95e3f2817af0  EFLAGS: 00000202
RAX: 0000000000000020 RBX: 0000000000000000 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000286
RBP: ffff95e3f2817b38 R08: ffffffffb38132a0 R09: ffffeb2110c21700
R10: 00000000000046c8 R11: 00003ffffffff000 R12: 73656761705f6873
R13: 696c627570223a33 R14: 0000000000000002 R15: ffff95e87ffcf008
FS:  00002ace889e7440(0000) GS:ffff95e86b140000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000056378c9127b8 CR3: 000000012e96a000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb3079c90>] ? leave_mm+0x110/0x110
[<ffffffffb3079c90>] ? leave_mm+0x110/0x110
[<ffffffffb3079c90>] ? leave_mm+0x110/0x110
[<ffffffffb310ab3f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb334edb5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb310b0eb>] smp_call_function_many+0x22b/0x270
[<ffffffffb3079e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb3079ff4>] flush_tlb_page+0x54/0xa0
[<ffffffffb31dc0f7>] ptep_clear_flush+0x67/0xa0
[<ffffffffb31c49cb>] wp_page_copy.isra.70+0x30b/0x560
[<ffffffffb31c6c3b>] do_wp_page+0xfb/0x720
[<ffffffffb31970cb>] ? unlock_page+0x2b/0x30
[<ffffffffb31c3c89>] ? do_read_fault.isra.60+0x139/0x1b0
[<ffffffffb31c85cd>] handle_pte_fault+0x37d/0xd10
[<ffffffffb31caefd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb3720547>] __do_page_fault+0x197/0x4f0
[<ffffffffb37208d5>] do_page_fault+0x35/0x90
[<ffffffffb371ca96>] ? error_swapgs+0xa7/0xbd
[<ffffffffb371c758>] page_fault+0x28/0x30
Code: 48 89 de 48 03 14 c5 a0 54 d3 b3 48 89 df e8 6a a1 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87
Complete report #941206
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff86f163c0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x100
2
? leave_mm
vmlinux 0xffffffff86e7e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff86e7e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff86e7e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff86f164df
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff87181635
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff86f16a8b
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff86e7e478
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff86e7e614
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff870059d8
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff86fed5b5
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff86fef9db
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff86fbd99b
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff86fec7f9
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff86ff1365
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x3d5
16
handle_mm_fault
vmlinux 0xffffffff86ff3ecd
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffff87588653
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x213
18
trace_do_page_fault
vmlinux 0xffffffff87588a26
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x56
19
do_async_page_fault
vmlinux 0xffffffff87587fa2
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22
20
async_page_fault
vmlinux 0xffffffff875847a8
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [rsync:28510]
Modules linked in: binfmt_misc tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag ilocker(OE) xt_set xt_multiport ip_set_hash_ip ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ppdev sg joydev pcspkr virtio_balloon i2c_piix4 parport_pc parport ip_tables xfs libcrc32c virtio_console virtio_net virtio_blk virtio_scsi sr_mod cdrom ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci serio_raw virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 28510 Comm: rsync Kdump: loaded Tainted: G           OEL ------------   3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff98deae71e2a0 ti: ffff98df074d0000 task.ti: ffff98df074d0000
RIP: 0010:[<ffffffff86f163c0>]  [<ffffffff86f163c0>] generic_exec_single+0x100/0x1c0
RSP: 0000:ffff98df074d3aa0  EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff98df074d3a70 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000286
RBP: ffff98df074d3af0 R08: ffffffff876137c0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff98df074d3db8
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007f5c7091e740(0000) GS:ffff98df3aa00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558d62c601a8 CR3: 000000013761a000 CR4: 00000000000006f0
Call Trace:
[<ffffffff86e7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff86e7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff86e7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff86f164df>] smp_call_function_single+0x5f/0xa0
[<ffffffff87181635>] ? cpumask_next_and+0x35/0x50
[<ffffffff86f16a8b>] smp_call_function_many+0x22b/0x270
[<ffffffff86e7e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff86e7e614>] flush_tlb_page+0x54/0xa0
[<ffffffff870059d8>] ptep_clear_flush+0x68/0xa0
[<ffffffff86fed5b5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff86fef9db>] do_wp_page+0xfb/0x720
[<ffffffff86fbd99b>] ? unlock_page+0x2b/0x30
[<ffffffff86fec7f9>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff86ff1365>] handle_pte_fault+0x3d5/0xe20
[<ffffffff86ff3ecd>] handle_mm_fault+0x39d/0x9b0
[<ffffffff87588653>] __do_page_fault+0x213/0x500
[<ffffffff87588a26>] trace_do_page_fault+0x56/0x150
[<ffffffff87587fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff875847a8>] async_page_fault+0x28/0x30
Code: 00 48 89 de 48 03 14 c5 20 e8 b4 87 48 89 df e8 f7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 <f3> 90 f6 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28
Complete report #770673
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8b914a22
Build id: 3.10.0-1062.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8b87d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff8b87d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff8b87d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff8b914b3f
Build id: 3.10.0-1062.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8bb7f475
Build id: 3.10.0-1062.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8b9150eb
Build id: 3.10.0-1062.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8b87d398
Build id: 3.10.0-1062.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8b87d534
Build id: 3.10.0-1062.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff8ba03b18
Build id: 3.10.0-1062.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff8b9eb6f5
Build id: 3.10.0-1062.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff8b9edb1b
Build id: 3.10.0-1062.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff8b9bbffb
Build id: 3.10.0-1062.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff8b9ea939
Build id: 3.10.0-1062.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff8b9ef4a5
Build id: 3.10.0-1062.el7.x86_64 		0x3d5
16
? path_openat
vmlinux 0xffffffff8ba59d06
Build id: 3.10.0-1062.el7.x86_64 		0x176
17
handle_mm_fault
vmlinux 0xffffffff8b9f200d
Build id: 3.10.0-1062.el7.x86_64 		0x39d
18
__do_page_fault
vmlinux 0xffffffff8bf86633
Build id: 3.10.0-1062.el7.x86_64 		0x213
19
do_page_fault
vmlinux 0xffffffff8bf86955
Build id: 3.10.0-1062.el7.x86_64 		0x35
20
page_fault
vmlinux 0xffffffff8bf82768
Build id: 3.10.0-1062.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 24s! [pgrep:19751]
Modules linked in: binfmt_misc nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq ppdev snd_seq_device snd_pcm vmw_balloon snd_timer snd iosf_mbi crc32_pclmul ghash_clmulni_intel sg soundcore aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit libnvdimm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel mptspi libata scsi_transport_spi e1000 mptscsih serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 19751 Comm: pgrep Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff93db0d7cb150 ti: ffff93d9d0a34000 task.ti: ffff93d9d0a34000
RIP: 0010:[<ffffffff8b914a22>]  [<ffffffff8b914a22>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff93d9d0a37ad0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff93d9d0a37aa0 RCX: 000000000000000a
RDX: ffffffff8c014b90 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff93d9d0a37b20 R08: ffffffff8c014b88 R09: fffffd104de52b00
R10: 0000000000004ed9 R11: 00003ffffffff000 R12: 0000000000000001
R13: 0000000000000002 R14: ffff93dc7ffdb008 R15: 0000000000000000
FS:  00007fa207e1f880(0000) GS:ffff93dc3d8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa206ae87c0 CR3: 0000000dcc7e8000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff8b87d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8b87d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8b87d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8b914b3f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8bb7f475>] ? cpumask_next_and+0x35/0x50
[<ffffffff8b9150eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8b87d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8b87d534>] flush_tlb_page+0x54/0xa0
[<ffffffff8ba03b18>] ptep_clear_flush+0x68/0xa0
[<ffffffff8b9eb6f5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff8b9edb1b>] do_wp_page+0xfb/0x720
[<ffffffff8b9bbffb>] ? unlock_page+0x2b/0x30
[<ffffffff8b9ea939>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff8b9ef4a5>] handle_pte_fault+0x3d5/0xe20
[<ffffffff8ba59d06>] ? path_openat+0x176/0x5a0
[<ffffffff8b9f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8bf86633>] __do_page_fault+0x213/0x500
[<ffffffff8bf86955>] do_page_fault+0x35/0x90
[<ffffffff8bf82768>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 d7 54 8c 48 89 df e8 d7 1a 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #704928
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9f51109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff9f47a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9f47a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9f47a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff9f5111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9f775465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9f51175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9f47a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9f47a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff9f5fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffff9f5e55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffff9f5e782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff9f5b64fb
Build id: 3.10.0-957.el7.x86_64 		0x2b
14
? do_read_fault.isra.61
vmlinux 0xffffffff9f5e4879
Build id: 3.10.0-957.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff9f5e91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
16
handle_mm_fault
vmlinux 0xffffffff9f5ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffff9fb6f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
18
do_page_fault
vmlinux 0xffffffff9fb6f915
Build id: 3.10.0-957.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffff9fb6b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [abrt-action-ure:53794]
Modules linked in: veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) overlay(T) sunrpc snd_seq_midi snd_seq_midi_event sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm joydev snd_timer snd pcspkr sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic
pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ata_piix mptspi libata scsi_transport_spi e1000 nfit mptscsih libnvdimm mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 53794 Comm: abrt-action-ure Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8b76b12f4100 ti: ffff8b72c3758000 task.ti: ffff8b72c3758000
RIP: 0010:[<ffffffff9f51109a>]  [<ffffffff9f51109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff8b72c375bae0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8b72c375bab0 RCX: 0000000000000008
RDX: ffffffff9fc13f10 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8b72c375bb30 R08: ffffffff9fc13f08 R09: 525f545300000001
R10: 00454d4954414c45 R11: 0000000000000000 R12: ffff8b76e93ce180
R13: 0000000000000002 R14: ffff8b76fffdb008 R15: 0000000000000000
FS:  00007f38681dd740(0000) GS:ffff8b76ed600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3868196540 CR3: 00000000037ea000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9f47a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9f47a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9f47a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9f5111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9f775465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9f51175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9f47a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9f47a734>] flush_tlb_page+0x54/0xa0
[<ffffffff9f5fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff9f5e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffff9f5e782b>] do_wp_page+0xfb/0x720
[<ffffffff9f5b64fb>] ? unlock_page+0x2b/0x30
[<ffffffff9f5e4879>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffff9f5e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffff9f5ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9fb6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9fb6f915>] do_page_fault+0x35/0x90
[<ffffffff9fb6b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 a0 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #700533
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9551109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff9547a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9547a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9547a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff955111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff95775465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9551175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9547a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9547a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff955fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffff955e55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffff955e782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff955b64fb
Build id: 3.10.0-957.el7.x86_64 		0x2b
14
? do_read_fault.isra.61
vmlinux 0xffffffff955e4879
Build id: 3.10.0-957.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff955e91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
16
? __switch_to
vmlinux 0xffffffff9542a621
Build id: 3.10.0-957.el7.x86_64 		0x151
17
handle_mm_fault
vmlinux 0xffffffff955ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
18
__do_page_fault
vmlinux 0xffffffff95b6f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
19
do_page_fault
vmlinux 0xffffffff95b6f915
Build id: 3.10.0-957.el7.x86_64 		0x35
20
page_fault
vmlinux 0xffffffff95b6b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [python:78657]
Modules linked in: vmw_vsock_vmci_transport vsock vfat fat sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon joydev sg pcspkr parport_pc parport tpm_crb vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw e1000e ahci libahci ata_piix drm mptsas scsi_transport_sas mptscsih libata ptp pps_core nfit mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 78657 Comm: python Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.17369862.B64.2012240522 12/24/2020
task: ffff97ef99a29040 ti: ffff97e9943f4000 task.ti: ffff97e9943f4000
RIP: 0010:[<ffffffff9551109a>]  [<ffffffff9551109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff97e9943f7ae0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff97e9943f7ab0 RCX: 000000000000000f
RDX: ffffffff95c15090 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff97e9943f7b30 R08: ffffffff95c15088 R09: fffff8ec0a9e4400
R10: 0000000000004b29 R11: 00003ffffffff000 R12: 000000081a10f025
R13: 0000000000000002 R14: ffff97efbffbf008 R15: 0000000000000000
FS:  00007f308bc50740(0000) GS:ffff97ef9d700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f308ba305b0 CR3: 0000000110a76000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9547a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9547a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9547a3d0>] ? leave_mm+0x110/0x110
[<ffffffff955111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff95775465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9551175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9547a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9547a734>] flush_tlb_page+0x54/0xa0
[<ffffffff955fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff955e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffff955e782b>] do_wp_page+0xfb/0x720
[<ffffffff955b64fb>] ? unlock_page+0x2b/0x30
[<ffffffff955e4879>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffff955e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffff9542a621>] ? __switch_to+0x151/0x580
[<ffffffff955ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff95b6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff95b6f915>] do_page_fault+0x35/0x90
[<ffffffff95b6b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 96 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #696548
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff81b16612
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff81a7e440
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff81a7e440
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff81a7e440
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff81b1672f
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff81d81845
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff81b16cdb
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff81a7e608
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff81a7e7a4
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff81c05638
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff81bed215
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff81bef63b
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xfb
13
handle_mm_fault
vmlinux 0xffffffff81bf3cd2
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xb22
14
__do_page_fault
vmlinux 0xffffffff82188653
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x213
15
trace_do_page_fault
vmlinux 0xffffffff82188a26
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x56
16
do_async_page_fault
vmlinux 0xffffffff82187fa2
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x22
17
async_page_fault
vmlinux 0xffffffff821847a8
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ofork.Daemon.pl:6155]
Modules linked in: nfsv3 nfs_acl nfs lockd grace fscache sunrpc ext4 dm_mirror dm_region_hash dm_log dm_mod mbcache jbd2 snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device iosf_mbi ppdev snd_pcm crc32_pclmul ghash_clmulni_intel aesni_intel snd_timer lrw gf128mul snd glue_helper ablk_helper cryptd sg virtio_rng soundcore pcspkr i2c_piix4 joydev virtio_balloon parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console qxl virtio_net virtio_blk drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata virtio_pci virtio_ring virtio floppy drm_panel_orientation_quirks
CPU: 0 PID: 6155 Comm: ofork.Daemon.pl Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9d8d3ce841c0 ti: ffff9d8bc4f80000 task.ti: ffff9d8bc4f80000
RIP: 0010:[<ffffffff81b16612>]  [<ffffffff81b16612>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9d8bc4f83b30  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff9d8bc4f83b30 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000286
RBP: ffff9d8bc4f83b78 R08: ffffffff822137c0 R09: ffffdd0c812b28c0
R10: 0000000000003149 R11: ffff9d8bc4e67510 R12: 0000000000000001
R13: 0000000000000001 R14: ffff9d8bc4f83c00 R15: 0000000000000001
FS:  00007f7de23f0740(0000) GS:ffff9d8dffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000091212000 CR3: 0000000114060000 CR4: 00000000003606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff81a7e440>] ? leave_mm+0x110/0x110
[<ffffffff81a7e440>] ? leave_mm+0x110/0x110
[<ffffffff81a7e440>] ? leave_mm+0x110/0x110
[<ffffffff81b1672f>] smp_call_function_single+0x5f/0xa0
[<ffffffff81d81845>] ? cpumask_next_and+0x35/0x50
[<ffffffff81b16cdb>] smp_call_function_many+0x22b/0x270
[<ffffffff81a7e608>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81a7e7a4>] flush_tlb_page+0x54/0xa0
[<ffffffff81c05638>] ptep_clear_flush+0x68/0xa0
[<ffffffff81bed215>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff81bef63b>] do_wp_page+0xfb/0x720
[<ffffffff81bf3cd2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff82188653>] __do_page_fault+0x213/0x500
[<ffffffff82188a26>] trace_do_page_fault+0x56/0x150
[<ffffffff82187fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff821847a8>] async_page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 60 e8 74 82 48 89 df e8 b7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #690621
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9c71109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff9c67a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9c67a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9c67a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff9c7111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9c975465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9c71175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9c67a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9c67a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff9c7fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffff9c7e55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffff9c7e782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff9c7b64fb
Build id: 3.10.0-957.el7.x86_64 		0x2b
14
? do_read_fault.isra.61
vmlinux 0xffffffff9c7e4879
Build id: 3.10.0-957.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff9c7e91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
16
? filename_lookup
vmlinux 0xffffffff9c851d2c
Build id: 3.10.0-957.el7.x86_64 		0x7c
17
? user_path_at_empty
vmlinux 0xffffffff9c853cd2
Build id: 3.10.0-957.el7.x86_64 		0x72
18
handle_mm_fault
vmlinux 0xffffffff9c7ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
19
__do_page_fault
vmlinux 0xffffffff9cd6f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
20
trace_do_page_fault
vmlinux 0xffffffff9cd6f9c6
Build id: 3.10.0-957.el7.x86_64 		0x56
21
do_async_page_fault
vmlinux 0xffffffff9cd6ef42
Build id: 3.10.0-957.el7.x86_64 		0x22
22
async_page_fault
vmlinux 0xffffffff9cd6b788
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [oracle:59516]
Modules linked in: netfilter_edr(OE) tun devlink ip_set nfnetlink bridge stp llc sunrpc ppdev pcspkr joydev virtio_balloon parport_pc parport sg i2c_piix4 binfmt_misc xfs libcrc32c sr_mod cdrom virtio_blk virtio_console virtio_scsi virtio_net ata_generic pata_acpi serio_raw floppy cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix drm_panel_orientation_quirks libata virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 2 PID: 59516 Comm: oracle Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff92510a480000 ti: ffff9250ec0c4000 task.ti: ffff9250ec0c4000
RIP: 0010:[<ffffffff9c71109a>]  [<ffffffff9c71109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff9250ec0c7ab0  EFLAGS: 00000202
RAX: 0000000000000060 RBX: ffff9250ec0c7a80 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 0000000000000060 RDI: 0000000000000282
RBP: ffff9250ec0c7b00 R08: ffffffff9ce13008 R09: ffffe3b113e0c380
R10: 0000000000002046 R11: 00003ffffffff000 R12: ffff9253590f2a80
R13: 0000000000000002 R14: ffff9258fffbf008 R15: 0000000000000000
FS:  00007fd401797f40(0000) GS:ffff9258dde80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000015771600 CR3: 0000000053048000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9c67a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9c67a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9c67a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9c7111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9c975465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9c71175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9c67a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9c67a734>] flush_tlb_page+0x54/0xa0
[<ffffffff9c7fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff9c7e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffff9c7e782b>] do_wp_page+0xfb/0x720
[<ffffffff9c7b64fb>] ? unlock_page+0x2b/0x30
[<ffffffff9c7e4879>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffff9c7e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffff9c851d2c>] ? filename_lookup+0x7c/0xc0
[<ffffffff9c853cd2>] ? user_path_at_empty+0x72/0xc0
[<ffffffff9c7ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9cd6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9cd6f9c6>] trace_do_page_fault+0x56/0x150
[<ffffffff9cd6ef42>] do_async_page_fault+0x22/0xf0
[<ffffffff9cd6b788>] async_page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 34 9d 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #592767
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffabb16ae2
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffaba7ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffaba7ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffaba7ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffabb16bff
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffabd86845
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffabb171ab
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaba7e258
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffaba7e3f4
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffabc08108
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffabbefc35
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffabbf205b
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffabbbe47b
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffffabbeee79
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffffabbf66d2
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xb22
16
? do_sync_read
vmlinux 0xffffffffabc4d943
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x93
17
__do_page_fault
vmlinux 0xffffffffac194653
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x213
18
do_page_fault
vmlinux 0xffffffffac194975
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x35
19
? error_swapgs
vmlinux 0xffffffffac190ac9
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xaa
20
page_fault
vmlinux 0xffffffffac190778
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [cat:64775]
Modules linked in: tcp_lp veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter binfmt_misc overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac snd_ens1371 coretemp snd_rawmidi snd_ac97_codec ac97_bus snd_seq iosf_mbi crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel snd_timer
lrw gf128mul snd glue_helper ablk_helper cryptd ppdev soundcore sg joydev vmw_balloon pcspkr vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata crct10dif_pclmul crct10dif_common drm crc32c_intel e1000 nfit mptspi serio_raw scsi_transport_spi libnvdimm mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 64775 Comm: cat Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff95bf1f3d2100 ti: ffff95bf60eec000 task.ti: ffff95bf60eec000
RIP: 0010:[<ffffffffabb16ae2>]  [<ffffffffabb16ae2>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff95bf60eefb60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff95bf60eefb30 RCX: 0000000000000001
RDX: ffffffffac2137d0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff95bf60eefba8 R08: ffffffffac2137c8 R09: ffffe68e82892f80
R10: 00000000000038ed R11: ffff95be83be8288 R12: ffff95bf60eefbe8
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fa884fbad48(0000) GS:ffff95c168e40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa884fb9038 CR3: 0000000038df6000 CR4: 00000000000607e0
Call Trace:
[<ffffffffaba7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffaba7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffaba7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffabb16bff>] smp_call_function_single+0x5f/0xa0
[<ffffffffabd86845>] ? cpumask_next_and+0x35/0x50
[<ffffffffabb171ab>] smp_call_function_many+0x22b/0x270
[<ffffffffaba7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaba7e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffffabc08108>] ptep_clear_flush+0x68/0xa0
[<ffffffffabbefc35>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffabbf205b>] do_wp_page+0xfb/0x720
[<ffffffffabbbe47b>] ? unlock_page+0x2b/0x30
[<ffffffffabbeee79>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffabbf66d2>] handle_mm_fault+0xb22/0xfb0
[<ffffffffabc4d943>] ? do_sync_read+0x93/0xe0
[<ffffffffac194653>] __do_page_fault+0x213/0x500
[<ffffffffac194975>] do_page_fault+0x35/0x90
[<ffffffffac190ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffac190778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 00 19 75 ac 48 89 df e8 d7 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #590637
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa5314a26
Build id: 3.10.0-1062.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffa527d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa527d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa527d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa5314b3f
Build id: 3.10.0-1062.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa557f475
Build id: 3.10.0-1062.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa53150eb
Build id: 3.10.0-1062.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa527d398
Build id: 3.10.0-1062.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffa527d534
Build id: 3.10.0-1062.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffa5403b18
Build id: 3.10.0-1062.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffa53eb6f5
Build id: 3.10.0-1062.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffa53edb1b
Build id: 3.10.0-1062.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffa53bbffb
Build id: 3.10.0-1062.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffffa53ea939
Build id: 3.10.0-1062.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffffa53ef4a5
Build id: 3.10.0-1062.el7.x86_64 		0x3d5
16
? alloc_pages_current
vmlinux 0xffffffffa5414c68
Build id: 3.10.0-1062.el7.x86_64 		0x98
17
handle_mm_fault
vmlinux 0xffffffffa53f200d
Build id: 3.10.0-1062.el7.x86_64 		0x39d
18
__do_page_fault
vmlinux 0xffffffffa5986633
Build id: 3.10.0-1062.el7.x86_64 		0x213
19
do_page_fault
vmlinux 0xffffffffa5986955
Build id: 3.10.0-1062.el7.x86_64 		0x35
20
page_fault
vmlinux 0xffffffffa5982768
Build id: 3.10.0-1062.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ksmtuned:12182]
Modules linked in: nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 ppdev vmw_balloon snd_ens1371 sb_edac snd_rawmidi snd_ac97_codec iosf_mbi ac97_bus crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_timer snd joydev sg pcspkr soundcore i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit libnvdimm crct10dif_pclmul crct10dif_common mptspi crc32c_intel libata e1000 scsi_transport_spi mptscsih serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 12182 Comm: ksmtuned Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff9320e0380000 ti: ffff93211b388000 task.ti: ffff93211b388000
RIP: 0010:[<ffffffffa5314a26>]  [<ffffffffa5314a26>] generic_exec_single+0x106/0x1c0
RSP: 0000:ffff93211b38bad0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff93211b38baa0 RCX: 0000000000000008
RDX: ffffffffa5a14690 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff93211b38bb20 R08: ffffffffa5a14688 R09: 0000000001757380
R10: 000000000000a100 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000002 R14: ffff9321bffdb008 R15: 0000000000000000
FS:  00007f25f4e0b740(0000) GS:ffff93219d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000175ac00 CR3: 00000001933f8000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa527d1d0>] ? leave_mm+0x110/0x110
[<ffffffffa527d1d0>] ? leave_mm+0x110/0x110
[<ffffffffa527d1d0>] ? leave_mm+0x110/0x110
[<ffffffffa5314b3f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa557f475>] ? cpumask_next_and+0x35/0x50
[<ffffffffa53150eb>] smp_call_function_many+0x22b/0x270
[<ffffffffa527d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa527d534>] flush_tlb_page+0x54/0xa0
[<ffffffffa5403b18>] ptep_clear_flush+0x68/0xa0
[<ffffffffa53eb6f5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffa53edb1b>] do_wp_page+0xfb/0x720
[<ffffffffa53bbffb>] ? unlock_page+0x2b/0x30
[<ffffffffa53ea939>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffa53ef4a5>] handle_pte_fault+0x3d5/0xe20
[<ffffffffa5414c68>] ? alloc_pages_current+0x98/0x110
[<ffffffffa53f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa5986633>] __do_page_fault+0x213/0x500
[<ffffffffa5986955>] do_page_fault+0x35/0x90
[<ffffffffa5982768>] page_fault+0x28/0x30
Code: 14 c5 e0 d7 f4 a5 48 89 df e8 d7 1a 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #543447
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffacb16b62
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffaca7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffaca7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffaca7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffacb16c7f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffacd86865
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffacb1722b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaca7e258
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffaca7e3f4
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffacc08128
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffacbefc55
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffacbf207b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffacbbe49b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffffacbeee99
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffffacbf66f2
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xb22
16
? update_curr
vmlinux 0xffffffffacae495c
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x14c
17
__do_page_fault
vmlinux 0xffffffffad194653
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x213
18
do_page_fault
vmlinux 0xffffffffad194975
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x35
19
? error_swapgs
vmlinux 0xffffffffad190ac9
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xaa
20
page_fault
vmlinux 0xffffffffad190778
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [httpd:21795]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter tun rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache binfmt_misc overlay(T) nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr
sg ppdev vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm serio_raw ata_piix e1000 libata vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 21795 Comm: httpd Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 10/22/2013
task: ffff9c21b17a2100 ti: ffff9c1f07590000 task.ti: ffff9c1f07590000
RIP: 0010:[<ffffffffacb16b62>]  [<ffffffffacb16b62>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9c1f07593b60  EFLAGS: 00000202
RAX: 00000000000008fd RBX: 00000001000200da RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff9c1f07593ba8 R08: 0000000000000001 R09: b1618aa142871339
R10: e1cd059c28ab3492 R11: e40f5672e159a7d4 R12: ffffffffad48f2ef
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007f81bff1c880(0000) GS:ffff9c233fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f81bf886110 CR3: 000000037630a000 CR4: 00000000000007e0
Call Trace:
[<ffffffffaca7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffaca7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffaca7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffacb16c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffffacd86865>] ? cpumask_next_and+0x35/0x50
[<ffffffffacb1722b>] smp_call_function_many+0x22b/0x270
[<ffffffffaca7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaca7e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffffacc08128>] ptep_clear_flush+0x68/0xa0
[<ffffffffacbefc55>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffacbf207b>] do_wp_page+0xfb/0x720
[<ffffffffacbbe49b>] ? unlock_page+0x2b/0x30
[<ffffffffacbeee99>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffacbf66f2>] handle_mm_fault+0xb22/0xfb0
[<ffffffffacae495c>] ? update_curr+0x14c/0x1e0
[<ffffffffad194653>] __do_page_fault+0x213/0x500
[<ffffffffad194975>] do_page_fault+0x35/0x90
[<ffffffffad190ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffad190778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 60 18 75 ad 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #515212
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9ef16a32
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9ee7ddd0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff9ee7ddd0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff9ee7ddd0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff9ef16b4f
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9f186a55
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9ef170fb
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9ee7e258
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9ee7e3f4
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff9f008108
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff9efefc05
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff9eff202b
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xfb
13
? hrtimer_get_res
vmlinux 0xffffffff9eec9aa0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x50
14
handle_mm_fault
vmlinux 0xffffffff9eff66a2
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xb22
15
__do_page_fault
vmlinux 0xffffffff9f594653
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x213
16
trace_do_page_fault
vmlinux 0xffffffff9f594a26
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x56
17
do_async_page_fault
vmlinux 0xffffffff9f593fa2
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x22
18
async_page_fault
vmlinux 0xffffffff9f5907a8
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 23s! [jemalloc_bg_thd:1165]
Modules linked in: devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod sg joydev i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover virtio_console virtio_blk failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci serio_raw
virtio_ring virtio drm_panel_orientation_quirks floppy fuse
CPU: 11 PID: 1165 Comm: jemalloc_bg_thd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
task: ffff8f998a423180 ti: ffff8f998a42c000 task.ti: ffff8f998a42c000
RIP: 0010:[<ffffffff9ef16a32>]  [<ffffffff9ef16a32>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff8f998a42fb30  EFLAGS: 00000202
RAX: 000000000000000e RBX: ffff8f998a42fb00 RCX: 0000000000000032
RDX: 0000000000003fff RSI: 000000000000000e RDI: 0000000000000286
RBP: ffff8f998a42fb78 R08: ffffffff9f6146c0 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fbf80cb9700(0000) GS:ffff8f999f4c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbf80cb73f8 CR3: 000000081b678000 CR4: 00000000000006e0
Call Trace:
[<ffffffff9ee7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9ee7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9ee7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9ef16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9f186a55>] ? cpumask_next_and+0x35/0x50
[<ffffffff9ef170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff9ee7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9ee7e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffff9f008108>] ptep_clear_flush+0x68/0xa0
[<ffffffff9efefc05>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff9eff202b>] do_wp_page+0xfb/0x720
[<ffffffff9eec9aa0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff9eff66a2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff9f594653>] __do_page_fault+0x213/0x500
[<ffffffff9f594a26>] trace_do_page_fault+0x56/0x150
[<ffffffff9f593fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9f5907a8>] async_page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 60 18 b5 9f 48 89 df e8 97 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #452480
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb2d0779a
Build id: 3.10.0-862.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffb2c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffb2c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffb2c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffb2d078af
Build id: 3.10.0-862.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb2f4a275
Build id: 3.10.0-862.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb2d07e5b
Build id: 3.10.0-862.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb2c77018
Build id: 3.10.0-862.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffb2c771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffb2dd8837
Build id: 3.10.0-862.el7.x86_64 		0x67
11
wp_page_copy.isra.70
vmlinux 0xffffffffb2dc18be
Build id: 3.10.0-862.el7.x86_64 		0x29e
12
do_wp_page
vmlinux 0xffffffffb2dc3abb
Build id: 3.10.0-862.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffb2d93d3b
Build id: 3.10.0-862.el7.x86_64 		0x2b
14
? do_read_fault.isra.60
vmlinux 0xffffffffb2dc0c31
Build id: 3.10.0-862.el7.x86_64 		0x121
15
handle_pte_fault
vmlinux 0xffffffffb2dc536d
Build id: 3.10.0-862.el7.x86_64 		0x36d
16
handle_mm_fault
vmlinux 0xffffffffb2dc747d
Build id: 3.10.0-862.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffffb331a597
Build id: 3.10.0-862.el7.x86_64 		0x197
18
do_page_fault
vmlinux 0xffffffffb331a925
Build id: 3.10.0-862.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffffb3316768
Build id: 3.10.0-862.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [python2.7:253597]
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [bash:253658]
Modules linked in: iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_iprange ip6table_filter ip6_tables iptable_filter binfmt_misc joydev bochs_drm ttm sb_edac drm_kms_helper iosf_mbi ppdev virtio_rng sg syscopyarea kvm_intel kvm sysfillrect sysimgblt irqbypass fb_sys_fops crc32_pclmul ghash_clmulni_intel i2c_piix4 drm aesni_intel lrw parport_pc i2c_core gf128mul parport glue_helper ablk_helper cryptd pcspkr ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net virtio_console virtio_scsi crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata serio_raw virtio_pci virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 253658 Comm: bash Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9972a0a24f10 ti: ffff99736d704000 task.ti: ffff99736d704000
RIP: 0010:[<ffffffffb2d0779a>]  [<ffffffffb2d0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff99736d707af0  EFLAGS: 00000202
RAX: 00000000000000ff RBX: 0000000000000246 RCX: 0000000000000001
RDX: 00000000000000ff RSI: 000000000000003f RDI: 0000000000000286
RBP: ffff99736d707b38 R08: ffffffffb3413168 R09: fffffc19b35e1740
R10: fffffc19b35e1740 R11: 00003ffffffff000 R12: 0000000100000000
R13: ffff99736d707a80 R14: ffffffffb2da335e R15: 000200da00000000
FS:  0000000000000000(0000) GS:ffff99737f7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f052ac6ebd8 CR3: 0000000e463d0000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb2c76e50>] ? leave_mm+0x110/0x110
[<ffffffffb2c76e50>] ? leave_mm+0x110/0x110
[<ffffffffb2c76e50>] ? leave_mm+0x110/0x110
[<ffffffffb2d078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb2f4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffb2d07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffb2c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb2c771b4>] flush_tlb_page+0x54/0xa0
[<ffffffffb2dd8837>] ptep_clear_flush+0x67/0xa0
[<ffffffffb2dc18be>] wp_page_copy.isra.70+0x29e/0x4f0
[<ffffffffb2dc3abb>] do_wp_page+0xfb/0x710
[<ffffffffb2d93d3b>] ? unlock_page+0x2b/0x30
[<ffffffffb2dc0c31>] ? do_read_fault.isra.60+0x121/0x1a0
[<ffffffffb2dc536d>] handle_pte_fault+0x36d/0xc30
[<ffffffffb2dc747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb331a597>] __do_page_fault+0x197/0x4f0
[<ffffffffb331a925>] do_page_fault+0x35/0x90
[<ffffffffb3316768>] page_fault+0x28/0x30
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 93 b3 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #430825
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa2b16842
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa2a7e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa2a7e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa2a7e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa2b1695f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa2d86395
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa2b16f0b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa2a7e868
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffa2a7ea04
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffa2c079d8
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffa2bef4e5
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffa2bf190b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffa2bbe20b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffffa2bee729
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffffa2bf5f82
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xb22
16
__do_page_fault
vmlinux 0xffffffffa3190653
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x213
17
do_page_fault
vmlinux 0xffffffffa3190975
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x35
18
? error_swapgs
vmlinux 0xffffffffa318cac9
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xaa
19
page_fault
vmlinux 0xffffffffa318c778
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [manager-api:23003]
Modules linked in: tcp_lp nls_utf8 isofs veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vboxsf(OE) sunrpc snd_intel8x0 snd_ac97_codec ppdev ac97_bus hid_multitouch snd_seq snd_seq_device joydev snd_pcm vboxguest(OE) snd_timer sg snd pcspkr iosf_mbi soundcore parport_pc i2c_piix4 parport video ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea ahci sysfillrect sysimgblt fb_sys_fops libahci ata_piix
ttm serio_raw drm e1000 libata crc32c_intel drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 23003 Comm: manager-api Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff882535ced280 ti: ffff8824c0580000 task.ti: ffff8824c0580000
RIP: 0010:[<ffffffffa2b16842>]  [<ffffffffa2b16842>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff8824c0583b60  EFLAGS: 00010202
RAX: 00000000000008fb RBX: 00000000000000fb RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff8824c0583ba8 R08: 0000000000000001 R09: ffffd7bcc482fc00
R10: 0000000000002f10 R11: ffff882530401510 R12: 0000000000000000
R13: 00000000000008fb R14: 0000000000000001 R15: ffffd7bcc482fc00
FS:  0000000001a14a70(0063) GS:ffff8825a0000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fafc7fc99f0 CR3: 00000001b01da000 CR4: 00000000000206f0
Call Trace:
[<ffffffffa2a7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa2a7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa2a7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa2b1695f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa2d86395>] ? cpumask_next_and+0x35/0x50
[<ffffffffa2b16f0b>] smp_call_function_many+0x22b/0x270
[<ffffffffa2a7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2a7ea04>] flush_tlb_page+0x54/0xa0
[<ffffffffa2c079d8>] ptep_clear_flush+0x68/0xa0
[<ffffffffa2bef4e5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffa2bf190b>] do_wp_page+0xfb/0x720
[<ffffffffa2bbe20b>] ? unlock_page+0x2b/0x30
[<ffffffffa2bee729>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffa2bf5f82>] handle_mm_fault+0xb22/0xfb0
[<ffffffffa3190653>] __do_page_fault+0x213/0x500
[<ffffffffa3190975>] do_page_fault+0x35/0x90
[<ffffffffa318cac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffa318c778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 a0 15 75 a3 48 89 df e8 c7 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #417704
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8c7113ea
Build id: 3.10.0-957.21.3.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff8c67a4e0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff8c67a4e0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff8c67a4e0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff8c7114ff
Build id: 3.10.0-957.21.3.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8c975cd5
Build id: 3.10.0-957.21.3.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8c711aab
Build id: 3.10.0-957.21.3.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8c67a6a8
Build id: 3.10.0-957.21.3.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8c67a844
Build id: 3.10.0-957.21.3.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff8c7fdb97
Build id: 3.10.0-957.21.3.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffff8c7e5bcb
Build id: 3.10.0-957.21.3.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffff8c7e7e3b
Build id: 3.10.0-957.21.3.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff8c7b6abb
Build id: 3.10.0-957.21.3.el7.x86_64 		0x2b
14
? do_read_fault.isra.61
vmlinux 0xffffffff8c7e4e89
Build id: 3.10.0-957.21.3.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff8c7e97cd
Build id: 3.10.0-957.21.3.el7.x86_64 		0x37d
16
handle_mm_fault
vmlinux 0xffffffff8c7ec27d
Build id: 3.10.0-957.21.3.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffff8cd70603
Build id: 3.10.0-957.21.3.el7.x86_64 		0x203
18
trace_do_page_fault
vmlinux 0xffffffff8cd709d6
Build id: 3.10.0-957.21.3.el7.x86_64 		0x56
19
do_async_page_fault
vmlinux 0xffffffff8cd6ff62
Build id: 3.10.0-957.21.3.el7.x86_64 		0x22
20
async_page_fault
vmlinux 0xffffffff8cd6c798
Build id: 3.10.0-957.21.3.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [bash:183398]
Modules linked in: xt_nat veth gsch(OE) redirfs(OE) nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) dsa_filter(POE) sunrpc ppdev sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg virtio_balloon joydev pcspkr parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common
crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 183398 Comm: bash Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-20180921_191058-build10b184b164b178 04/01/2014
task: ffff9ff910314100 ti: ffff9ff15dae8000 task.ti: ffff9ff15dae8000
RIP: 0010:[<ffffffff8c7113ea>]  [<ffffffff8c7113ea>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff9ff15daebab0  EFLAGS: 00000202
RAX: 00000000000000ff RBX: ffff9ff15daeba80 RCX: 0000000000000001
RDX: 7fffffffffffffff RSI: 00000000000000ff RDI: 0000000000000282
RBP: ffff9ff15daebb00 R08: ffffffff8ce13148 R09: fffff1940be9d440
R10: 0000000000004b28 R11: 00003ffffffff000 R12: ffffffff8c82f35b
R13: 0000000000000002 R14: ffff9ff97ffdb008 R15: 0000000000000000
FS:  00007fa084489740(0000) GS:ffff9ff95b840000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000ac5240 CR3: 00000000bad2e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8c67a4e0>] ? leave_mm+0x110/0x110
[<ffffffff8c67a4e0>] ? leave_mm+0x110/0x110
[<ffffffff8c67a4e0>] ? leave_mm+0x110/0x110
[<ffffffff8c7114ff>] smp_call_function_single+0x5f/0xa0
[<ffffffff8c975cd5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8c711aab>] smp_call_function_many+0x22b/0x270
[<ffffffff8c67a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c67a844>] flush_tlb_page+0x54/0xa0
[<ffffffff8c7fdb97>] ptep_clear_flush+0x67/0xa0
[<ffffffff8c7e5bcb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffff8c7e7e3b>] do_wp_page+0xfb/0x720
[<ffffffff8c7b6abb>] ? unlock_page+0x2b/0x30
[<ffffffff8c7e4e89>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffff8c7e97cd>] handle_pte_fault+0x37d/0xd10
[<ffffffff8c7ec27d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8cd70603>] __do_page_fault+0x203/0x4f0
[<ffffffff8cd709d6>] trace_do_page_fault+0x56/0x150
[<ffffffff8cd6ff62>] do_async_page_fault+0x22/0xf0
[<ffffffff8cd6c798>] async_page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 bc 34 8d 48 89 df e8 4a b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #361622
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb7f14af2
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb7e7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
3
? async_page_fault
vmlinux 0xffffffffb85827a8
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x28
4
? leave_mm
vmlinux 0xffffffffb7e7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
5
? leave_mm
vmlinux 0xffffffffb7e7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
6
smp_call_function_single
vmlinux 0xffffffffb7f14c0f
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x5f
7
? cpumask_next_and
vmlinux 0xffffffffb817fce5
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x35
8
smp_call_function_many
vmlinux 0xffffffffb7f151bb
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x22b
9
native_flush_tlb_others
vmlinux 0xffffffffb7e7d478
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xb8
10
flush_tlb_page
vmlinux 0xffffffffb7e7d614
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x54
11
ptep_clear_flush
vmlinux 0xffffffffb8004108
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x68
12
wp_page_copy.isra.73
vmlinux 0xffffffffb7febce5
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x335
13
do_wp_page
vmlinux 0xffffffffb7fee10b
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xfb
14
? unlock_page
vmlinux 0xffffffffb7fbc0cb
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x2b
15
? do_read_fault.isra.63
vmlinux 0xffffffffb7feaf29
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x139
16
handle_pte_fault
vmlinux 0xffffffffb7fefa95
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x3d5
17
handle_mm_fault
vmlinux 0xffffffffb7ff25fd
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x39d
18
__do_page_fault
vmlinux 0xffffffffb8586653
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x213
19
trace_do_page_fault
vmlinux 0xffffffffb8586a26
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x56
20
do_async_page_fault
vmlinux 0xffffffffb8585fa2
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x22
21
async_page_fault
vmlinux 0xffffffffb85827a8
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [pgrep:14907]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13005(E) nls_utf8 isofs fuse rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache devlink binfmt_misc falcon_lsm_pinned_12905(E) ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_hda_codec_generic lrw gf128mul
glue_helper ablk_helper cryptd snd_hda_intel joydev snd_hda_codec virtio_balloon snd_hda_core sg snd_hwdep snd_seq parport_pc snd_seq_device parport snd_pcm snd_timer snd pcspkr i2c_piix4 soundcore auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel floppy virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 2 PID: 14907 Comm: pgrep Kdump: loaded Tainted: P            EL ------------   3.10.0-1062.4.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffffa0d6dc6f8000 ti: ffffa0d608730000 task.ti: ffffa0d608730000
RIP: 0010:[<ffffffffb7f14af2>]  [<ffffffffb7f14af2>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffffa0d608733aa0  EFLAGS: 00000202
RAX: 0000000000000018 RBX: ffffa0d608733a70 RCX: 0000000000000028
RDX: 0000000000ffffff RSI: 0000000000000018 RDI: 0000000000000286
RBP: ffffa0d608733af0 R08: ffffffffb86141c0 R09: fffffbf03c650ac0
R10: 0000000000005111 R11: 00003ffffffff000 R12: 0000000000000092
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fcec43cf880(0000) GS:ffffa0d71f080000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000076036002 CR3: 0000002e48636000 CR4: 00000000001606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb7e7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffb85827a8>] ? async_page_fault+0x28/0x30
[<ffffffffb7e7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffb7e7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffb7f14c0f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb817fce5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb7f151bb>] smp_call_function_many+0x22b/0x270
[<ffffffffb7e7d478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb7e7d614>] flush_tlb_page+0x54/0xa0
[<ffffffffb8004108>] ptep_clear_flush+0x68/0xa0
[<ffffffffb7febce5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffb7fee10b>] do_wp_page+0xfb/0x720
[<ffffffffb7fbc0cb>] ? unlock_page+0x2b/0x30
[<ffffffffb7feaf29>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffb7fefa95>] handle_pte_fault+0x3d5/0xe20
[<ffffffffb7ff25fd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb8586653>] __do_page_fault+0x213/0x500
[<ffffffffb8586a26>] trace_do_page_fault+0x56/0x150
[<ffffffffb8585fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb85827a8>] async_page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 d6 b4 b8 48 89 df e8 77 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #221841
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff84114a22
Build id: 3.10.0-1062.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8407d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff8407d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff8407d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff84114b3f
Build id: 3.10.0-1062.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8437f475
Build id: 3.10.0-1062.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff841150eb
Build id: 3.10.0-1062.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8407d398
Build id: 3.10.0-1062.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8407d534
Build id: 3.10.0-1062.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff84203b18
Build id: 3.10.0-1062.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff841eb6f5
Build id: 3.10.0-1062.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff841edb1b
Build id: 3.10.0-1062.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff841bbffb
Build id: 3.10.0-1062.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff841ea939
Build id: 3.10.0-1062.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffff841ef4a5
Build id: 3.10.0-1062.el7.x86_64 		0x3d5
16
? __check_object_size
vmlinux 0xffffffff8424521a
Build id: 3.10.0-1062.el7.x86_64 		0x1ca
17
handle_mm_fault
vmlinux 0xffffffff841f200d
Build id: 3.10.0-1062.el7.x86_64 		0x39d
18
__do_page_fault
vmlinux 0xffffffff84786633
Build id: 3.10.0-1062.el7.x86_64 		0x213
19
do_page_fault
vmlinux 0xffffffff84786955
Build id: 3.10.0-1062.el7.x86_64 		0x35
20
page_fault
vmlinux 0xffffffff84782768
Build id: 3.10.0-1062.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [awk:44769]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_pcm joydev pcspkr snd_timer snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport sch_fq_codel ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea nfit
sysfillrect sysimgblt fb_sys_fops libnvdimm ttm ata_piix crct10dif_pclmul crct10dif_common drm mptspi scsi_transport_spi crc32c_intel libata mptscsih mptbase e1000 serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 44769 Comm: awk Kdump: loaded Tainted: G             L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9b84d06b3150 ti: ffff9b858bab8000 task.ti: ffff9b858bab8000
RIP: 0010:[<ffffffff84114a22>]  [<ffffffff84114a22>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9b858babbad0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9b858babbaa0 RCX: 0000000000000004
RDX: ffffffff84813c90 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9b858babbb20 R08: ffffffff84813c88 R09: ffffef2249159c00
R10: 0000000000003677 R11: 00003ffffffff000 R12: 0000000000000028
R13: 0000000000000002 R14: ffff9b86e4bdb008 R15: 0000000000000000
FS:  00007fda4d1f8740(0000) GS:ffff9b86d9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000667000 CR3: 0000000186bd8000 CR4: 00000000003607f0
Call Trace:
[<ffffffff8407d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8407d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8407d1d0>] ? leave_mm+0x110/0x110
[<ffffffff84114b3f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8437f475>] ? cpumask_next_and+0x35/0x50
[<ffffffff841150eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8407d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8407d534>] flush_tlb_page+0x54/0xa0
[<ffffffff84203b18>] ptep_clear_flush+0x68/0xa0
[<ffffffff841eb6f5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff841edb1b>] do_wp_page+0xfb/0x720
[<ffffffff841bbffb>] ? unlock_page+0x2b/0x30
[<ffffffff841ea939>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff841ef4a5>] handle_pte_fault+0x3d5/0xe20
[<ffffffff8424521a>] ? __check_object_size+0x1ca/0x250
[<ffffffff841f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff84786633>] __do_page_fault+0x213/0x500
[<ffffffff84786955>] do_page_fault+0x35/0x90
[<ffffffff84782768>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 d7 d4 84 48 89 df e8 d7 1a 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #204573
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8c516f76
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffff8c47e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8c47e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8c47e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8c51708f
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8c785005
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8c51763b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8c47e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8c47ea94
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff8c607888
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff8c5ef3c5
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff8c5f17eb
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xfb
13
? pick_next_entity
vmlinux 0xffffffff8c4e2e89
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xa9
14
? set_next_entity
vmlinux 0xffffffff8c4e103c
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x3c
15
? __switch_to
vmlinux 0xffffffff8c42b621
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x151
16
handle_mm_fault
vmlinux 0xffffffff8c5f5e82
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb22
17
? __schedule
vmlinux 0xffffffff8cb85942
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x402
18
__do_page_fault
vmlinux 0xffffffff8cb8d653
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x213
19
do_page_fault
vmlinux 0xffffffff8cb8d975
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x35
20
? error_swapgs
vmlinux 0xffffffff8cb89ac9
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xaa
21
page_fault
vmlinux 0xffffffff8cb89778
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [httpd:5227]
Modules linked in: binfmt_misc tcp_diag inet_diag vfat fat isofs ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 ip6table_filter xt_time nf_log_ipv4 nf_log_common nf_nat_ftp xt_conntrack nf_conntrack_ftp xt_LOG xt_limit kcare(OE) ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 xt_REDIRECT nf_nat_redirect iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_owner xt_multiport ext4 mbcache jbd2 loop sunrpc ppdev vmw_balloon pcspkr coretemp joydev sg vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper crc32c_intel syscopyarea sysfillrect
sysimgblt serio_raw fb_sys_fops ttm mptspi scsi_transport_spi ata_piix mptscsih drm libata vmxnet3 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 5227 Comm: httpd Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff9c07aabe9070 ti: ffff9c054dca4000 task.ti: ffff9c054dca4000
RIP: 0010:[<ffffffff8c516f76>]  [<ffffffff8c516f76>] generic_exec_single+0x106/0x1c0
RSP: 0000:ffff9c054dca7b60  EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffff9c07bffce000 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff9c054dca7ba8 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
R13: ffff9c07bffcf008 R14: 0000000000000000 R15: 0000000300000000
FS:  00002aac38803700(0000) GS:ffff9c07bfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002aac40024b50 CR3: 0000000173588000 CR4: 00000000000007e0
Call Trace:
[<ffffffff8c47e730>] ? leave_mm+0x120/0x120
[<ffffffff8c47e730>] ? leave_mm+0x120/0x120
[<ffffffff8c47e730>] ? leave_mm+0x120/0x120
[<ffffffff8c51708f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8c785005>] ? cpumask_next_and+0x35/0x50
[<ffffffff8c51763b>] smp_call_function_many+0x22b/0x270
[<ffffffff8c47e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c47ea94>] flush_tlb_page+0x54/0xa0
[<ffffffff8c607888>] ptep_clear_flush+0x68/0xa0
[<ffffffff8c5ef3c5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff8c5f17eb>] do_wp_page+0xfb/0x720
[<ffffffff8c4e2e89>] ? pick_next_entity+0xa9/0x190
[<ffffffff8c4e103c>] ? set_next_entity+0x3c/0xe0
[<ffffffff8c42b621>] ? __switch_to+0x151/0x580
[<ffffffff8c5f5e82>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8cb85942>] ? __schedule+0x402/0x840
[<ffffffff8cb8d653>] __do_page_fault+0x213/0x500
[<ffffffff8cb8d975>] do_page_fault+0x35/0x90
[<ffffffff8cb89ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffff8cb89778>] page_fault+0x28/0x30
Code: 14 c5 a0 10 15 8d 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #192601
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff841167a2
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8407e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8407e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8407e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff841168bf
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff843862c5
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff84116e6b
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8407e868
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8407ea04
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff84207908
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff841ef415
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff841f183b
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff841be15b
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff841ee659
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffff841f5eb2
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xb22
16
__do_page_fault
vmlinux 0xffffffff8478f653
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x213
17
do_page_fault
vmlinux 0xffffffff8478f975
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x35
18
page_fault
vmlinux 0xffffffff8478b778
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [pgrep:23484]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon sg pcspkr joydev parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx
drm_kms_helper syscopyarea sysfillrect nfit sysimgblt fb_sys_fops ttm drm libnvdimm e1000e ahci ata_piix libahci sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw ptp pps_core vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 23484 Comm: pgrep Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9e86c45f4200 ti: ffff9e86c3624000 task.ti: ffff9e86c3624000
RIP: 0010:[<ffffffff841167a2>]  [<ffffffff841167a2>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9e86c3627b60  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff9e86c3627b30 RCX: 000000000000003e
RDX: 0000000000000003 RSI: 0000000000000002 RDI: 0000000000000286
RBP: ffff9e86c3627ba8 R08: ffffffff848137c0 R09: ffffec0484afdb40
R10: 000000000000026e R11: ffff9e86c2f42ca8 R12: 0000000000000002
R13: ffff9e87fffd9008 R14: 0000000000000000 R15: 00000003c3627ad8
FS:  00007f2c929c3880(0000) GS:ffff9e87ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2c916897e0 CR3: 0000000006a82000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff8407e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8407e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8407e6a0>] ? leave_mm+0x120/0x120
[<ffffffff841168bf>] smp_call_function_single+0x5f/0xa0
[<ffffffff843862c5>] ? cpumask_next_and+0x35/0x50
[<ffffffff84116e6b>] smp_call_function_many+0x22b/0x270
[<ffffffff8407e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8407ea04>] flush_tlb_page+0x54/0xa0
[<ffffffff84207908>] ptep_clear_flush+0x68/0xa0
[<ffffffff841ef415>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff841f183b>] do_wp_page+0xfb/0x720
[<ffffffff841be15b>] ? unlock_page+0x2b/0x30
[<ffffffff841ee659>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff841f5eb2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8478f653>] __do_page_fault+0x213/0x500
[<ffffffff8478f975>] do_page_fault+0x35/0x90
[<ffffffff8478b778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 13 d5 84 48 89 df e8 97 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #182428
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffaf11109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffaf07a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffaf07a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffaf07a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffaf1111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffaf375465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffaf11175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaf07a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffaf07a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffaf1fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffffaf1e55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffffaf1e782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? cpumask_next_and
vmlinux 0xffffffffaf375465
Build id: 3.10.0-957.el7.x86_64 		0x35
14
? smp_call_function_many
vmlinux 0xffffffffaf11175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
15
handle_pte_fault
vmlinux 0xffffffffaf1e91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
16
handle_mm_fault
vmlinux 0xffffffffaf1ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffffaf76f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
18
trace_do_page_fault
vmlinux 0xffffffffaf76f9c6
Build id: 3.10.0-957.el7.x86_64 		0x56
19
do_async_page_fault
vmlinux 0xffffffffaf76ef42
Build id: 3.10.0-957.el7.x86_64 		0x22
20
async_page_fault
vmlinux 0xffffffffaf76b788
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 24s! [rpt_main:8943]
Modules linked in: kbase(OE) udp_diag tcp_diag inet_diag binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel virtio_balloon joydev lrw gf128mul parport_pc glue_helper ablk_helper parport cryptd i2c_piix4 pcspkr ip_tables ext4 mbcache jbd2 ata_generic pata_acpi virtio_net virtio_blk virtio_console cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio
drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kbase]
CPU: 7 PID: 8943 Comm: rpt_main Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
task: ffff933dbaa4e180 ti: ffff933d8ebc8000 task.ti: ffff933d8ebc8000
RIP: 0010:[<ffffffffaf11109a>]  [<ffffffffaf11109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff933d8ebcbab0  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffff933d8ebcba80 RCX: 0000000000000030
RDX: 000000000000ffff RSI: 0000000000000010 RDI: 0000000000000282
RBP: ffff933d8ebcbb00 R08: ffffffffaf814680 R09: ffffdc363ac3a140
R10: 0000000000004ed9 R11: 00003ffffffff000 R12: 0000000000000000
R13: 0000000000000002 R14: ffff933dfffdb008 R15: 0000000000000000
FS:  00007f1301869700(0000) GS:ffff933dbf3c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1300a4baa0 CR3: 0000000fcebf4000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaf07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaf07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaf07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaf1111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffaf375465>] ? cpumask_next_and+0x35/0x50
[<ffffffffaf11175b>] smp_call_function_many+0x22b/0x270
[<ffffffffaf07a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaf07a734>] flush_tlb_page+0x54/0xa0
[<ffffffffaf1fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffffaf1e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffffaf1e782b>] do_wp_page+0xfb/0x720
[<ffffffffaf375465>] ? cpumask_next_and+0x35/0x50
[<ffffffffaf11175b>] ? smp_call_function_many+0x22b/0x270
[<ffffffffaf1e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffffaf1ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffaf76f5e3>] __do_page_fault+0x203/0x500
[<ffffffffaf76f9c6>] trace_do_page_fault+0x56/0x150
[<ffffffffaf76ef42>] do_async_page_fault+0x22/0xf0
[<ffffffffaf76b788>] async_page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 d4 af 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #177288
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb0516566
Build id: 3.10.0-1160.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffb047e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb047e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb047e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb051667f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb0785de5
Build id: 3.10.0-1160.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb0516c2b
Build id: 3.10.0-1160.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb047e868
Build id: 3.10.0-1160.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffb047ea04
Build id: 3.10.0-1160.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffb0607728
Build id: 3.10.0-1160.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffb05ef295
Build id: 3.10.0-1160.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffb05f16bb
Build id: 3.10.0-1160.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffb05bdf6b
Build id: 3.10.0-1160.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffffb05ee4d9
Build id: 3.10.0-1160.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffffb05f5d32
Build id: 3.10.0-1160.el7.x86_64 		0xb22
16
? __check_object_size
vmlinux 0xffffffffb064a6fa
Build id: 3.10.0-1160.el7.x86_64 		0x1ca
17
__do_page_fault
vmlinux 0xffffffffb0b8e653
Build id: 3.10.0-1160.el7.x86_64 		0x213
18
do_page_fault
vmlinux 0xffffffffb0b8e975
Build id: 3.10.0-1160.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffffb0b8a778
Build id: 3.10.0-1160.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [ksmtuned:69561]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ppdev snd_ens1371 snd_rawmidi ghash_clmulni_intel vmw_balloon snd_ac97_codec ac97_bus snd_seq aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm pcspkr
joydev snd_timer sg snd soundcore i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic vmwgfx sr_mod cdrom drm_kms_helper ata_generic syscopyarea pata_acpi sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi ata_piix scsi_transport_spi nfit serio_raw libata mptscsih libnvdimm mptbase e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 69561 Comm: ksmtuned Kdump: loaded Not tainted 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9e4a70a0c200 ti: ffff9e49f90b8000 task.ti: ffff9e49f90b8000
RIP: 0010:[<ffffffffb0516566>]  [<ffffffffb0516566>] generic_exec_single+0x106/0x1c0
RSP: 0000:ffff9e49f90bbb60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9e49f90bbb30 RCX: 0000000000000001
RDX: ffffffffb0c13790 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9e49f90bbba8 R08: ffffffffb0c13788 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
R13: ffff9e4abffdb008 R14: 0000000000000000 R15: 00000003b0785de5
FS:  00007fe95db90740(0000) GS:ffff9e4a9d680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe95d553490 CR3: 000000077a20a000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb047e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb047e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb047e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb051667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb0785de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb0516c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffb047e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb047ea04>] flush_tlb_page+0x54/0xa0
[<ffffffffb0607728>] ptep_clear_flush+0x68/0xa0
[<ffffffffb05ef295>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffb05f16bb>] do_wp_page+0xfb/0x720
[<ffffffffb05bdf6b>] ? unlock_page+0x2b/0x30
[<ffffffffb05ee4d9>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffb05f5d32>] handle_mm_fault+0xb22/0xfb0
[<ffffffffb064a6fa>] ? __check_object_size+0x1ca/0x250
[<ffffffffb0b8e653>] __do_page_fault+0x213/0x500
[<ffffffffb0b8e975>] do_page_fault+0x35/0x90
[<ffffffffb0b8a778>] page_fault+0x28/0x30
Code: 14 c5 e0 13 15 b1 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #24587
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffad516f22
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffad47e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffad47e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffad47e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffad51703f
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffad784fc5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffad5175eb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffad47e8f8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffad47ea94
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffad607858
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffad5ef3a5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffad5f17cb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xfb
13
? native_queued_spin_lock_slowpath
vmlinux 0xffffffffad517e8d
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x1d
14
handle_mm_fault
vmlinux 0xffffffffad5f5e62
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb22
15
? put_prev_entity
vmlinux 0xffffffffad4e69fb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x8b
16
? pick_next_entity
vmlinux 0xffffffffad4e2e89
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xa9
17
__do_page_fault
vmlinux 0xffffffffadb8d653
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x213
18
do_page_fault
vmlinux 0xffffffffadb8d975
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35
19
? error_swapgs
vmlinux 0xffffffffadb89ac9
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xaa
20
page_fault
vmlinux 0xffffffffadb89778
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#31 stuck for 22s! [consul:751]
Modules linked in: tcp_diag inet_diag sunrpc sb_edac coretemp ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper pcspkr joydev parport_pc cryptd vmw_vmci sg parport i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptspi crct10dif_pclmul libata crct10dif_common scsi_transport_spi crc32c_intel mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 31 PID: 751 Comm: consul Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff9cf5cfd80000 ti: ffff9cf968b54000 task.ti: ffff9cf968b54000
RIP: 0010:[<ffffffffad516f22>]  [<ffffffffad516f22>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9cf968b57b60  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffff9cfcc9639900 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff9cf968b57ba8 R08: ffffffffadc15ec0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
R13: ffff9cfdfffc3008 R14: 0000000000000000 R15: 00000003ad47e8f8
FS:  000000c420882868(0000) GS:ffff9cfdff9c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4208828d8 CR3: 000000010bc9c000 CR4: 00000000000407e0
Call Trace:
[<ffffffffad47e730>] ? leave_mm+0x120/0x120
[<ffffffffad47e730>] ? leave_mm+0x120/0x120
[<ffffffffad47e730>] ? leave_mm+0x120/0x120
[<ffffffffad51703f>] smp_call_function_single+0x5f/0xa0
[<ffffffffad784fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffffad5175eb>] smp_call_function_many+0x22b/0x270
[<ffffffffad47e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffad47ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffad607858>] ptep_clear_flush+0x68/0xa0
[<ffffffffad5ef3a5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffad5f17cb>] do_wp_page+0xfb/0x720
[<ffffffffad517e8d>] ? native_queued_spin_lock_slowpath+0x1d/0x200
[<ffffffffad5f5e62>] handle_mm_fault+0xb22/0xfb0
[<ffffffffad4e69fb>] ? put_prev_entity+0x8b/0x400
[<ffffffffad4e2e89>] ? pick_next_entity+0xa9/0x190
[<ffffffffadb8d653>] __do_page_fault+0x213/0x500
[<ffffffffadb8d975>] do_page_fault+0x35/0x90
[<ffffffffadb89ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffadb89778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 60 10 15 ae 48 89 df e8 17 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #23918
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9b3163c2
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9b27e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9b27e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9b27e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff9b3164df
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9b581635
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9b316a8b
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9b27e478
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9b27e614
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff9b4059d8
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff9b3ed5b5
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff9b3ef9db
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xfb
13
handle_pte_fault
vmlinux 0xffffffff9b3f1365
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x3d5
14
? alloc_pages_current
vmlinux 0xffffffff9b416b28
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x98
15
handle_mm_fault
vmlinux 0xffffffff9b3f3ecd
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x39d
16
__do_page_fault
vmlinux 0xffffffff9b988653
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x213
17
trace_do_page_fault
vmlinux 0xffffffff9b988a26
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x56
18
do_async_page_fault
vmlinux 0xffffffff9b987fa2
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22
19
async_page_fault
vmlinux 0xffffffff9b9847a8
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [httpd:9113]
Modules linked in: xt_time nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ip6table_filter ip6_tables tcp_diag inet_diag ipt_REJECT nf_reject_ipv4 xt_owner xt_multiport iptable_filter ext4 mbcache jbd2 loop dm_mirror dm_region_hash dm_log dm_mod ppdev sg parport_pc parport pcspkr i2c_piix4 joydev ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_blk drm ata_piix e1000 libata virtio_pci serio_raw virtio_ring virtio drm_panel_orientation_quirks floppy
CPU: 0 PID: 9113 Comm: httpd Kdump: loaded Tainted: G             L ------------   3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff9f48e18d9070 ti: ffff9f45b123c000 task.ti: ffff9f45b123c000
RIP: 0010:[<ffffffff9b3163c2>]  [<ffffffff9b3163c2>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9f45b123faa0  EFLAGS: 00000202
RAX: 0000000000000006 RBX: ffff9f45b123fa70 RCX: 000000000000003a
RDX: 000000000000003f RSI: 0000000000000006 RDI: 0000000000000286
RBP: ffff9f45b123faf0 R08: ffffffff9ba137c0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff9f45b123fa48
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00002ac6b0708440(0000) GS:ffff9f4927c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055b7bafa7048 CR3: 00000000b123a000 CR4: 00000000000006f0
Call Trace:
[<ffffffff9b27e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9b27e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9b27e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9b3164df>] smp_call_function_single+0x5f/0xa0
[<ffffffff9b581635>] ? cpumask_next_and+0x35/0x50
[<ffffffff9b316a8b>] smp_call_function_many+0x22b/0x270
[<ffffffff9b27e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9b27e614>] flush_tlb_page+0x54/0xa0
[<ffffffff9b4059d8>] ptep_clear_flush+0x68/0xa0
[<ffffffff9b3ed5b5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff9b3ef9db>] do_wp_page+0xfb/0x720
[<ffffffff9b3f1365>] handle_pte_fault+0x3d5/0xe20
[<ffffffff9b416b28>] ? alloc_pages_current+0x98/0x110
[<ffffffff9b3f3ecd>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9b988653>] __do_page_fault+0x213/0x500
[<ffffffff9b988a26>] trace_do_page_fault+0x56/0x150
[<ffffffff9b987fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9b9847a8>] async_page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 20 e8 f4 9b 48 89 df e8 f7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #16809
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffadf16f72
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffade7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffade7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffade7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffadf1708f
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffae185005
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffadf1763b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffade7e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffade7ea94
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffae007888
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffadfef3c5
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffadff17eb
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xfb
13
? try_to_wake_up
vmlinux 0xffffffffadedb6d0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x190
14
handle_mm_fault
vmlinux 0xffffffffadff5e82
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb22
15
__do_page_fault
vmlinux 0xffffffffae58d653
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x213
16
trace_do_page_fault
vmlinux 0xffffffffae58da26
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x56
17
do_async_page_fault
vmlinux 0xffffffffae58cfa2
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x22
18
async_page_fault
vmlinux 0xffffffffae5897a8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [GC Thread#1:24025]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod sg pcspkr virtio_balloon joydev i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom virtio_net net_failover virtio_blk failover ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata virtio_pci serio_raw virtio_ring virtio drm_panel_orientation_quirks
floppy
CPU: 1 PID: 24025 Comm: GC Thread#1 Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org 04/01/2014
task: ffff9f48759062a0 ti: ffff9f485e134000 task.ti: ffff9f485e134000
RIP: 0010:[<ffffffffadf16f72>]  [<ffffffffadf16f72>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9f485e137b30  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff9f485e137b00 RCX: 000000000000003e
RDX: 0000000000000003 RSI: 0000000000000002 RDI: 0000000000000286
RBP: ffff9f485e137b78 R08: ffffffffae6136c0 R09: ffffddf3c85243c0
R10: 0000000000003368 R11: ffff9f4875a1a798 R12: 0000000000000002
R13: ffff9f487ffd7008 R14: 0000000000000000 R15: 00000003254d65ee
FS:  00007f82c4a6d700(0000) GS:ffff9f487fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f82d466a6e0 CR3: 0000000235fac000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffade7e730>] ? leave_mm+0x120/0x120
[<ffffffffade7e730>] ? leave_mm+0x120/0x120
[<ffffffffade7e730>] ? leave_mm+0x120/0x120
[<ffffffffadf1708f>] smp_call_function_single+0x5f/0xa0
[<ffffffffae185005>] ? cpumask_next_and+0x35/0x50
[<ffffffffadf1763b>] smp_call_function_many+0x22b/0x270
[<ffffffffade7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffade7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffae007888>] ptep_clear_flush+0x68/0xa0
[<ffffffffadfef3c5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffadff17eb>] do_wp_page+0xfb/0x720
[<ffffffffadedb6d0>] ? try_to_wake_up+0x190/0x390
[<ffffffffadff5e82>] handle_mm_fault+0xb22/0xfb0
[<ffffffffae58d653>] __do_page_fault+0x213/0x500
[<ffffffffae58da26>] trace_do_page_fault+0x56/0x150
[<ffffffffae58cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffffae5897a8>] async_page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 a0 10 b5 ae 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #14798
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb4f16f72
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb4e7e730
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb4e7e730
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb4e7e730
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb4f1708f
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb5184fc5
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb4f1763b
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb4e7e8f8
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffb4e7ea94
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffb5007888
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffb4fef3c5
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffb4ff17eb
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffb4fbe7ab
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffffb4fee609
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffffb4ff5e82
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xb22
16
__do_page_fault
vmlinux 0xffffffffb558d653
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x213
17
do_page_fault
vmlinux 0xffffffffb558d975
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x35
18
page_fault
vmlinux 0xffffffffb5589778
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [pgrep:52986]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus aesni_intel snd_seq ppdev lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm vmw_balloon
joydev pcspkr snd_timer snd soundcore sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel drm serio_raw libata mptspi nfit libnvdimm e1000 scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 52986 Comm: pgrep Tainted: G             L ------------   3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff9dad27e13150 ti: ffff9dad8f89c000 task.ti: ffff9dad8f89c000
RIP: 0010:[<ffffffffb4f16f72>]  [<ffffffffb4f16f72>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff9dad8f89fb60  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffffffffb5190bf3 RCX: 0000000000000002
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000282
RBP: ffff9dad8f89fba8 R08: ffffffffb5613948 R09: fffff85982e0c900
R10: 00000000000002e7 R11: ffff9dad3ce05d88 R12: 0000000000000002
R13: ffff9dad7ffdb008 R14: 0000000000000000 R15: 000000038f89fad8
FS:  00007fd4cf03b880(0000) GS:ffff9dad7c600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd4cdd017e0 CR3: 00000000a79fa000 CR4: 00000000001607f0
Call Trace:
[<ffffffffb4e7e730>] ? leave_mm+0x120/0x120
[<ffffffffb4e7e730>] ? leave_mm+0x120/0x120
[<ffffffffb4e7e730>] ? leave_mm+0x120/0x120
[<ffffffffb4f1708f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb5184fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb4f1763b>] smp_call_function_many+0x22b/0x270
[<ffffffffb4e7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb4e7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffb5007888>] ptep_clear_flush+0x68/0xa0
[<ffffffffb4fef3c5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffb4ff17eb>] do_wp_page+0xfb/0x720
[<ffffffffb4fbe7ab>] ? unlock_page+0x2b/0x30
[<ffffffffb4fee609>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffb4ff5e82>] handle_mm_fault+0xb22/0xfb0
[<ffffffffb558d653>] __do_page_fault+0x213/0x500
[<ffffffffb558d975>] do_page_fault+0x35/0x90
[<ffffffffb5589778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 a0 10 b5 b5 48 89 df e8 c7 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #14157
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff85b16f72
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff85a7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff85a7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff85a7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff85b1708f
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff85d85005
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff85b1763b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff85a7e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff85a7ea94
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff85c07888
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff85bef3c5
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff85bf17eb
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffff85bbe7ab
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffff85bee609
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffff85bf5e82
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb22
16
__do_page_fault
vmlinux 0xffffffff8618d653
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x213
17
do_page_fault
vmlinux 0xffffffff8618d975
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x35
18
? error_swapgs
vmlinux 0xffffffff86189ac9
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xaa
19
page_fault
vmlinux 0xffffffff86189778
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [pgrep:17140]
Modules linked in: arc4 md4 nls_utf8 cifs ccm dns_resolver fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel ata_piix serio_raw drm libata mptspi scsi_transport_spi nfit libnvdimm mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmfsrtx_bridge_helper]
CPU: 1 PID: 17140 Comm: pgrep Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff92a4e374a0e0 ti: ffff92a446860000 task.ti: ffff92a446860000
RIP: 0010:[<ffffffff85b16f72>]  [<ffffffff85b16f72>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff92a446863b60  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffffffff85d90c33 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff92a446863ba8 R08: ffffffff862136c0 R09: fffff115c234db40
R10: 0000000000004111 R11: ffff92a4f6af3440 R12: 0000000000000002
R13: ffff92a57ffd8008 R14: 0000000000000000 R15: 0000000346863ad8
FS:  00007fc4c277c880(0000) GS:ffff92a57ae40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc4c144c7e0 CR3: 00000000b69ca000 CR4: 00000000000007e0
Call Trace:
[<ffffffff85a7e730>] ? leave_mm+0x120/0x120
[<ffffffff85a7e730>] ? leave_mm+0x120/0x120
[<ffffffff85a7e730>] ? leave_mm+0x120/0x120
[<ffffffff85b1708f>] smp_call_function_single+0x5f/0xa0
[<ffffffff85d85005>] ? cpumask_next_and+0x35/0x50
[<ffffffff85b1763b>] smp_call_function_many+0x22b/0x270
[<ffffffff85a7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85a7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffff85c07888>] ptep_clear_flush+0x68/0xa0
[<ffffffff85bef3c5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff85bf17eb>] do_wp_page+0xfb/0x720
[<ffffffff85bbe7ab>] ? unlock_page+0x2b/0x30
[<ffffffff85bee609>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff85bf5e82>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8618d653>] __do_page_fault+0x213/0x500
[<ffffffff8618d975>] do_page_fault+0x35/0x90
[<ffffffff86189ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffff86189778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 a0 10 75 86 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #9351
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb931109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffb927a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffb927a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffb927a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffb93111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb9575465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb931175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb927a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffb927a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffb93fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffffb93e55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffffb93e782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffb93b64fb
Build id: 3.10.0-957.el7.x86_64 		0x2b
14
? do_read_fault.isra.61
vmlinux 0xffffffffb93e4879
Build id: 3.10.0-957.el7.x86_64 		0x139
15
handle_pte_fault
vmlinux 0xffffffffb93e91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
16
handle_mm_fault
vmlinux 0xffffffffb93ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffffb996f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
18
do_page_fault
vmlinux 0xffffffffb996f915
Build id: 3.10.0-957.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffffb996b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [nm-dhcp-helper:9583]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec joydev pcspkr ac97_bus snd_seq btusb btrtl btbcm snd_seq_device btintel snd_pcm bluetooth snd_timer snd sg rfkill soundcore parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif
crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic ahci libahci nfit e1000 libnvdimm pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix scsi_transport_spi drm mptscsih mptbase libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 9583 Comm: nm-dhcp-helper Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8a2a75d3d140 ti: ffff8a2a64690000 task.ti: ffff8a2a64690000
RIP: 0010:[<ffffffffb931109a>]  [<ffffffffb931109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff8a2a64693ae0  EFLAGS: 00000202
RAX: ffff8a2a92f39b00 RBX: ffff8a2a6477c798 RCX: ffff8a2a92f39b10
RDX: ffff8a2abb65b700 RSI: ffff8a2a64693ae0 RDI: ffff8a2a64693ae0
RBP: ffff8a2a64693b30 R08: 0000000000000001 R09: fffff7ffc0e415c0
R10: 0000000000003690 R11: 00003ffffffff000 R12: ffff8a2a00000028
R13: 0000000000000001 R14: ffff8a2abffdb008 R15: 0000000000000000
FS:  00007f7293a46880(0000) GS:ffff8a2abb6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7291cba7c0 CR3: 0000000038010000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb927a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb927a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb927a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb93111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb9575465>] ? cpumask_next_and+0x35/0x50
[<ffffffffb931175b>] smp_call_function_many+0x22b/0x270
[<ffffffffb927a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb927a734>] flush_tlb_page+0x54/0xa0
[<ffffffffb93fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffffb93e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffffb93e782b>] do_wp_page+0xfb/0x720
[<ffffffffb93b64fb>] ? unlock_page+0x2b/0x30
[<ffffffffb93e4879>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffffb93e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffffb93ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb996f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb996f915>] do_page_fault+0x35/0x90
[<ffffffffb996b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 f4 b9 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #512
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa3f16f26
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffa3e7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa3e7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa3e7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa3f1703f
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa4184fc5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa3f175eb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa3e7e8f8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffa3e7ea94
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffa4007858
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffa3fef3a5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffa3ff17cb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xfb
13
? unlock_page
vmlinux 0xffffffffa3fbe75b
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x2b
14
? do_read_fault.isra.63
vmlinux 0xffffffffa3fee5e9
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x139
15
handle_mm_fault
vmlinux 0xffffffffa3ff5e62
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb22
16
__do_page_fault
vmlinux 0xffffffffa458d653
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x213
17
trace_do_page_fault
vmlinux 0xffffffffa458da26
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x56
18
do_async_page_fault
vmlinux 0xffffffffa458cfa2
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x22
19
async_page_fault
vmlinux 0xffffffffa45897a8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [antispam-mysql.:6348]
Modules linked in: nf_conntrack_netlink nfnetlink binfmt_misc vboxnetadp(OE) vboxnetflt(OE) xt_conntrack iptable_filter xt_REDIRECT nf_nat_redirect vboxdrv(OE) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sg ppdev joydev pcspkr parport_pc virtio_balloon parport i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net virtio_console net_failover virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 6348 Comm: antispam-mysql. Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org 04/01/2014
task: ffff8f9bb8b31070 ti: ffff8f9c52360000 task.ti: ffff8f9c52360000
RIP: 0010:[<ffffffffa3f16f26>]  [<ffffffffa3f16f26>] generic_exec_single+0x106/0x1c0
RSP: 0000:ffff8f9c52363b30  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8f9c52363b00 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000286
RBP: ffff8f9c52363b78 R08: ffffffffa46136c0 R09: 7261646e656c6163
R10: 5f6863726165735f R11: 737961645f78616d R12: 0000000000000002
R13: ffff8f9d3ffd5008 R14: 0000000000000000 R15: 0000000300000000
FS:  00007f1580bbd740(0000) GS:ffff8f9d3fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000025e5278 CR3: 000000012dc32000 CR4: 00000000000006e0
Call Trace:
[<ffffffffa3e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa3e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa3e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa3f1703f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa4184fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa3f175eb>] smp_call_function_many+0x22b/0x270
[<ffffffffa3e7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa3e7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffa4007858>] ptep_clear_flush+0x68/0xa0
[<ffffffffa3fef3a5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffa3ff17cb>] do_wp_page+0xfb/0x720
[<ffffffffa3fbe75b>] ? unlock_page+0x2b/0x30
[<ffffffffa3fee5e9>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffa3ff5e62>] handle_mm_fault+0xb22/0xfb0
[<ffffffffa458d653>] __do_page_fault+0x213/0x500
[<ffffffffa458da26>] trace_do_page_fault+0x56/0x150
[<ffffffffa458cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffffa45897a8>] async_page_fault+0x28/0x30
Code: 14 c5 60 10 b5 a4 48 89 df e8 17 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f