Info

Permalink
Function
generic_exec_single
First occurrence
2020-09-30
Last occurrence
2026-07-31
State
NEW
Components
kernel
Quality
-41

Statistics

Operating system Count
CentOS 7 414
Architecture Count
x86_64 414
Related packages Count
kernel 414
 0:3.10.0-1160.59.1.el7 121
 0:3.10.0-1062.12.1.el7 106
 0:3.10.0-957.el7 82
 0:3.10.0-862.el7 21
 0:3.10.0-1160.71.1.el7 17
 0:3.10.0-1160.el7 14
 0:3.10.0-1160.45.1.el7 9
 0:3.10.0-1160.66.1.el7 8
 0:3.10.0-1062.el7 7
 0:3.10.0-862.2.3.el7 5
 0:3.10.0-1062.4.1.el7 5
 0:3.10.0-1160.11.1.el7 4
 0:3.10.0-1160.53.1.el7 4
 0:3.10.0-1160.24.1.el7 2
 0:3.10.0-1160.36.2.el7 2
 0:3.10.0-862.11.6.el7 1
 0:3.10.0-862.6.3.el7 1
 0:3.10.0-1160.49.1.el7 1
 0:3.10.0-1062.9.1.el7 1
 0:3.10.0-1160.62.1.el7 1
 0:3.10.0-862.3.3.el7 1
 0:3.10.0-1160.76.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #913850
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9871109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff9867a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9867a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9867a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff987111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff98975465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9871175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9867a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9867a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff987e4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff987e5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffff987e87c2
Build id: 3.10.0-957.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffff987e368d
Build id: 3.10.0-957.el7.x86_64 		0x49d
14
? system_call_after_swapgs
vmlinux 0xffffffff98d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffff98d74d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
16
system_call_fastpath
vmlinux 0xffffffff98d74ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
17
? system_call_after_swapgs
vmlinux 0xffffffff98d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 22s! [calico-node:24923]
Modules linked in: xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth iptable_raw xt_CT rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_addrtype xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipport ip_set_hash_ipportip ip_set dummy ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 devlink tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev sg joydev parport_pc parport i2c_piix4 virtio_balloon pcspkr br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net cirrus drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata drm serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 24923 Comm: calico-node Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9496eac85140 ti: ffff94a5b0a04000 task.ti: ffff94a5b0a04000
RIP: 0010:[<ffffffff9871109a>]  [<ffffffff9871109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff94a5b0a07bf0  EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff94a5b0a07bc0 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000282
RBP: ffff94a5b0a07c38 R08: ffffffff98e12d80 R09: 00003ffffffff000
R10: ffff94a6212d1440 R11: 0000000000000246 R12: ffff94a67ff9e000
R13: ffffffff987c0c06 R14: ffff94a5b0a07c00 R15: 0000000000020200
FS:  000000c00007e090(0000) GS:ffff94a63ecc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4236cb000 CR3: 0000000fb1b20000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff987111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff98975465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9871175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9867a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9867a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff987e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff987e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff987e87c2>] zap_page_range+0x142/0x190
[<ffffffff987e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff98d74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff98d74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff98d74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff98d74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 34 99 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #826248
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffaed1109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffaec7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffaec7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffaec7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffaed111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffaef75465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffaed1175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaec7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffaec7a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffaede4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffaede5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffffaede87c2
Build id: 3.10.0-957.el7.x86_64 		0x142
13
? handle_mm_fault
vmlinux 0xffffffffaedebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
14
SyS_madvise
vmlinux 0xffffffffaede368d
Build id: 3.10.0-957.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffffaf374d21
Build id: 3.10.0-957.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffffaf374d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffffaf374ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
18
? system_call_after_swapgs
vmlinux 0xffffffffaf374d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [agent_service:18619]
Modules linked in: binfmt_misc sysmon_edr(OE) netfilter_edr(OE) ppdev sg i2c_piix4 joydev virtio_balloon parport_pc parport pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net serio_raw cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 18619 Comm: agent_service Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff9cfd8e7c1040 ti: ffff9d00d023c000 task.ti: ffff9d00d023c000
RIP: 0010:[<ffffffffaed1109a>]  [<ffffffffaed1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9d00d023fbf0  EFLAGS: 00000202
RAX: ffff9d00d963c600 RBX: 00007fccc9f60000 RCX: ffff9d00d963c610
RDX: ffff9d00dec5b700 RSI: ffff9d00d023fbf0 RDI: ffff9d00d023fbf0
RBP: ffff9d00d023fc38 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9cfd7bb0be60 R11: 0000000000000206 R12: ffff9cfb5079b7a0
R13: ffffffffaedf6e08 R14: ffff9d00d023fb78 R15: ffffffffaedc588e
FS:  00007fccbf24d700(0000) GS:ffff9d00ded00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fccc86413a0 CR3: 000000081075c000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaed111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffaef75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffaed1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffaec7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaec7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaede4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffaede5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffaede87c2>] zap_page_range+0x142/0x190
[<ffffffffaedebc6d>] ? handle_mm_fault+0x39d/0x9b0
[<ffffffffaede368d>] SyS_madvise+0x49d/0xac0
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaf374d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaf374ddb>] system_call_fastpath+0x22/0x27
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 94 af 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #746579
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff94b1109e
Build id: 3.10.0-957.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffff94a7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff94a7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff94a7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff94b111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff94d75465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff94b1175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff94a7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff94a7a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff94be4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff94be5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffff94be87c2
Build id: 3.10.0-957.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffff94be368d
Build id: 3.10.0-957.el7.x86_64 		0x49d
14
? system_call_after_swapgs
vmlinux 0xffffffff95174d21
Build id: 3.10.0-957.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffff95174d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
16
system_call_fastpath
vmlinux 0xffffffff95174ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
17
? system_call_after_swapgs
vmlinux 0xffffffff95174d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 29s! [java:26616]
Modules linked in: drbd_transport_tcp(OE) drbd(OE) tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc sg pcspkr i2c_piix4 joydev virtio_balloon sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_gpu drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm virtio_scsi virtio_net drm_panel_orientation_quirks ata_piix libata floppy serio_raw virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 26616 Comm: java Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffff8f661bf20000 ti: ffff8f6a193f0000 task.ti: ffff8f6a193f0000
RIP: 0010:[<ffffffff94b1109e>]  [<ffffffff94b1109e>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff8f6a193f3bf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8f6a193f3bc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8f6a193f3c38 R08: ffffffff95213000 R09: 00003ffffffff000
R10: ffff8f6709b32510 R11: 00007f8b3baf1000 R12: ffff8f6a3ffce000
R13: ffffffff94bc0c06 R14: ffff8f6a193f3c00 R15: 0000000000020200
FS:  00007f8b3bbee700(0000) GS:ffff8f6a3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8b40287a78 CR3: 000000042e5f4000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94b111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff94d75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff94b1175b>] smp_call_function_many+0x22b/0x270
[<ffffffff94a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff94a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff94be4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff94be5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff94be87c2>] zap_page_range+0x142/0x190
[<ffffffff94be368d>] SyS_madvise+0x49d/0xac0
[<ffffffff95174d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff95174d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff95174ddb>] system_call_fastpath+0x22/0x27
[<ffffffff95174d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 89 de 48 03 14 c5 60 b9 74 95 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87
Complete report #399761
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa3f07ace
Build id: 3.10.0-862.3.3.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffffa3e77110
Build id: 3.10.0-862.3.3.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa3e77110
Build id: 3.10.0-862.3.3.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa3e77110
Build id: 3.10.0-862.3.3.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa3f07bdf
Build id: 3.10.0-862.3.3.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa414a6b5
Build id: 3.10.0-862.3.3.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa3f0818b
Build id: 3.10.0-862.3.3.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa3e772d8
Build id: 3.10.0-862.3.3.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa3e77348
Build id: 3.10.0-862.3.3.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffa3fc07d7
Build id: 3.10.0-862.3.3.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa3fc1fb5
Build id: 3.10.0-862.3.3.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffffa3fc4d42
Build id: 3.10.0-862.3.3.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffffa3fbfe3d
Build id: 3.10.0-862.3.3.el7.x86_64 		0x3cd
14
? system_call_after_swapgs
vmlinux 0xffffffffa452077b
Build id: 3.10.0-862.3.3.el7.x86_64 		0xc8
15
? system_call_after_swapgs
vmlinux 0xffffffffa452076f
Build id: 3.10.0-862.3.3.el7.x86_64 		0xbc
16
system_call_fastpath
vmlinux 0xffffffffa452082f
Build id: 3.10.0-862.3.3.el7.x86_64 		0x1c
17
? system_call_after_swapgs
vmlinux 0xffffffffa452077b
Build id: 3.10.0-862.3.3.el7.x86_64 		0xc8 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [spades-hammer:5712]
Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxsf(OE) sunrpc vboxguest(OE) snd_intel8x0 vmwgfx snd_ac97_codec ttm ac97_bus ppdev drm_kms_helper snd_seq joydev syscopyarea snd_seq_device sysfillrect snd_pcm sysimgblt iosf_mbi fb_sys_fops snd_timer snd drm sg soundcore i2c_piix4 pcspkr parport_pc video parport i2c_core ip_tables
xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod crct10dif_common cdrom ata_generic pata_acpi ata_piix libata crc32c_intel serio_raw e1000 dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 5712 Comm: spades-hammer Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.3.3.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff957741033f40 ti: ffff957757b2c000 task.ti: ffff957757b2c000
RIP: 0010:[<ffffffffa3f07ace>]  [<ffffffffa3f07ace>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff957757b2fbf0  EFLAGS: 00010202
RAX: ffff9576aa145500 RBX: ffff95775ffd5800 RCX: ffff9576aa145510
RDX: ffff95775fc19680 RSI: ffff957757b2fbf0 RDI: ffff957757b2fbf0
RBP: ffff957757b2fc38 R08: 0000000000000001 R09: ffff95770e4d9af8
R10: 0000000000000006 R11: 0000000000000202 R12: 000280da00000000
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fc9c03fd700(0000) GS:ffff95775fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc9c7c0c000 CR3: 0000000035c94000 CR4: 00000000000206e0
Call Trace:
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3f07bdf>] smp_call_function_single+0x5f/0xa0
[<ffffffffa414a6b5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa3f0818b>] smp_call_function_many+0x22b/0x270
[<ffffffffa3e772d8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa3e77348>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa3fc07d7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa3fc1fb5>] tlb_finish_mmu+0x55/0x60
[<ffffffffa3fc4d42>] zap_page_range+0x142/0x190
[<ffffffffa3fbfe3d>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa452077b>] ? system_call_after_swapgs+0xc8/0x160
[<ffffffffa452076f>] ? system_call_after_swapgs+0xbc/0x160
[<ffffffffa452082f>] system_call_fastpath+0x1c/0x21
[<ffffffffa452077b>] ? system_call_after_swapgs+0xc8/0x160
Code: 48 89 de 48 03 14 c5 e0 52 b3 a4 48 89 df e8 5a 88 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87
Complete report #3953
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa270779a
Build id: 3.10.0-862.2.3.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffa2676e60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa2676e60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa2676e60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa27078af
Build id: 3.10.0-862.2.3.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa294a215
Build id: 3.10.0-862.2.3.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa2707e5b
Build id: 3.10.0-862.2.3.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa2677028
Build id: 3.10.0-862.2.3.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa2677098
Build id: 3.10.0-862.2.3.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffa27c0437
Build id: 3.10.0-862.2.3.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa27c1c15
Build id: 3.10.0-862.2.3.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffffa27c49a2
Build id: 3.10.0-862.2.3.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffffa27bfa9d
Build id: 3.10.0-862.2.3.el7.x86_64 		0x3cd
14
? system_call_after_swapgs
vmlinux 0xffffffffa2d1f6e1
Build id: 3.10.0-862.2.3.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffffa2d1f6d5
Build id: 3.10.0-862.2.3.el7.x86_64 		0xa2
16
system_call_fastpath
vmlinux 0xffffffffa2d1f795
Build id: 3.10.0-862.2.3.el7.x86_64 		0x1c
17
? system_call_after_swapgs
vmlinux 0xffffffffa2d1f6e1
Build id: 3.10.0-862.2.3.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [mysqld:23520]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr nfit joydev vmw_balloon sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih libata e1000 crct10dif_pclmul crct10dif_common mptbase crc32c_intel i2c_core serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 23520 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8b12ddefbf40 ti: ffff8b12eea8c000 task.ti: ffff8b12eea8c000
RIP: 0010:[<ffffffffa270779a>]  [<ffffffffa270779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8b12eea8fbf0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8b12eea8fbc0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8b12eea8fc38 R08: ffffffffa2e13920 R09: ffff8b12e1259288
R10: ffffdf04c670a3c0 R11: 0000000000000206 R12: ffff8b12ddefbf40
R13: ffff8b12ddefbf40 R14: 0000000000000002 R15: 0000000000000030
FS:  00007ff026e2bb00(0000) GS:ffff8b12ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005631f8c5e000 CR3: 0000000017cb4000 CR4: 00000000000607f0
Call Trace:
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa27078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa294a215>] ? cpumask_next_and+0x35/0x50
[<ffffffffa2707e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffa2677028>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2677098>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa27c0437>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa27c1c15>] tlb_finish_mmu+0x55/0x60
[<ffffffffa27c49a2>] zap_page_range+0x142/0x190
[<ffffffffa27bfa9d>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa2d1f6e1>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa2d1f6d5>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffa2d1f795>] system_call_fastpath+0x1c/0x21
[<ffffffffa2d1f6e1>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 33 a3 48 89 df e8 ea 86 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #579827
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa7316ae2
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa727ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa727ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa727ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa7316bff
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa7586845
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa73171ab
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa727e258
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa727e2cb
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa73ee6a6
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa73f016f
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa73f0293
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffa73f375a
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xea
14
? kmem_cache_free
vmlinux 0xffffffffa7427022
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x1e2
15
SyS_madvise
vmlinux 0xffffffffa73edc4d
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x49d
16
? system_call_after_swapgs
vmlinux 0xffffffffa7999ed5
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffa7999ec9
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffffa7999f92
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffffa7999ed5
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [mysqld:27149]
Modules linked in: binfmt_misc ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod squashfs loop pcspkr edac_mce_amd joydev sg i2c_piix4 virtio_balloon ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_scsi virtio_net net_failover virtio_console failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix serio_raw libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy fuse
CPU: 1 PID: 27149 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
task: ffff8d70dafe5280 ti: ffff8d710c4b8000 task.ti: ffff8d710c4b8000
RIP: 0010:[<ffffffffa7316ae2>]  [<ffffffffa7316ae2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8d710c4bbbe0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8d710c4bbbb0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8d710c4bbc30 R08: ffffffffa7a13f40 R09: 00003ffffffff000
R10: ffff8d71f5b36000 R11: 00007f17ac9ae000 R12: ffff8d710c4bbcd0
R13: 0000000000000005 R14: ffff8d70ca9c4490 R15: ffffffffa7842c54
FS:  00007f17ac9f6700(0000) GS:ffff8d71ffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f17ac9eeff8 CR3: 0000000134b6a000 CR4: 00000000000007e0
Call Trace:
 [<ffffffffa727ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffa727ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffa727ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffa7316bff>] smp_call_function_single+0x5f/0xa0
 [<ffffffffa7586845>] ? cpumask_next_and+0x35/0x50
 [<ffffffffa73171ab>] smp_call_function_many+0x22b/0x270
 [<ffffffffa727e258>] native_flush_tlb_others+0xb8/0xc0
 [<ffffffffa727e2cb>] flush_tlb_mm_range+0x6b/0x140
 [<ffffffffa73ee6a6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
 [<ffffffffa73f016f>] arch_tlb_finish_mmu+0x3f/0x80
 [<ffffffffa73f0293>] tlb_finish_mmu+0x23/0x30
 [<ffffffffa73f375a>] zap_page_range+0xea/0x130
 [<ffffffffa7427022>] ? kmem_cache_free+0x1e2/0x200
 [<ffffffffa73edc4d>] SyS_madvise+0x49d/0xac0
 [<ffffffffa7999ed5>] ? system_call_after_swapgs+0xa2/0x13a
 [<ffffffffa7999ec9>] ? system_call_after_swapgs+0x96/0x13a
 [<ffffffffa7999f92>] system_call_fastpath+0x25/0x2a
 [<ffffffffa7999ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 00 19 f5 a7 48 89 df e8 d7 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #546629
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa57163d2
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa567e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa567e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa567e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa57164ef
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa5981685
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa5716a9b
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa567e478
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa567e4eb
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa57ec096
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa57edb5f
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa57edc83
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffa57f098a
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffa57eb69d
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffffa5d8de21
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffffa5d8de15
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffffa5d8dede
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffffa5d8de21
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 67s! [java:158508]
Modules linked in: tcp_lp fuse binfmt_misc xt_NFQUEUE nfnetlink_log nfnetlink tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter nf_conntrack_ipv6 nf_defrag_ipv6 rpcsec_gss_krb5 nf_conntrack_ipv4 auth_rpcgss nf_defrag_ipv4 nf_conntrack nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev sg i2c_piix4 parport_pc parport virtio_balloon joydev pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci floppy virtio_ring serio_raw drm_panel_orientation_quirks
virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kbase]
CPU: 13 PID: 158508 Comm: java Kdump: loaded Tainted: GF          OEL ------------   3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff939c6af920e0 ti: ffff93aa6cf18000 task.ti: ffff93aa6cf18000
RIP: 0010:[<ffffffffa57163d2>]  [<ffffffffa57163d2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff93aa6cf1bbe0  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff93aa6cf1bbb0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff93aa6cf1bc30 R08: ffffffffa5e137c8 R09: 00003ffffffff000
R10: ffff93aa3e31cbd0 R11: 00007f4bf0861000 R12: ffffffffa58b753a
R13: ffff93aa6cf1bb90 R14: 0000000000001000 R15: ffffffffa57c9512
FS:  00007f4bf095e700(0000) GS:ffff93aa7d140000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4c5f6bb530 CR3: 00000004d6a78000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa567e2b0>] ? leave_mm+0x110/0x110
[<ffffffffa567e2b0>] ? leave_mm+0x110/0x110
[<ffffffffa567e2b0>] ? leave_mm+0x110/0x110
[<ffffffffa57164ef>] smp_call_function_single+0x5f/0xa0
[<ffffffffa5981685>] ? cpumask_next_and+0x35/0x50
[<ffffffffa5716a9b>] smp_call_function_many+0x22b/0x270
[<ffffffffa567e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa567e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa57ec096>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa57edb5f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa57edc83>] tlb_finish_mmu+0x23/0x30
[<ffffffffa57f098a>] zap_page_range+0xea/0x130
[<ffffffffa57eb69d>] SyS_madvise+0x49d/0xac0
[<ffffffffa5d8de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa5d8de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffa5d8dede>] system_call_fastpath+0x25/0x2a
[<ffffffffa5d8de21>] ? system_call_after_swapgs+0xae/0x146
Code: 89 de 48 03 14 c5 20 e8 34 a6 48 89 df e8 37 23 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #467668
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff87316b62
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8727ddd0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8727ddd0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8727ddd0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff87316c7f
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff87586835
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8731722b
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8727e258
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8727e2cb
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff873ee676
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff873f013f
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff873f0263
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff873f372a
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff873edc1d
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffff87999ed5
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xa2
16
? system_call_after_swapgs
vmlinux 0xffffffff87999ec9
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x96
17
system_call_fastpath
vmlinux 0xffffffff87999f92
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffff87999ed5
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [mysqld:3753]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) vboxsf(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod vboxvideo(OE) ttm snd_intel8x0 snd_ac97_codec ext4 drm_kms_helper ac97_bus syscopyarea mbcache snd_seq sysfillrect jbd2 snd_seq_device sysimgblt ppdev joydev snd_pcm fb_sys_fops pcspkr
snd_timer video drm snd sg parport_pc soundcore vboxguest(OE) parport i2c_piix4 drm_panel_orientation_quirks ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi ahci libahci ata_piix crc32c_intel serio_raw libata e1000 fuse
CPU: 3 PID: 3753 Comm: mysqld Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff894778aee300 ti: ffff89464bfec000 task.ti: ffff89464bfec000
RIP: 0010:[<ffffffff87316b62>]  [<ffffffff87316b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff89464bfefbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 0000000000000001 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff89464bfefc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff894778208510 R11: 00007f97ac91b000 R12: ffff894778aee300
R13: ffff8945b4cab480 R14: 0000000000000002 R15: ffff894796d9ad40
FS:  00007f97ac964700(0000) GS:ffff894796d80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4634cb7c70 CR3: 000000020a2e4000 CR4: 00000000000206e0
Call Trace:
[<ffffffff8727ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8727ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8727ddd0>] ? leave_mm+0x120/0x120
[<ffffffff87316c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffff87586835>] ? cpumask_next_and+0x35/0x50
[<ffffffff8731722b>] smp_call_function_many+0x22b/0x270
[<ffffffff8727e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8727e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff873ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff873f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff873f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff873f372a>] zap_page_range+0xea/0x130
[<ffffffff873edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff87999ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff87999ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff87999f92>] system_call_fastpath+0x25/0x2a
[<ffffffff87999ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 f5 87 48 89 df e8 47 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #362073
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8a516a32
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8a47ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8a47ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8a47ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8a516b4f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8a786a45
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8a5170fb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8a47e258
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8a47e2cb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8a5ee676
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff8a5f013f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff8a5f0263
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff8a5f372a
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff8a5edc1d
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x49d
15
? __switch_to
vmlinux 0xffffffff8a42b981
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x4b1
16
? system_call_after_swapgs
vmlinux 0xffffffff8ab95ed5
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffff8ab95ec9
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffff8ab95f92
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff8ab95ed5
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [Cache2 I/O:4803]
Modules linked in: nls_utf8 isofs vboxsf(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_intel8x0 snd_ac97_codec ac97_bus snd_seq ppdev snd_seq_device joydev pcspkr iosf_mbi snd_pcm vboxguest(OE) snd_timer sg parport_pc video parport snd i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic crct10dif_common pata_acpi
hid_multitouch serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm ata_piix libahci drm_panel_orientation_quirks crc32c_intel e1000 libata dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 4803 Comm: Cache2 I/O Kdump: loaded Tainted: G           OE  ------------   3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff90bbd67b6300 ti: ffff90baae164000 task.ti: ffff90baae164000
RIP: 0010:[<ffffffff8a516a32>]  [<ffffffff8a516a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff90baae167be0  EFLAGS: 00010202
RAX: 00000000000008fc RBX: 00000000000000fb RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff90baae167c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff90baa9d40bd0 R11: 0000000000000206 R12: 0000000000000000
R13: 00000000000008fd R14: 0000000000000001 R15: 00003ffffffff000
FS:  00007f5d11b36700(0000) GS:ffff90bbdfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3c06a09000 CR3: 0000000144514000 CR4: 00000000000006e0
Call Trace:
[<ffffffff8a47ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8a47ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8a47ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8a516b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8a786a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff8a5170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff8a47e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8a47e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8a5ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8a5f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8a5f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff8a5f372a>] zap_page_range+0xea/0x130
[<ffffffff8a5edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff8a42b981>] ? __switch_to+0x4b1/0x580
[<ffffffff8ab95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff8ab95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff8ab95f92>] system_call_fastpath+0x25/0x2a
[<ffffffff8ab95ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 15 8b 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #285043
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff86716a32
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8667ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8667ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8667ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff86716b4f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff86986a45
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff867170fb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8667e258
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8667e2cb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff867ee676
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff867f013f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff867f0263
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff867f372a
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff867edc1d
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffff86d95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2
16
? system_call_after_swapgs
vmlinux 0xffffffff86d95ec9
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x96
17
system_call_fastpath
vmlinux 0xffffffff86d95f92
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffff86d95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [.NET ThreadPool:21362]
Modules linked in: tcp_lp nls_utf8 isofs nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) sunrpc snd_intel8x0 ppdev snd_ac97_codec ac97_bus snd_seq joydev snd_seq_device snd_pcm iosf_mbi snd_timer crc32_pclmul sg snd ghash_clmulni_intel vboxguest(OE) aesni_intel soundcore parport_pc parport lrw gf128mul i2c_piix4 glue_helper video ablk_helper pcspkr cryptd ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ata_piix ttm drm libata crct10dif_pclmul crct10dif_common crc32c_intel drm_panel_orientation_quirks e1000 serio_raw dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 21362 Comm: .NET ThreadPool Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8ad56c191080 ti: ffff8ad0c5818000 task.ti: ffff8ad0c5818000
RIP: 0010:[<ffffffff86716a32>]  [<ffffffff86716a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8ad0c581bbe0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8ad0c581bbb0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8ad0c581bc30 R08: ffffffff86e137c0 R09: 00003ffffffff000
R10: ffff8ad287296288 R11: 00007fea1efff000 R12: ffff8ad56c191080
R13: ffff8ad035d11a80 R14: 0000000000000140 R15: ffff8ad5fbc9ad40
FS:  00007fea1f7fe700(0000) GS:ffff8ad5fbc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1aa5fa4000 CR3: 00000005767a8000 CR4: 00000000000606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8667ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8667ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8667ddd0>] ? leave_mm+0x120/0x120
[<ffffffff86716b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff86986a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff867170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff8667e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8667e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff867ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff867f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff867f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff867f372a>] zap_page_range+0xea/0x130
[<ffffffff867edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff86d95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff86d95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff86d95f92>] system_call_fastpath+0x25/0x2a
[<ffffffff86d95ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 35 87 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #65135
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff98116802
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9807e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff9807e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff9807e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff9811691f
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff98385fe5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff98116ecb
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9807e868
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9807e8db
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff981edee6
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff981ef9af
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff981efad3
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff981f2f9a
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xea
14
? handle_mm_fault
vmlinux 0xffffffff981f5fed
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xbfd
15
SyS_madvise
vmlinux 0xffffffff981ed48d
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x49d
16
? system_call_after_swapgs
vmlinux 0xffffffff98793ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffff98793ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffff98793f92
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff98793ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 26s! [ossec-agentd:9009]
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix serio_raw mptspi scsi_transport_spi libata mptscsih e1000 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 9009 Comm: ossec-agentd Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff9cf166765280 ti: ffff9cf1664bc000 task.ti: ffff9cf1664bc000
RIP: 0010:[<ffffffff98116802>]  [<ffffffff98116802>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9cf1664bfbe0  EFLAGS: 00000202
RAX: 00000000000008fd RBX: 0000000000000020 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9cf1664bfc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9cf2c3ee76c8 R11: 0000000000000246 R12: ffffffff981c5d1e
R13: ffff9cf1664bfbb8 R14: ffff9cf2fffd6090 R15: fffff61d04de11c0
FS:  00007fd7f7323700(0000) GS:ffff9cf2ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd7ec03e040 CR3: 00000002241ce000 CR4: 00000000000407e0
Call Trace:
[<ffffffff9807e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9807e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9807e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9811691f>] smp_call_function_single+0x5f/0xa0
[<ffffffff98385fe5>] ? cpumask_next_and+0x35/0x50
[<ffffffff98116ecb>] smp_call_function_many+0x22b/0x270
[<ffffffff9807e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9807e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff981edee6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff981ef9af>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff981efad3>] tlb_finish_mmu+0x23/0x30
[<ffffffff981f2f9a>] zap_page_range+0xea/0x130
[<ffffffff981f5fed>] ? handle_mm_fault+0xbfd/0xfb0
[<ffffffff981ed48d>] SyS_madvise+0x49d/0xac0
[<ffffffff98793ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff98793ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff98793f92>] system_call_fastpath+0x25/0x2a
[<ffffffff98793ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 13 d5 98 48 89 df e8 57 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #24806
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffbbf14af6
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffbbe7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffbbe7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffbbe7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffbbf14c0f
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffbc17fce5
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffbbf151bb
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffbbe7d478
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffbbe7d4eb
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffbbfea756
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffbbfec21f
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffbbfec343
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffbbfef04a
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffbbfe9d5d
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffffbc58be21
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffffbc58be15
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffffbc58bede
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffffbc58be21
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [mysqld:121583]
pcc_cpufreq snd_timer snd soundcore i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm crc32c_intel libnvdimm ata_piix serio_raw mptspi scsi_transport_spi mptscsih libata e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 121583 Comm: mysqld Kdump: loaded Not tainted 3.10.0-1062.4.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff9f8b8a5cd230 ti: ffff9f8b91e04000 task.ti: ffff9f8b91e04000
RIP: 0010:[<ffffffffbbf14af6>]  [<ffffffffbbf14af6>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff9f8b91e07be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9f8b91e07bb0 RCX: 0000000000000003
RDX: ffffffffbc613a50 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9f8b91e07c30 R08: ffffffffbc613a48 R09: 00003ffffffff000
R10: ffff9f8cea97bcb0 R11: 0000000000000202 R12: ffff9f8b8a5cd230
R13: 0000000000000002 R14: 0000000000000030 R15: 0000000000000005
FS:  00007faaaa3b4700(0000) GS:ffff9f8cf86c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007faa780c4000 CR3: 00000001733ce000 CR4: 00000000000007e0
Call Trace:
[<ffffffffbbe7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffbbe7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffbbe7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffbbf14c0f>] smp_call_function_single+0x5f/0xa0
[<ffffffffbc17fce5>] ? cpumask_next_and+0x35/0x50
[<ffffffffbbf151bb>] smp_call_function_many+0x22b/0x270
[<ffffffffbbe7d478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbbe7d4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbbfea756>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbbfec21f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbbfec343>] tlb_finish_mmu+0x23/0x30
[<ffffffffbbfef04a>] zap_page_range+0xea/0x130
[<ffffffffbbfe9d5d>] SyS_madvise+0x49d/0xac0
[<ffffffffbc58be21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffbc58be15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffbc58bede>] system_call_fastpath+0x25/0x2a
[<ffffffffbc58be21>] ? system_call_after_swapgs+0xae/0x146
Code: 14 c5 e0 d6 b4 bc 48 89 df e8 77 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f