Info

Permalink
Function
generic_exec_single
First occurrence
2020-10-04
Last occurrence
2025-06-24
State
NEW
Components
kernel
Quality
-39

Statistics

Operating system Count
CentOS 7 1,648
Architecture Count
x86_64 1,648
Related packages Count
kernel 1,647
 0:3.10.0-1160.15.2.el7 359
 0:3.10.0-1160.41.1.el7 254
 0:3.10.0-1160.62.1.el7 228
 0:3.10.0-1160.el7 148
 0:3.10.0-1127.8.2.el7 85
 0:3.10.0-957.el7 78
 0:3.10.0-1160.88.1.el7 67
 0:3.10.0-1160.114.2.el7 65
 0:3.10.0-1160.76.1.el7 32
 0:3.10.0-1062.12.1.el7 28
 0:3.10.0-1160.25.1.el7 24
 0:3.10.0-1160.66.1.el7 22
 0:3.10.0-1160.49.1.el7 21
 0:3.10.0-1160.71.1.el7 20
 0:3.10.0-693.el7 18
 0:3.10.0-1160.80.1.el7 18
 0:3.10.0-1160.90.1.el7 16
 0:3.10.0-862.el7 15
 0:3.10.0-1160.119.1.el7 13
 0:3.10.0-1160.31.1.el7 12
 0:3.10.0-1127.10.1.el7 10
 0:3.10.0-1160.59.1.el7 8
 0:3.10.0-1160.45.1.el7 8
 0:3.10.0-1127.el7 7
 0:3.10.0-1160.108.1.el7 6
 0:3.10.0-862.14.4.el7 6
 0:3.10.0-693.11.1.el7 5
 0:3.10.0-1062.9.1.el7 5
 0:3.10.0-1127.19.1.el7 5
 0:3.10.0-1160.53.1.el7 5
 0:3.10.0-1160.92.1.el7 5
 0:3.10.0-957.12.1.el7 4
 0:3.10.0-957.27.2.el7 4
 0:3.10.0-1062.el7 4
 0:3.10.0-1160.81.1.el7 4
 0:3.10.0-1062.18.1.el7 4
 0:3.10.0-1160.11.1.el7 4
 0:3.10.0-693.21.1.el7 4
 0:3.10.0-957.21.3.el7 3
 0:3.10.0-1160.105.1.el7 3
 0:3.10.0-1160.118.1.el7 3
 0:3.10.0-957.12.2.el7 2
 0:3.10.0-1062.4.1.el7 2
 0:3.10.0-1160.102.1.el7 2
 0:3.10.0-1160.6.1.el7 2
 0:3.10.0-1160.99.1.el7 2
 0:3.10.0-1160.42.2.el7 2
 0:3.10.0-1127.13.1.el7 1
 0:3.10.0-1160.24.1.el7 1
 0:3.10.0-1160.36.2.el7 1
 0:3.10.0-957.5.1.el7 1
 0:3.10.0-862.11.6.el7 1
kernel-plus 1
 0:3.10.0-693.11.1.el7.centos.plus 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #724413
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f933a
Build id: 3.10.0-693.11.1.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x70
5
smp_call_function_single
vmlinux 0xffffffff810f943f
Build id: 3.10.0-693.11.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff81322645
Build id: 3.10.0-693.11.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99d3
Build id: 3.10.0-693.11.1.el7.x86_64 		0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.11.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.11.1.el7.x86_64 		0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811acfc3
Build id: 3.10.0-693.11.1.el7.x86_64 		0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae2e5
Build id: 3.10.0-693.11.1.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffff811b53b4
Build id: 3.10.0-693.11.1.el7.x86_64 		0xf4
13
? lockref_put_or_lock
vmlinux 0xffffffff81331e98
Build id: 3.10.0-693.11.1.el7.x86_64 		0x48
14
? __vma_rb_erase
vmlinux 0xffffffff811b5971
Build id: 3.10.0-693.11.1.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffff811b7972
Build id: 3.10.0-693.11.1.el7.x86_64 		0x292
16
vm_munmap
vmlinux 0xffffffff811b7bb5
Build id: 3.10.0-693.11.1.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffff811b8cc2
Build id: 3.10.0-693.11.1.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffff816b5089
Build id: 3.10.0-693.11.1.el7.x86_64 		0x16 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [nbdisco:4510]
Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev vmw_vmci sg pcspkr i2c_piix4 shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt ahci fb_sys_fops ttm libahci drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel libata serio_raw vmw_pvscsi vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 4510 Comm: nbdisco Tainted: G        W    L ------------   3.10.0-693.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff88081c84af70 ti: ffff8807f1f40000 task.ti: ffff8807f1f40000
RIP: 0010:[<ffffffff810f933a>]  [<ffffffff810f933a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8807f1f43c60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8807f1f43c30 RCX: 0000000000000001
RDX: ffffffff816ce470 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8807f1f43cb0 R08: ffffffff816ce468 R09: ffffffff811bd7cc
R10: ffff88081d759bc0 R11: ffffea00200eb680 R12: ffffffff816b03a5
R13: ffff8807f1f43bf8 R14: ffffea0020654d00 R15: 0000000000000000
FS:  00007f3fc0ed3700(0000) GS:ffff88081d740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3fdf020000 CR3: 00000007f876f000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff8807f1f43d38
0000000000000002 000000008f066a05 000000008f066a05 0000000000000000
ffffffff8106e4a0 ffffffff8106e4a0 ffff8807f1f43ce0 ffffffff810f943f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f943f>] smp_call_function_single+0x5f/0xa0
[<ffffffff81322645>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99d3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811acfc3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae2e5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b53b4>] unmap_region+0xf4/0x140
[<ffffffff81331e98>] ? lockref_put_or_lock+0x48/0x80
[<ffffffff811b5971>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7972>] do_munmap+0x292/0x470
[<ffffffff811b7bb5>] vm_munmap+0x65/0xb0
[<ffffffff811b8cc2>] SyS_munmap+0x22/0x30
[<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 ba d9 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #287030
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff94d16802
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff94c7e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff94c7e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff94c7e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff94d1691f
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff94f85fe5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff94d16ecb
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff94c7e868
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff94c7e8db
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff94dedee6
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff94def9af
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff94defad3
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffff94df92c4
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xf4
14
? remove_wait_queue
vmlinux 0xffffffff94cc6b50
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x30
15
do_munmap
vmlinux 0xffffffff94dfb8b5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x2a5
16
vm_munmap
vmlinux 0xffffffff94dfbaf5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffff94dfcd92
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffff95393f92
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff95393ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [f2b/f.proftpd:1448]
Modules linked in: nfsv3 nfs_acl ppp_deflate ppp_async crc_ccitt ppp_generic slhc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod pcspkr sg virtio_balloon joydev i2c_piix4 ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_common
ata_generic pata_acpi virtio_net net_failover failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata e1000 virtio_pci serio_raw virtio_ring virtio drm_panel_orientation_quirks floppy
CPU: 1 PID: 1448 Comm: f2b/f.proftpd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org 04/01/2014
task: ffffa0b4f7fc4200 ti: ffffa0b4fd394000 task.ti: ffffa0b4fd394000
RIP: 0010:[<ffffffff94d16802>]  [<ffffffff94d16802>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffffa0b4fd397c20  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffffa0b4fd397bf0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffffa0b4fd397c70 R08: ffffffff954137c0 R09: 000000000001fca0
R10: ffffa0b4fffb7800 R11: 0000000000000000 R12: ffffa0b4fd397b90
R13: ffffa0b400000000 R14: ffffa0b4fffb7800 R15: ffffffff94dc8c76
FS:  00007fe8a1d37700(0000) GS:ffffa0b4ffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6a74923000 CR3: 000000007761c000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff94c7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff94c7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff94c7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff94d1691f>] smp_call_function_single+0x5f/0xa0
[<ffffffff94f85fe5>] ? cpumask_next_and+0x35/0x50
[<ffffffff94d16ecb>] smp_call_function_many+0x22b/0x270
[<ffffffff94c7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff94c7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff94dedee6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff94def9af>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff94defad3>] tlb_finish_mmu+0x23/0x30
[<ffffffff94df92c4>] unmap_region+0xf4/0x140
[<ffffffff94cc6b50>] ? remove_wait_queue+0x30/0x40
[<ffffffff94dfb8b5>] do_munmap+0x2a5/0x480
[<ffffffff94dfbaf5>] vm_munmap+0x65/0xb0
[<ffffffff94dfcd92>] SyS_munmap+0x22/0x30
[<ffffffff95393f92>] system_call_fastpath+0x25/0x2a
[<ffffffff95393ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 13 95 95 48 89 df e8 57 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #273602
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9fd163d6
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffff9fc7e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9fc7e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9fc7e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff9fd164ef
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9ff81685
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9fd16a9b
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9fc7e478
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9fc7e4eb
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9fdec096
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9fdedb5f
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9fdedc83
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffff9fdf7474
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xf4
14
? list_del
vmlinux 0xffffffff9ffa133d
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xd
15
? remove_wait_queue
vmlinux 0xffffffff9fcc6ed1
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x31
16
? __vma_rb_erase
vmlinux 0xffffffff9fdf7a31
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x121
17
do_munmap
vmlinux 0xffffffff9fdf9a75
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x2a5
18
vm_munmap
vmlinux 0xffffffff9fdf9cb5
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x65
19
SyS_munmap
vmlinux 0xffffffff9fdfaf52
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x22
20
system_call_fastpath
vmlinux 0xffffffffa038dede
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x25
21
? system_call_after_swapgs
vmlinux 0xffffffffa038de21
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sdsvrd:134785]
Modules linked in: binfmt_misc xt_multiport nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev virtio_balloon pcspkr sg joydev parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk
virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 134785 Comm: sdsvrd Kdump: loaded Tainted: G             L ------------   3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9db3993c0000 ti: ffff9db399ad4000 task.ti: ffff9db399ad4000
RIP: 0010:[<ffffffff9fd163d6>]  [<ffffffff9fd163d6>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff9db399ad7c20  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff9db399ad7bf0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff9db399ad7c70 R08: ffffffffa0413cc8 R09: 000000018040003e
R10: 0000000030f32701 R11: ffff9db030f32bc0 R12: ffffffff9ff8d2d3
R13: ffff9db399ad7c28 R14: ffff9db399ad7d40 R15: ffff9db399ad7d33
FS:  000000004061b940(0063) GS:ffff9db3ace40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002b50cf5dcbd0 CR3: 0000000375ebe000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9fc7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9fc7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9fc7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9fd164ef>] smp_call_function_single+0x5f/0xa0
[<ffffffff9ff81685>] ? cpumask_next_and+0x35/0x50
[<ffffffff9fd16a9b>] smp_call_function_many+0x22b/0x270
[<ffffffff9fc7e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9fc7e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9fdec096>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9fdedb5f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9fdedc83>] tlb_finish_mmu+0x23/0x30
[<ffffffff9fdf7474>] unmap_region+0xf4/0x140
[<ffffffff9ffa133d>] ? list_del+0xd/0x30
[<ffffffff9fcc6ed1>] ? remove_wait_queue+0x31/0x40
[<ffffffff9fdf7a31>] ? __vma_rb_erase+0x121/0x220
[<ffffffff9fdf9a75>] do_munmap+0x2a5/0x480
[<ffffffff9fdf9cb5>] vm_munmap+0x65/0xb0
[<ffffffff9fdfaf52>] SyS_munmap+0x22/0x30
[<ffffffffa038dede>] system_call_fastpath+0x25/0x2a
[<ffffffffa038de21>] ? system_call_after_swapgs+0xae/0x146
Code: 14 c5 20 e8 94 a0 48 89 df e8 37 23 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #8839
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f932a
Build id: 3.10.0-693.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64 		0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64 		0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64 		0x70
5
smp_call_function_single
vmlinux 0xffffffff810f942f
Build id: 3.10.0-693.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813226f5
Build id: 3.10.0-693.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99c3
Build id: 3.10.0-693.el7.x86_64 		0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64 		0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64 		0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffff811b5444
Build id: 3.10.0-693.el7.x86_64 		0xf4
13
? lockref_put_or_lock
vmlinux 0xffffffff81331f48
Build id: 3.10.0-693.el7.x86_64 		0x48
14
? __vma_rb_erase
vmlinux 0xffffffff811b5a01
Build id: 3.10.0-693.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffff811b7a02
Build id: 3.10.0-693.el7.x86_64 		0x292
16
vm_munmap
vmlinux 0xffffffff811b7c45
Build id: 3.10.0-693.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffff811b8d52
Build id: 3.10.0-693.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64 		0x16 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [java:88037]
Modules linked in: xt_nat veth xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event edac_core snd_ens1371 iosf_mbi crc32_pclmul snd_rawmidi ghash_clmulni_intel snd_ac97_codec ppdev ac97_bus aesni_intel snd_seq lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm vmw_balloon joydev sg snd_timer pcspkr vmw_vmci snd soundcore i2c_piix4 nfit shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic
sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix crct10dif_pclmul mptscsih crct10dif_common libata crc32c_intel mptbase e1000 serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 88037 Comm: java Tainted: G               ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8801f4fb5ee0 ti: ffff88025c7b4000 task.ti: ffff88025c7b4000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff88025c7b7c60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88025c7b7c30 RCX: 0000000000000001
RDX: ffffffff816ce430 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff88025c7b7cb0 R08: ffffffff816ce428 R09: ffffffff811bd768
R10: ffff88042d799bc0 R11: ffffea000bd4ff00 R12: ffffffff816b02e5
R13: ffff88025c7b7bf8 R14: ffffea00107c4200 R15: 0000000000000000
FS:  00007f423c5ef700(0000) GS:ffff88042d780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f42a0615000 CR3: 00000001d0a82000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff88025c7b7d38
0000000000000003 000000009c791199 ffff88025c7b7ca8 0000000000000000
ffffffff8106e4a0 ffffffff8106e4a0 ffff88025c7b7ce0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b5444>] unmap_region+0xf4/0x140
[<ffffffff81331f48>] ? lockref_put_or_lock+0x48/0x80
[<ffffffff811b5a01>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7a02>] do_munmap+0x292/0x470
[<ffffffff811b7c45>] vm_munmap+0x65/0xb0
[<ffffffff811b8d52>] SyS_munmap+0x22/0x30
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1175902
Frame # Function Binary Source or offset Line
1
? leave_mm
vmlinux 0xffffffff9487e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
2
? leave_mm
vmlinux 0xffffffff9487e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
3
smp_call_function_single
vmlinux 0xffffffff9491667f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
4
? cpumask_next_and
vmlinux 0xffffffff94b85de5
Build id: 3.10.0-1160.el7.x86_64 		0x35
5
smp_call_function_many
vmlinux 0xffffffff94916c2b
Build id: 3.10.0-1160.el7.x86_64 		0x22b
6
native_flush_tlb_others
vmlinux 0xffffffff9487e868
Build id: 3.10.0-1160.el7.x86_64 		0xb8
7
flush_tlb_mm_range
vmlinux 0xffffffff9487e8db
Build id: 3.10.0-1160.el7.x86_64 		0x6b
8
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff949edd06
Build id: 3.10.0-1160.el7.x86_64 		0xa6
9
arch_tlb_finish_mmu
vmlinux 0xffffffff949ef7cf
Build id: 3.10.0-1160.el7.x86_64 		0x3f
10
tlb_finish_mmu
vmlinux 0xffffffff949ef8f3
Build id: 3.10.0-1160.el7.x86_64 		0x23
11
unmap_region
vmlinux 0xffffffff949f90e4
Build id: 3.10.0-1160.el7.x86_64 		0xf4
12
? d_free
vmlinux 0xffffffff94a665a0
Build id: 3.10.0-1160.el7.x86_64 		0x50
13
? __vma_rb_erase
vmlinux 0xffffffff949f96a1
Build id: 3.10.0-1160.el7.x86_64 		0x121
14
do_munmap
vmlinux 0xffffffff949fb6d5
Build id: 3.10.0-1160.el7.x86_64 		0x2a5
15
vm_munmap
vmlinux 0xffffffff949fb915
Build id: 3.10.0-1160.el7.x86_64 		0x65
16
SyS_munmap
vmlinux 0xffffffff949fcbb2
Build id: 3.10.0-1160.el7.x86_64 		0x22
17
system_call_fastpath
vmlinux 0xffffffff94f93f92
Build id: 3.10.0-1160.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [gsd-housekeepin:3581]
[<ffffffff9487e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9487e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9491667f>] smp_call_function_single+0x5f/0xa0
[<ffffffff94b85de5>] ? cpumask_next_and+0x35/0x50
[<ffffffff94916c2b>] smp_call_function_many+0x22b/0x270
[<ffffffff9487e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9487e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff949edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff949ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff949ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffff949f90e4>] unmap_region+0xf4/0x140
[<ffffffff94a665a0>] ? d_free+0x50/0x70
[<ffffffff949f96a1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff949fb6d5>] do_munmap+0x2a5/0x480
[<ffffffff949fb915>] vm_munmap+0x65/0xb0
[<ffffffff949fcbb2>] SyS_munmap+0x22/0x30
[<ffffffff94f93f92>] system_call_fastpath+0x25/0x2a
Complete report #1093422
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff99b0779a
Build id: 3.10.0-862.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff99a76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff99a76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff99a76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff99b078af
Build id: 3.10.0-862.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff99d4a275
Build id: 3.10.0-862.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff99b07e5b
Build id: 3.10.0-862.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff99a77018
Build id: 3.10.0-862.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff99a77088
Build id: 3.10.0-862.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffff99bc0497
Build id: 3.10.0-862.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff99bc1c75
Build id: 3.10.0-862.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffff99bca624
Build id: 3.10.0-862.el7.x86_64 		0xf4
13
? d_free
vmlinux 0xffffffff99c32c0f
Build id: 3.10.0-862.el7.x86_64 		0x4f
14
? __vma_rb_erase
vmlinux 0xffffffff99bcabe1
Build id: 3.10.0-862.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffff99bccbf5
Build id: 3.10.0-862.el7.x86_64 		0x295
16
vm_munmap
vmlinux 0xffffffff99bcce35
Build id: 3.10.0-862.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffff99bce052
Build id: 3.10.0-862.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffff9a11f7d5
Build id: 3.10.0-862.el7.x86_64 		0x1c
19
? system_call_after_swapgs
vmlinux 0xffffffff9a11f721
Build id: 3.10.0-862.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [java:28580]
Modules linked in: tcp_diag inet_diag binfmt_misc nfnetlink_queue nfnetlink syshook_linux(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack sunrpc ext4 mbcache jbd2 sg ppdev joydev pcspkr virtio_balloon parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_net virtio_console virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci i2c_core virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmsecmod]
CPU: 3 PID: 28580 Comm: java Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff88a7a0f18fd0 ti: ffff88a755374000 task.ti: ffff88a755374000
RIP: 0010:[<ffffffff99b0779a>]  [<ffffffff99b0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff88a755377c30  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff88a755377c00 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000286
RBP: ffff88a755377c78 R08: ffffffff9a2133e8 R09: ffffffff99bd35bc
R10: ffff88a7acedba40 R11: ffffd51e0ee96dc0 R12: 00000000003ffffc
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff99bc556d
FS:  00002b1e56007700(0000) GS:ffff88a7acec0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002b1e07fb7000 CR3: 000000040ba86000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff99a76e50>] ? leave_mm+0x110/0x110
[<ffffffff99a76e50>] ? leave_mm+0x110/0x110
[<ffffffff99a76e50>] ? leave_mm+0x110/0x110
[<ffffffff99b078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff99d4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff99b07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff99a77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff99a77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff99bc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff99bc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff99bca624>] unmap_region+0xf4/0x140
[<ffffffff99c32c0f>] ? d_free+0x4f/0x70
[<ffffffff99bcabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff99bccbf5>] do_munmap+0x295/0x470
[<ffffffff99bcce35>] vm_munmap+0x65/0xb0
[<ffffffff99bce052>] SyS_munmap+0x22/0x30
[<ffffffff9a11f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff9a11f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 73 9a 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #654684
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb671db12
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb6681b10
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x130
3
? leave_mm
vmlinux 0xffffffffb6681b10
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x130
4
? leave_mm
vmlinux 0xffffffffb6681b10
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x130
5
smp_call_function_single
vmlinux 0xffffffffb671dc2f
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb699c485
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb671e1eb
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb6681fa8
Build id: 3.10.0-1160.88.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb668201b
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb67fa00e
Build id: 3.10.0-1160.88.1.el7.x86_64 		0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb67fbb8f
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb67fbcc3
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffffb6805844
Build id: 3.10.0-1160.88.1.el7.x86_64 		0xf4
14
? d_free
vmlinux 0xffffffffb6874cb0
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x50
15
? __vma_rb_erase
vmlinux 0xffffffffb6805e51
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x121
16
do_munmap
vmlinux 0xffffffffb6807ea5
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x2a5
17
vm_munmap
vmlinux 0xffffffffb68080e5
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x65
18
SyS_munmap
vmlinux 0xffffffffb6809412
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x22
19
system_call_fastpath
vmlinux 0xffffffffb6dc539a
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [vmtoolsd:784]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink vmnet(OE) vmmon(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi ppdev snd_ac97_codec ac97_bus snd_seq snd_seq_device aesni_intel vmw_balloon lrw snd_pcm gf128mul glue_helper
ablk_helper cryptd snd_timer snd joydev pcspkr sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper nfit libnvdimm syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel scsi_transport_spi mptscsih libata serio_raw mptbase drm_panel_orientation_quirks e1000 dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: hinic]
CPU: 2 PID: 784 Comm: vmtoolsd Kdump: loaded Tainted: GF          OE  ------------   3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9305f6903180 ti: ffff9305f6b24000 task.ti: ffff9305f6b24000
RIP: 0010:[<ffffffffb671db12>]  [<ffffffffb671db12>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9305f6b27c20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9305f6b27bf0 RCX: 0000000000000001
RDX: ffffffffb6e138d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9305f6b27c70 R08: ffffffffb6e138c8 R09: ffffffffb680efa5
R10: ffff93067969f0a0 R11: fffff18c04d10340 R12: 00000000000280da
R13: ffffffffb684bea2 R14: ffff9305f6b27bd8 R15: 0000000000000000
FS:  00007f6878d1e780(0000) GS:ffff930679680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6878d32000 CR3: 00000001340ae000 CR4: 0000000000760fe0
PKRU: 55555554
Call Trace:
[<ffffffffb6681b10>] ? leave_mm+0x130/0x130
[<ffffffffb6681b10>] ? leave_mm+0x130/0x130
[<ffffffffb6681b10>] ? leave_mm+0x130/0x130
[<ffffffffb671dc2f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb699c485>] ? cpumask_next_and+0x35/0x50
[<ffffffffb671e1eb>] smp_call_function_many+0x22b/0x270
[<ffffffffb6681fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb668201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb67fa00e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffb67fbb8f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb67fbcc3>] tlb_finish_mmu+0x23/0x40
[<ffffffffb6805844>] unmap_region+0xf4/0x150
[<ffffffffb6874cb0>] ? d_free+0x50/0x80
[<ffffffffb6805e51>] ? __vma_rb_erase+0x121/0x230
[<ffffffffb6807ea5>] do_munmap+0x2a5/0x480
[<ffffffffb68080e5>] vm_munmap+0x65/0xb0
[<ffffffffb6809412>] SyS_munmap+0x22/0x30
[<ffffffffb6dc539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 35 b7 48 89 df e8 c7 66 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #462994
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffabd0a9fa
Build id: 3.10.0-862.11.6.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffabc79c90
Build id: 3.10.0-862.11.6.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffabc79c90
Build id: 3.10.0-862.11.6.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffabc79c90
Build id: 3.10.0-862.11.6.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffabd0ab0f
Build id: 3.10.0-862.11.6.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffabf4ec95
Build id: 3.10.0-862.11.6.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffabd0b0bb
Build id: 3.10.0-862.11.6.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffabc79e58
Build id: 3.10.0-862.11.6.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffabc79ec8
Build id: 3.10.0-862.11.6.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffabdc33c7
Build id: 3.10.0-862.11.6.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffabdc4cc5
Build id: 3.10.0-862.11.6.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffffabdcdab4
Build id: 3.10.0-862.11.6.el7.x86_64 		0xf4
13
? d_free
vmlinux 0xffffffffabe3705f
Build id: 3.10.0-862.11.6.el7.x86_64 		0x4f
14
? __vma_rb_erase
vmlinux 0xffffffffabdce071
Build id: 3.10.0-862.11.6.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffffabdd0085
Build id: 3.10.0-862.11.6.el7.x86_64 		0x295
16
vm_munmap
vmlinux 0xffffffffabdd02c5
Build id: 3.10.0-862.11.6.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffffabdd14e2
Build id: 3.10.0-862.11.6.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffffac32579b
Build id: 3.10.0-862.11.6.el7.x86_64 		0x22 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [sqlservr:28394]
Modules linked in: event_collector_2_4_933667(OE) veth binfmt_misc xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_1_9_8557(POE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd sg pcspkr ppdev vmw_balloon joydev nfit libnvdimm parport_pc parport shpchp i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel i2c_core serio_raw vmxnet3 vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 28394 Comm: sqlservr Kdump: loaded Tainted: P           OEL ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8ae847aa0fd0 ti: ffff8ae89039c000 task.ti: ffff8ae89039c000
RIP: 0010:[<ffffffffabd0a9fa>]  [<ffffffffabd0a9fa>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8ae89039fc30  EFLAGS: 00000202
RAX: 000000000000000a RBX: ffff8ae89039fc00 RCX: 0000000000000036
RDX: 00000000000003ff RSI: 000000000000000a RDI: 0000000000000286
RBP: ffff8ae89039fc78 R08: ffffffffac413aa0 R09: ffffffffabdd6d9c
R10: ffff8add8d91ba80 R11: ffffd132e2b4ea00 R12: ffffd132ebdba730
R13: ffff8ad828f77008 R14: ffffffffabdd57b8 R15: ffff8ae89039fbb0
FS:  00007f8511294700(0000) GS:ffff8add8d900000(0000) knlGS:00003fff83e9a000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f85113a5000 CR3: 0000000c39386000 CR4: 00000000003607e0
Call Trace:
[<ffffffffabc79c90>] ? leave_mm+0x110/0x110
[<ffffffffabc79c90>] ? leave_mm+0x110/0x110
[<ffffffffabc79c90>] ? leave_mm+0x110/0x110
[<ffffffffabd0ab0f>] smp_call_function_single+0x5f/0xa0
[<ffffffffabf4ec95>] ? cpumask_next_and+0x35/0x50
[<ffffffffabd0b0bb>] smp_call_function_many+0x22b/0x270
[<ffffffffabc79e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffabc79ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffffabdc33c7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffabdc4cc5>] tlb_finish_mmu+0x55/0x60
[<ffffffffabdcdab4>] unmap_region+0xf4/0x140
[<ffffffffabe3705f>] ? d_free+0x4f/0x70
[<ffffffffabdce071>] ? __vma_rb_erase+0x121/0x220
[<ffffffffabdd0085>] do_munmap+0x295/0x470
[<ffffffffabdd02c5>] vm_munmap+0x65/0xb0
[<ffffffffabdd14e2>] SyS_munmap+0x22/0x30
[<ffffffffac32579b>] system_call_fastpath+0x22/0x27
Code: c0 96 01 00 48 89 de 48 03 14 c5 60 54 93 ac 48 89 df e8 7a a0 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #372505
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810fcf5a
Build id: 3.10.0-693.21.1.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff810fd06f
Build id: 3.10.0-693.21.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813288a5
Build id: 3.10.0-693.21.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff810fd61b
Build id: 3.10.0-693.21.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff81070b98
Build id: 3.10.0-693.21.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff81070c68
Build id: 3.10.0-693.21.1.el7.x86_64 		0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811b18c3
Build id: 3.10.0-693.21.1.el7.x86_64 		0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811b2be5
Build id: 3.10.0-693.21.1.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffff811b9cb4
Build id: 3.10.0-693.21.1.el7.x86_64 		0xf4
13
? evict
vmlinux 0xffffffff812224b8
Build id: 3.10.0-693.21.1.el7.x86_64 		0x108
14
? call_rcu_sched
vmlinux 0xffffffff8113d14d
Build id: 3.10.0-693.21.1.el7.x86_64 		0x1d
15
? d_free
vmlinux 0xffffffff8121ce9f
Build id: 3.10.0-693.21.1.el7.x86_64 		0x4f
16
? __vma_rb_erase
vmlinux 0xffffffff811ba271
Build id: 3.10.0-693.21.1.el7.x86_64 		0x121
17
do_munmap
vmlinux 0xffffffff811bc285
Build id: 3.10.0-693.21.1.el7.x86_64 		0x295
18
vm_munmap
vmlinux 0xffffffff811bc4c5
Build id: 3.10.0-693.21.1.el7.x86_64 		0x65
19
SyS_munmap
vmlinux 0xffffffff811bd5d2
Build id: 3.10.0-693.21.1.el7.x86_64 		0x22
20
system_call_fastpath
vmlinux 0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64 		0x1c 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [vmtoolsd:821]
Modules linked in: nls_utf8 isofs fuse ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac edac_core coretemp snd_ens1371 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw snd_rawmidi gf128mul snd_ac97_codec glue_helper ablk_helper cryptd ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd ppdev pcspkr vmw_balloon soundcore joydev sg vmw_vmci shpchp i2c_piix4 nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm crct10dif_pclmul drm crct10dif_common crc32c_intel ata_piix mptspi libata scsi_transport_spi serio_raw mptscsih e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 821 Comm: vmtoolsd Not tainted 3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8804274cbf40 ti: ffff880422258000 task.ti: ffff880422258000
RIP: 0010:[<ffffffff810fcf5a>]  [<ffffffff810fcf5a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff88042225bc30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88042225bbc0 RCX: 0000000000000004
RDX: ffffffff816dd070 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff88042225bc80 R08: ffffffff816dd068 R09: ffffffff811c20dc
R10: ffff88042d75b940 R11: ffffea001079ff00 R12: 0000000000000010
R13: ffffffff8105b516 R14: ffffffffffffff02 R15: 0000000000000286
FS:  00007f4db7db5880(0000) GS:ffff88042d740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4db7dd4000 CR3: 0000000421a56000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810fd06f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813288a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810fd61b>] smp_call_function_many+0x22b/0x270
[<ffffffff81070b98>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81070c68>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811b18c3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b2be5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b9cb4>] unmap_region+0xf4/0x140
[<ffffffff812224b8>] ? evict+0x108/0x180
[<ffffffff8113d14d>] ? call_rcu_sched+0x1d/0x20
[<ffffffff8121ce9f>] ? d_free+0x4f/0x70
[<ffffffff811ba271>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811bc285>] do_munmap+0x295/0x470
[<ffffffff811bc4c5>] vm_munmap+0x65/0xb0
[<ffffffff811bd5d2>] SyS_munmap+0x22/0x30
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 80 95 01 00 48 89 de 48 03 14 c5 a0 10 b2 81 48 89 df e8 7a 03 24 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #371313
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb8716a32
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb867ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb867ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb867ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb8716b4f
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb8986a25
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb87170fb
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb867e258
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb867e2cb
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb87ee676
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb87f013f
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb87f0263
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffffb87f9a54
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xf4
14
? d_free
vmlinux 0xffffffffb8866d70
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x50
15
? __vma_rb_erase
vmlinux 0xffffffffb87fa011
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x121
16
do_munmap
vmlinux 0xffffffffb87fc045
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x2a5
17
vm_munmap
vmlinux 0xffffffffb87fc285
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x65
18
SyS_munmap
vmlinux 0xffffffffb87fd582
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x22
19
system_call_fastpath
vmlinux 0xffffffffb8d95f92
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [vmtoolsd:624]
Modules linked in: lp veeamsnap(OE) tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 iosf_mbi crc32_pclmul snd_rawmidi snd_ac97_codec ghash_clmulni_intel aesni_intel ac97_bus lrw snd_seq gf128mul glue_helper ablk_helper snd_seq_device snd_pcm
ppdev cryptd pcspkr snd_timer joydev snd soundcore vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx nfit libnvdimm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix serio_raw mptspi scsi_transport_spi libata e1000 mptscsih mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 624 Comm: vmtoolsd Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff944a75d0e300 ti: ffff944bf6a14000 task.ti: ffff944bf6a14000
RIP: 0010:[<ffffffffb8716a32>]  [<ffffffffb8716a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff944bf6a17c20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff944bf6a17bf0 RCX: 0000000000000004
RDX: ffffffffb8e13f50 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff944bf6a17c70 R08: ffffffffb8e13f48 R09: ffffffffb880307c
R10: ffff944bf761f0a0 R11: ffffd9e386d98e80 R12: 00000000000280da
R13: ffffffffb883ed12 R14: ffff944bf6a17bd8 R15: 0000000000000000
FS:  00007f7ec2ecc780(0000) GS:ffff944bf7600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ec2ee1000 CR3: 00000001b6a26000 CR4: 00000000003607f0
Call Trace:
[<ffffffffb867ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb867ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb867ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb8716b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb8986a25>] ? cpumask_next_and+0x35/0x50
[<ffffffffb87170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffb867e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb867e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb87ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb87f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb87f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffb87f9a54>] unmap_region+0xf4/0x140
[<ffffffffb8866d70>] ? d_free+0x50/0x70
[<ffffffffb87fa011>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb87fc045>] do_munmap+0x2a5/0x480
[<ffffffffb87fc285>] vm_munmap+0x65/0xb0
[<ffffffffb87fd582>] SyS_munmap+0x22/0x30
[<ffffffffb8d95f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 60 15 35 b9 48 89 df e8 67 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #313337
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffbb316a36
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffbb27ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffbb27ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffbb27ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffbb316b4f
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffbb586a45
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffbb3170fb
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffbb27e258
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffbb27e2cb
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffbb3ee676
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffbb3f013f
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffbb3f0263
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffffbb3f9a54
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xf4
14
? d_free
vmlinux 0xffffffffbb466d90
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x50
15
? __vma_rb_erase
vmlinux 0xffffffffbb3fa011
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x121
16
do_munmap
vmlinux 0xffffffffbb3fc045
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x2a5
17
vm_munmap
vmlinux 0xffffffffbb3fc285
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x65
18
SyS_munmap
vmlinux 0xffffffffbb3fd582
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x22
19
system_call_fastpath
vmlinux 0xffffffffbb995f92
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x25
20
? system_call_after_swapgs
vmlinux 0xffffffffbb995ed5
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [vmtoolsd:876]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6_tables iptable_filter falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_12904(E) vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul vmw_balloon glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit ata_piix libnvdimm crct10dif_pclmul crct10dif_common mptspi crc32c_intel libata e1000 scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 876 Comm: vmtoolsd Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff98a95d10c200 ti: ffff98a95e574000 task.ti: ffff98a95e574000
RIP: 0010:[<ffffffffbb316a36>]  [<ffffffffbb316a36>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff98a95e577c20  EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffffd11bdc293b00 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff98a95e577c70 R08: 0000000000000001 R09: ffffffffbb40307c
R10: ffff98a97fcdf0a0 R11: ffffd11bdcaf19c0 R12: ffffffffbb3cd150
R13: 0000000000000000 R14: ffffffffbb3cda6c R15: ffff98a95e577bc8
FS:  00007fc288318780(0000) GS:ffff98a97fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc288331000 CR3: 000000081bf06000 CR4: 00000000000407e0
Call Trace:
[<ffffffffbb27ddd0>] ? leave_mm+0x120/0x120
[<ffffffffbb27ddd0>] ? leave_mm+0x120/0x120
[<ffffffffbb27ddd0>] ? leave_mm+0x120/0x120
[<ffffffffbb316b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffbb586a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffbb3170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffbb27e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbb27e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbb3ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbb3f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbb3f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffbb3f9a54>] unmap_region+0xf4/0x140
[<ffffffffbb466d90>] ? d_free+0x50/0x70
[<ffffffffbb3fa011>] ? __vma_rb_erase+0x121/0x220
[<ffffffffbb3fc045>] do_munmap+0x2a5/0x480
[<ffffffffbb3fc285>] vm_munmap+0x65/0xb0
[<ffffffffbb3fd582>] SyS_munmap+0x22/0x30
[<ffffffffbb995f92>] system_call_fastpath+0x25/0x2a
[<ffffffffbb995ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 14 c5 60 15 f5 bb 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #131284
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffaf70aa2a
Build id: 3.10.0-862.14.4.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffaf679c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffaf679c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffaf679c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffaf70ab3f
Build id: 3.10.0-862.14.4.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffaf94edb5
Build id: 3.10.0-862.14.4.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffaf70b0eb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaf679e58
Build id: 3.10.0-862.14.4.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffaf679ec8
Build id: 3.10.0-862.14.4.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffaf7c3487
Build id: 3.10.0-862.14.4.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffaf7c4d85
Build id: 3.10.0-862.14.4.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffffaf7cdb74
Build id: 3.10.0-862.14.4.el7.x86_64 		0xf4
13
? d_free
vmlinux 0xffffffffaf83711f
Build id: 3.10.0-862.14.4.el7.x86_64 		0x4f
14
? __vma_rb_erase
vmlinux 0xffffffffaf7ce131
Build id: 3.10.0-862.14.4.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffffaf7d0145
Build id: 3.10.0-862.14.4.el7.x86_64 		0x295
16
vm_munmap
vmlinux 0xffffffffaf7d0385
Build id: 3.10.0-862.14.4.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffffaf7d15a2
Build id: 3.10.0-862.14.4.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffffafd2579b
Build id: 3.10.0-862.14.4.el7.x86_64 		0x22 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [vmtoolsd:804]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter fuse vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac snd_ens1371 snd_rawmidi iosf_mbi crc32_pclmul snd_ac97_codec ghash_clmulni_intel aesni_intel ac97_bus snd_seq lrw gf128mul glue_helper ablk_helper snd_seq_device
cryptd snd_pcm ppdev snd_timer vmw_balloon snd soundcore joydev pcspkr vmw_vmci sg nfit i2c_piix4 libnvdimm parport_pc parport shpchp ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm e1000 ata_piix mptspi scsi_transport_spi libata mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 804 Comm: vmtoolsd Kdump: loaded Not tainted 3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff90123197cf10 ti: ffff9011b67dc000 task.ti: ffff9011b67dc000
RIP: 0010:[<ffffffffaf70aa2a>]  [<ffffffffaf70aa2a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9011b67dfc30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9011b67dfc00 RCX: 0000000000000001
RDX: ffffffffafe132b0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9011b67dfc78 R08: ffffffffafe132a8 R09: ffffffffaf7d6e5c
R10: ffff90123969ba80 R11: ffffe938c25b1980 R12: ffffe938c2d9f230
R13: ffff9012333ca870 R14: ffffffffaf7d5878 R15: ffff9011b67dfbb0
FS:  00007f9aadf4a880(0000) GS:ffff901239680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9aadf13000 CR3: 00000000b50a6000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaf679c90>] ? leave_mm+0x110/0x110
[<ffffffffaf679c90>] ? leave_mm+0x110/0x110
[<ffffffffaf679c90>] ? leave_mm+0x110/0x110
[<ffffffffaf70ab3f>] smp_call_function_single+0x5f/0xa0
[<ffffffffaf94edb5>] ? cpumask_next_and+0x35/0x50
[<ffffffffaf70b0eb>] smp_call_function_many+0x22b/0x270
[<ffffffffaf679e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaf679ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaf7c3487>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffaf7c4d85>] tlb_finish_mmu+0x55/0x60
[<ffffffffaf7cdb74>] unmap_region+0xf4/0x140
[<ffffffffaf83711f>] ? d_free+0x4f/0x70
[<ffffffffaf7ce131>] ? __vma_rb_erase+0x121/0x220
[<ffffffffaf7d0145>] do_munmap+0x295/0x470
[<ffffffffaf7d0385>] vm_munmap+0x65/0xb0
[<ffffffffaf7d15a2>] SyS_munmap+0x22/0x30
[<ffffffffafd2579b>] system_call_fastpath+0x22/0x27
Code: c0 96 01 00 48 89 de 48 03 14 c5 a0 54 33 b0 48 89 df e8 6a a1 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #105470
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8491109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff8487a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff8487a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff8487a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff849111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff84b75465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8491175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8487a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8487a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff849e4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff849e5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffff849ef014
Build id: 3.10.0-957.el7.x86_64 		0xf4
13
? d_free
vmlinux 0xffffffff84a593df
Build id: 3.10.0-957.el7.x86_64 		0x4f
14
? __vma_rb_erase
vmlinux 0xffffffff849ef5d1
Build id: 3.10.0-957.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffff849f1605
Build id: 3.10.0-957.el7.x86_64 		0x2a5
16
vm_munmap
vmlinux 0xffffffff849f1845
Build id: 3.10.0-957.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffff849f2ad2
Build id: 3.10.0-957.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffff84f74ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
19
? system_call_after_swapgs
vmlinux 0xffffffff84f74d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [vmtoolsd:9076]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper vmw_balloon cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt sr_mod fb_sys_fops crct10dif_pclmul cdrom mptspi crct10dif_common ttm ata_generic pata_acpi scsi_transport_spi ata_piix crc32c_intel mptscsih serio_raw
nfit drm libata e1000 drm_panel_orientation_quirks mptbase libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 9076 Comm: vmtoolsd Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff94997712d140 ti: ffff94996bf34000 task.ti: ffff94996bf34000
RIP: 0010:[<ffffffff8491109a>]  [<ffffffff8491109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff94996bf37c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff94996bf37c00 RCX: 0000000000000007
RDX: ffffffff85013c90 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff94996bf37c78 R08: ffffffff85013c88 R09: ffffffff849f838c
R10: ffff94997575f0a0 R11: fffffded06bc9980 R12: 00007fcadbeb8000
R13: fffffded08aff270 R14: ffff949957f4e5e8 R15: ffffffff849f6e08
FS:  00007fcadbec1880(0000) GS:ffff949975740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcadbeb8000 CR3: 00000000bad70000 CR4: 00000000000607e0
Call Trace:
[<ffffffff8487a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8487a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8487a3d0>] ? leave_mm+0x110/0x110
[<ffffffff849111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff84b75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff8491175b>] smp_call_function_many+0x22b/0x270
[<ffffffff8487a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8487a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff849e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff849e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff849ef014>] unmap_region+0xf4/0x140
[<ffffffff84a593df>] ? d_free+0x4f/0x70
[<ffffffff849ef5d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff849f1605>] do_munmap+0x2a5/0x480
[<ffffffff849f1845>] vm_munmap+0x65/0xb0
[<ffffffff849f2ad2>] SyS_munmap+0x22/0x30
[<ffffffff84f74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff84f74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 54 85 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #66067
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb19111ba
Build id: 3.10.0-957.5.1.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffb187a400
Build id: 3.10.0-957.5.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffb187a400
Build id: 3.10.0-957.5.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffb187a400
Build id: 3.10.0-957.5.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffb19112cf
Build id: 3.10.0-957.5.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb1b75a45
Build id: 3.10.0-957.5.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb191187b
Build id: 3.10.0-957.5.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb187a5c8
Build id: 3.10.0-957.5.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb187a638
Build id: 3.10.0-957.5.1.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffb19e4427
Build id: 3.10.0-957.5.1.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffb19e5d25
Build id: 3.10.0-957.5.1.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffffb19ef3c4
Build id: 3.10.0-957.5.1.el7.x86_64 		0xf4
13
? d_free
vmlinux 0xffffffffb1a5998f
Build id: 3.10.0-957.5.1.el7.x86_64 		0x4f
14
? __vma_rb_erase
vmlinux 0xffffffffb19ef981
Build id: 3.10.0-957.5.1.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffffb19f19b5
Build id: 3.10.0-957.5.1.el7.x86_64 		0x2a5
16
vm_munmap
vmlinux 0xffffffffb19f1bf5
Build id: 3.10.0-957.5.1.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffffb19f2e82
Build id: 3.10.0-957.5.1.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffffb1f74ddb
Build id: 3.10.0-957.5.1.el7.x86_64 		0x22 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [vmtoolsd:8845]
Modules linked in: vfat fat uas usb_storage cmac arc4 md4 nls_utf8 cifs ccm dns_resolver fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc nfsv3 nfs_acl ebtable_filter nfs ebtables lockd grace ip6table_filter fscache ip6_tables devlink iptable_filter mvfs(OE) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd sg soundcore nfit vmw_vmci i2c_piix4 libnvdimm parport_pc
parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper syscopyarea e1000 sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy
CPU: 5 PID: 8845 Comm: vmtoolsd Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9d28f085a080 ti: ffff9d28eb758000 task.ti: ffff9d28eb758000
RIP: 0010:[<ffffffffb19111ba>]  [<ffffffffb19111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9d28eb75bc30  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffffffffb19e97a8 RCX: 0000000000000001
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000282
RBP: ffff9d28eb75bc78 R08: ffffffffb2012dc8 R09: ffffffffb19f873c
R10: ffff9d28f655f0a0 R11: fffff77a1051d9c0 R12: 00007f779c49b000
R13: fffff77a18a4d930 R14: ffff9d28dfeddb00 R15: ffffffffb19f71b8
FS:  00007f779c4a4880(0000) GS:ffff9d28f6540000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f779c49b000 CR3: 000000062a6aa000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb187a400>] ? leave_mm+0x110/0x110
[<ffffffffb187a400>] ? leave_mm+0x110/0x110
[<ffffffffb187a400>] ? leave_mm+0x110/0x110
[<ffffffffb19112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffffb1b75a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffb191187b>] smp_call_function_many+0x22b/0x270
[<ffffffffb187a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb187a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb19e4427>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffb19e5d25>] tlb_finish_mmu+0x55/0x60
[<ffffffffb19ef3c4>] unmap_region+0xf4/0x140
[<ffffffffb1a5998f>] ? d_free+0x4f/0x70
[<ffffffffb19ef981>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb19f19b5>] do_munmap+0x2a5/0x480
[<ffffffffb19f1bf5>] vm_munmap+0x65/0xb0
[<ffffffffb19f2e82>] SyS_munmap+0x22/0x30
[<ffffffffb1f74ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 54 b2 48 89 df e8 da b6 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #20405
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa6f16f72
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa6e7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa6e7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa6e7e730
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa6f1708f
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa7185005
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa6f1763b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa6e7e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa6e7e96b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa6fede36
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa6fef8ff
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa6fefa23
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffffa6ff9234
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xf4
14
? d_free
vmlinux 0xffffffffa7065b80
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x50
15
? __vma_rb_erase
vmlinux 0xffffffffa6ff97f1
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x121
16
do_munmap
vmlinux 0xffffffffa6ffb825
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2a5
17
vm_munmap
vmlinux 0xffffffffa6ffba65
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x65
18
SyS_munmap
vmlinux 0xffffffffa6ffcd02
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x22
19
system_call_fastpath
vmlinux 0xffffffffa7592ed2
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [elasticsearch[a:15138]
Modules linked in: xt_set ipt_rpfilter ip_set_hash_ip ip_set_hash_net vxlan ip6_udp_tunnel udp_tunnel xt_multiport xt_mark xt_comment ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log
dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi snd_ens1371 crc32_pclmul snd_rawmidi ghash_clmulni_intel ppdev snd_ac97_codec aesni_intel ac97_bus snd_seq lrw gf128mul vmw_balloon glue_helper ablk_helper snd_seq_device cryptd snd_pcm joydev snd_timer pcspkr snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt crct10dif_pclmul crct10dif_common fb_sys_fops ttm crc32c_intel nfit drm libnvdimm mptspi ata_piix scsi_transport_spi serio_raw mptscsih libata e1000 mptbase drm_panel_orientation_quirks
CPU: 4 PID: 15138 Comm: elasticsearch[a Tainted: G               ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff92ac0367e2a0 ti: ffff92ad7c41c000 task.ti: ffff92ad7c41c000
RIP: 0010:[<ffffffffa6f16f72>]  [<ffffffffa6f16f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff92ad7c41fc20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff92ad7c41fbf0 RCX: 0000000000000008
RDX: ffffffffa7614850 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff92ad7c41fc70 R08: ffffffffa7614848 R09: ffffffffa700280c
R10: ffff92ad8031f0a0 R11: fffff0eb4457fd80 R12: 00000000000280da
R13: ffffffffa703e2b2 R14: ffff92ad7c41fbd8 R15: 0000000000000000
FS:  00007ff6e3dd8700(0000) GS:ffff92ad80300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff70c059000 CR3: 0000000164b84000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa6e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa6e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa6e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa6f1708f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa7185005>] ? cpumask_next_and+0x35/0x50
[<ffffffffa6f1763b>] smp_call_function_many+0x22b/0x270
[<ffffffffa6e7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa6e7e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa6fede36>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa6fef8ff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa6fefa23>] tlb_finish_mmu+0x23/0x30
[<ffffffffa6ff9234>] unmap_region+0xf4/0x140
[<ffffffffa7065b80>] ? d_free+0x50/0x70
[<ffffffffa6ff97f1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffa6ffb825>] do_munmap+0x2a5/0x480
[<ffffffffa6ffba65>] vm_munmap+0x65/0xb0
[<ffffffffa6ffcd02>] SyS_munmap+0x22/0x30
[<ffffffffa7592ed2>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 a0 10 b5 a7 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #17454
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffbcf1109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffbce7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffbce7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffbce7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffbcf111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffbd175465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffbcf1175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffbce7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffbce7a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffbcfe4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffbcfe5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffffbcfef014
Build id: 3.10.0-957.el7.x86_64 		0xf4
13
? d_free
vmlinux 0xffffffffbd0593df
Build id: 3.10.0-957.el7.x86_64 		0x4f
14
? __vma_rb_erase
vmlinux 0xffffffffbcfef5d1
Build id: 3.10.0-957.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffffbcff1605
Build id: 3.10.0-957.el7.x86_64 		0x2a5
16
vm_munmap
vmlinux 0xffffffffbcff1845
Build id: 3.10.0-957.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffffbcff2ad2
Build id: 3.10.0-957.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffffbd574ddb
Build id: 3.10.0-957.el7.x86_64 		0x22 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [vmtoolsd:9532]
Modules linked in: veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) overlay(T) sunrpc snd_seq_midi snd_seq_midi_event coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi ppdev snd_ac97_codec ac97_bus snd_seq snd_seq_device vmw_balloon snd_pcm snd_timer snd soundcore pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul
crct10dif_common crc32c_intel serio_raw ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm e1000 mptspi scsi_transport_spi mptscsih mptbase ata_piix nfit libata libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 9532 Comm: vmtoolsd Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8e5fe8a5c100 ti: ffff8e5fe574c000 task.ti: ffff8e5fe574c000
RIP: 0010:[<ffffffffbcf1109a>]  [<ffffffffbcf1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8e5fe574fc30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8e5fe574fc00 RCX: 0000000000000004
RDX: ffffffffbd613510 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8e5fe574fc78 R08: ffffffffbd613508 R09: ffffffffbcff838c
R10: ffff8e5ff585f0a0 R11: fffffdc74731c540 R12: 00007f8bedc79000
R13: fffffdc748aa9070 R14: ffff8e5fcba86360 R15: ffffffffbcff6e08
FS:  00007f8bedc73740(0000) GS:ffff8e5ff5840000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8bedc79000 CR3: 000000022aa5a000 CR4: 00000000003607e0
Call Trace:
[<ffffffffbce7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbce7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbce7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbcf111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffbd175465>] ? cpumask_next_and+0x35/0x50
[<ffffffffbcf1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffbce7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbce7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbcfe4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffbcfe5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffbcfef014>] unmap_region+0xf4/0x140
[<ffffffffbd0593df>] ? d_free+0x4f/0x70
[<ffffffffbcfef5d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffbcff1605>] do_munmap+0x2a5/0x480
[<ffffffffbcff1845>] vm_munmap+0x65/0xb0
[<ffffffffbcff2ad2>] SyS_munmap+0x22/0x30
[<ffffffffbd574ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 b4 bd 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #10158
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff84d163c2
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff84c7e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff84c7e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff84c7e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff84d164df
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff84f81635
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff84d16a8b
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff84c7e478
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff84c7e4eb
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff84dec026
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff84dedaef
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff84dedc13
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffff84df7404
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xf4
14
? d_free
vmlinux 0xffffffff84e62be0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x50
15
? __vma_rb_erase
vmlinux 0xffffffff84df79c1
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x121
16
do_munmap
vmlinux 0xffffffff84df9a05
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2a5
17
vm_munmap
vmlinux 0xffffffff84df9c45
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x65
18
SyS_munmap
vmlinux 0xffffffff84dfaee2
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22
19
system_call_fastpath
vmlinux 0xffffffff8538dede
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [vmtoolsd:1845]
Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel
lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm nfit crct10dif_pclmul libata mptspi crct10dif_common crc32c_intel libnvdimm scsi_transport_spi serio_raw mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 1845 Comm: vmtoolsd Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff97c6d22a8000 ti: ffff97c6d9e7c000 task.ti: ffff97c6d9e7c000
RIP: 0010:[<ffffffff84d163c2>]  [<ffffffff84d163c2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff97c6d9e7fc20  EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffffffff84df1644 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff97c6d9e7fc70 R08: 0000000000000001 R09: ffffffff84e0079c
R10: ffff97c6ffc9f0a0 R11: ffffcd8581e50f40 R12: ffff97c6952f4e58
R13: ffffcd85a0529d30 R14: ffff97c6d9e7fc60 R15: ffffffff84dff208
FS:  00007fbe0a0b6880(0000) GS:ffff97c6ffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbe0a0d3000 CR3: 0000000812eba000 CR4: 00000000001607e0
Call Trace:
[<ffffffff84c7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff84c7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff84c7e2b0>] ? leave_mm+0x110/0x110
[<ffffffff84d164df>] smp_call_function_single+0x5f/0xa0
[<ffffffff84f81635>] ? cpumask_next_and+0x35/0x50
[<ffffffff84d16a8b>] smp_call_function_many+0x22b/0x270
[<ffffffff84c7e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff84c7e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff84dec026>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff84dedaef>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff84dedc13>] tlb_finish_mmu+0x23/0x30
[<ffffffff84df7404>] unmap_region+0xf4/0x140
[<ffffffff84e62be0>] ? d_free+0x50/0x70
[<ffffffff84df79c1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff84df9a05>] do_munmap+0x2a5/0x480
[<ffffffff84df9c45>] vm_munmap+0x65/0xb0
[<ffffffff84dfaee2>] SyS_munmap+0x22/0x30
[<ffffffff8538dede>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 20 e8 94 85 48 89 df e8 f7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #6099
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb5d16f22
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb5c7e730
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb5c7e730
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb5c7e730
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb5d1703f
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb5f84f65
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb5d175eb
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb5c7e8f8
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb5c7e96b
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb5dede16
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb5def8df
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb5defa03
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x23
13
unmap_region
vmlinux 0xffffffffb5df9214
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xf4
14
? d_free
vmlinux 0xffffffffb5e65b30
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x50
15
? __vma_rb_erase
vmlinux 0xffffffffb5df97d1
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x121
16
do_munmap
vmlinux 0xffffffffb5dfb805
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x2a5
17
vm_munmap
vmlinux 0xffffffffb5dfba45
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x65
18
SyS_munmap
vmlinux 0xffffffffb5dfcce2
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x22
19
system_call_fastpath
vmlinux 0xffffffffb6392ed2
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [sqlservr:83169]
Modules linked in: ip_set nfnetlink tcp_lp fuse binfmt_misc xt_CHECKSUM iptable_mangle br_netfilter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus aesni_intel vmw_balloon snd_seq lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm sg joydev pcspkr snd_timer snd i2c_piix4 vmw_vmci soundcore parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm nfit crct10dif_pclmul crct10dif_common libnvdimm crc32c_intel ata_piix drm serio_raw e1000 libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: cvblk]
CPU: 1 PID: 83169 Comm: sqlservr Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-1127.8.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff946c90e30000 ti: ffff946c8107c000 task.ti: ffff946c8107c000
RIP: 0010:[<ffffffffb5d16f22>]  [<ffffffffb5d16f22>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff946c8107fc20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff946c8107fbf0 RCX: 0000000000000001
RDX: ffffffffb6413650 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff946c8107fc70 R08: ffffffffb6413648 R09: ffffffffb5e027dc
R10: ffff946db965f0a0 R11: ffffe3ad41cfc400 R12: 00000000000280da
R13: ffffffffb5e3e262 R14: ffff946c8107fbd8 R15: 0000000000000000
FS:  00007ff619234700(0000) GS:ffff946db9640000(006b) knlGS:000000006b2ab000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff61933c000 CR3: 00000001020c8000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb5c7e730>] ? leave_mm+0x120/0x120
[<ffffffffb5c7e730>] ? leave_mm+0x120/0x120
[<ffffffffb5c7e730>] ? leave_mm+0x120/0x120
[<ffffffffb5d1703f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb5f84f65>] ? cpumask_next_and+0x35/0x50
[<ffffffffb5d175eb>] smp_call_function_many+0x22b/0x270
[<ffffffffb5c7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb5c7e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb5dede16>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb5def8df>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb5defa03>] tlb_finish_mmu+0x23/0x30
[<ffffffffb5df9214>] unmap_region+0xf4/0x140
[<ffffffffb5e65b30>] ? d_free+0x50/0x70
[<ffffffffb5df97d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb5dfb805>] do_munmap+0x2a5/0x480
[<ffffffffb5dfba45>] vm_munmap+0x65/0xb0
[<ffffffffb5dfcce2>] SyS_munmap+0x22/0x30
[<ffffffffb6392ed2>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 20 10 95 b6 48 89 df e8 d7 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #5203
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff89b111ea
Build id: 3.10.0-957.12.1.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff89a7a400
Build id: 3.10.0-957.12.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff89a7a400
Build id: 3.10.0-957.12.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff89a7a400
Build id: 3.10.0-957.12.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff89b112ff
Build id: 3.10.0-957.12.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff89d75af5
Build id: 3.10.0-957.12.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff89b118ab
Build id: 3.10.0-957.12.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff89a7a5c8
Build id: 3.10.0-957.12.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff89a7a638
Build id: 3.10.0-957.12.1.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff89be44c7
Build id: 3.10.0-957.12.1.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff89be5dc5
Build id: 3.10.0-957.12.1.el7.x86_64 		0x55
12
unmap_region
vmlinux 0xffffffff89bef464
Build id: 3.10.0-957.12.1.el7.x86_64 		0xf4
13
? d_free
vmlinux 0xffffffff89c59a0f
Build id: 3.10.0-957.12.1.el7.x86_64 		0x4f
14
? __vma_rb_erase
vmlinux 0xffffffff89befa21
Build id: 3.10.0-957.12.1.el7.x86_64 		0x121
15
do_munmap
vmlinux 0xffffffff89bf1a55
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2a5
16
vm_munmap
vmlinux 0xffffffff89bf1c95
Build id: 3.10.0-957.12.1.el7.x86_64 		0x65
17
SyS_munmap
vmlinux 0xffffffff89bf2f22
Build id: 3.10.0-957.12.1.el7.x86_64 		0x22
18
system_call_fastpath
vmlinux 0xffffffff8a175d9b
Build id: 3.10.0-957.12.1.el7.x86_64 		0x22 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [db2sysc:12822]
Modules linked in: tcp_lp veth fuse xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc iosf_mbi kvm_intel ppdev kvm irqbypass crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common
crc32c_intel serio_raw ata_piix vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm e1000 libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 12822 Comm: db2sysc Tainted: G             L ------------   3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9b919cf9c100 ti: ffff9b9325f54000 task.ti: ffff9b9325f54000
RIP: 0010:[<ffffffff89b111ea>]  [<ffffffff89b111ea>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9b9325f57c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9b9325f57c00 RCX: 0000000000000002
RDX: ffffffff8a213050 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9b9325f57c78 R08: ffffffff8a213048 R09: ffffffff89bf87dc
R10: ffff9b937561f0a0 R11: fffff3924541c080 R12: 00007fe0271e0000
R13: fffff392460aacf0 R14: ffff9b91a5abad80 R15: ffffffff89bf7258
FS:  00007fe00b3fe700(0000) GS:ffff9b9375600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe0271e0000 CR3: 000000005fb94000 CR4: 00000000001607f0
Call Trace:
[<ffffffff89a7a400>] ? leave_mm+0x110/0x110
[<ffffffff89a7a400>] ? leave_mm+0x110/0x110
[<ffffffff89a7a400>] ? leave_mm+0x110/0x110
[<ffffffff89b112ff>] smp_call_function_single+0x5f/0xa0
[<ffffffff89d75af5>] ? cpumask_next_and+0x35/0x50
[<ffffffff89b118ab>] smp_call_function_many+0x22b/0x270
[<ffffffff89a7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff89a7a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff89be44c7>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff89be5dc5>] tlb_finish_mmu+0x55/0x60
[<ffffffff89bef464>] unmap_region+0xf4/0x140
[<ffffffff89c59a0f>] ? d_free+0x4f/0x70
[<ffffffff89befa21>] ? __vma_rb_erase+0x121/0x220
[<ffffffff89bf1a55>] do_munmap+0x2a5/0x480
[<ffffffff89bf1c95>] vm_munmap+0x65/0xb0
[<ffffffff89bf2f22>] SyS_munmap+0x22/0x30
[<ffffffff8a175d9b>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 ba 74 8a 48 89 df e8 6a b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00