Info

Permalink
Function
generic_exec_single
First occurrence
2021-02-05
Last occurrence
2026-03-21
State
NEW
Components
kernel
Quality
-37

Statistics

Operating system Count
CentOS 7 182
Architecture Count
x86_64 182
Related packages Count
kernel 182
 0:3.10.0-1160.88.1.el7 74
 0:3.10.0-1160.62.1.el7 21
 0:3.10.0-1160.el7 18
 0:3.10.0-1160.118.1.el7 12
 0:3.10.0-1160.71.1.el7 9
 0:3.10.0-1160.105.1.el7 8
 0:3.10.0-1160.15.2.el7 7
 0:3.10.0-1160.45.1.el7 6
 0:3.10.0-1160.25.1.el7 4
 0:3.10.0-1160.76.1.el7 4
 0:3.10.0-1160.59.1.el7 4
 0:3.10.0-1160.31.1.el7 3
 0:3.10.0-1160.102.1.el7 2
 0:3.10.0-1160.11.1.el7 2
 0:3.10.0-1160.21.1.el7 1
 0:3.10.0-1160.24.1.el7 1
 0:3.10.0-1160.53.1.el7 1
 0:3.10.0-1160.49.1.el7 1
 0:3.10.0-1160.99.1.el7 1
 0:3.10.0-1160.119.1.el7 1
 0:3.10.0-1160.92.1.el7 1
 0:3.10.0-1160.90.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1201412
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8231db82
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff82281b10
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x130
3
? leave_mm
vmlinux 0xffffffff82281b10
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x130
4
? leave_mm
vmlinux 0xffffffff82281b10
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x130
5
smp_call_function_single
vmlinux 0xffffffff8231dc9f
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8259c5d5
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8231e25b
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff82281fa8
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8228201b
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff823fa07e
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffff823fbbff
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff823fbd33
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff823ff27a
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xea
14
? dequeue_entity
vmlinux 0xffffffff822eb01c
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x11c
15
SyS_madvise
vmlinux 0xffffffff823f966d
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x49d
16
? __seccomp_filter
vmlinux 0xffffffff8235f93b
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x5b
17
? pick_next_entity
vmlinux 0xffffffff822e8699
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xa9
18
tracesys
vmlinux 0xffffffff829c562e
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [wdavdaemon:5155]
Modules linked in: rpcsec_gss_krb5 nfsv4 nfs lockd grace fscache tcp_lp cmac arc4 md4 nls_utf8 cifs ccm dns_resolver iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod xfs snd_seq libcrc32c sb_edac snd_seq_device snd_pcm iosf_mbi snd_timer crc32_pclmul ghash_clmulni_intel snd ppdev aesni_intel soundcore lrw gf128mul glue_helper vmw_balloon ablk_helper pcspkr cryptd joydev sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix libahci drm nfit crct10dif_pclmul crct10dif_common crc32c_intel libata libnvdimm serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks
floppy fuse
CPU: 3 PID: 5155 Comm: wdavdaemon Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.105.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9b3224314200 ti: ffff9b3303838000 task.ti: ffff9b3303838000
RIP: 0010:[<ffffffff8231db82>]  [<ffffffff8231db82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9b330383bbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9b330383bbb0 RCX: 0000000000000001
RDX: ffffffff82a139d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9b330383bc30 R08: ffffffff82a139c8 R09: 00003ffffffff000
R10: ffff9b32c4b4cca8 R11: 0000000000000206 R12: 0000000f00000000
R13: ffff9b35ed61acc0 R14: ffff9b3335a7b800 R15: 0000000000000087
FS:  00007fe23f439700(0000) GS:ffff9b35ed6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000002082b10 CR3: 00000001402ba000 CR4: 00000000003607e0
Call Trace:
[<ffffffff82281b10>] ? leave_mm+0x130/0x130
[<ffffffff82281b10>] ? leave_mm+0x130/0x130
[<ffffffff82281b10>] ? leave_mm+0x130/0x130
[<ffffffff8231dc9f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8259c5d5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8231e25b>] smp_call_function_many+0x22b/0x270
[<ffffffff82281fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8228201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff823fa07e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff823fbbff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff823fbd33>] tlb_finish_mmu+0x23/0x40
[<ffffffff823ff27a>] zap_page_range+0xea/0x140
[<ffffffff822eb01c>] ? dequeue_entity+0x11c/0x5d0
[<ffffffff823f966d>] SyS_madvise+0x49d/0xac0
[<ffffffff8235f93b>] ? __seccomp_filter+0x5b/0x310
[<ffffffff822e8699>] ? pick_next_entity+0xa9/0x190
[<ffffffff829c562e>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 80 19 f5 82 48 89 df e8 97 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1183065
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8291dba2
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff82881b10
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x130
3
? leave_mm
vmlinux 0xffffffff82881b10
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x130
4
? leave_mm
vmlinux 0xffffffff82881b10
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x130
5
smp_call_function_single
vmlinux 0xffffffff8291dcbf
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff82b9c615
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8291e27b
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff82881fa8
Build id: 3.10.0-1160.118.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8288201b
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff829fa0ce
Build id: 3.10.0-1160.118.1.el7.x86_64 		0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffff829fbc4f
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff829fbd83
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff829ff2ca
Build id: 3.10.0-1160.118.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffff828e1118
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffff829f96bd
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x49d
16
? __seccomp_filter
vmlinux 0xffffffff8295f95b
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x5b
17
tracesys
vmlinux 0xffffffff82fc562e
Build id: 3.10.0-1160.118.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kuboard-sso:10788]
Modules linked in: xt_statistic tcp_lp nls_utf8 isofs ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle vxlan ip6_udp_tunnel udp_tunnel xt_physdev xt_NFLOG nfnetlink_log xt_limit xt_mark xt_multiport xt_comment ip_set veth xt_nat nf_conntrack_netlink xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache overlay(T) br_netfilter bridge stp llc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus iosf_mbi snd_seq
crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel lrw gf128mul snd_timer snd soundcore glue_helper ablk_helper cryptd vmw_balloon ppdev pcspkr sg joydev parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm libata mptspi scsi_transport_spi e1000 mptscsih serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 10788 Comm: kuboard-sso Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff8c2a8080d280 ti: ffff8c2419f04000 task.ti: ffff8c2419f04000
RIP: 0010:[<ffffffff8291dba2>]  [<ffffffff8291dba2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8c2419f07be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8c2419f07bb0 RCX: 0000000000000001
RDX: ffffffff830139d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8c2419f07c30 R08: ffffffff830139c8 R09: 00003ffffffff000
R10: ffff8c242669cbd0 R11: 0000000000000202 R12: ffff8c2b994f4840
R13: ffff8c2b994f4840 R14: 00000000000001e2 R15: ffff8c2b78ff1400
FS:  00007f69c9e5f6c0(0000) GS:ffff8c2b9d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000f2dfa0 CR3: 0000000099d06000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff82881b10>] ? leave_mm+0x130/0x130
[<ffffffff82881b10>] ? leave_mm+0x130/0x130
[<ffffffff82881b10>] ? leave_mm+0x130/0x130
[<ffffffff8291dcbf>] smp_call_function_single+0x5f/0xa0
[<ffffffff82b9c615>] ? cpumask_next_and+0x35/0x50
[<ffffffff8291e27b>] smp_call_function_many+0x22b/0x270
[<ffffffff82881fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8288201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff829fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff829fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff829fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffff829ff2ca>] zap_page_range+0xea/0x140
[<ffffffff828e1118>] ? wake_up_q+0x58/0x90
[<ffffffff829f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffff8295f95b>] ? __seccomp_filter+0x5b/0x310
[<ffffffff82fc562e>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 c0 19 55 83 48 89 df e8 b7 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1012836
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8b91dba2
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8b881b10
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x130
3
? leave_mm
vmlinux 0xffffffff8b881b10
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x130
4
? leave_mm
vmlinux 0xffffffff8b881b10
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x130
5
smp_call_function_single
vmlinux 0xffffffff8b91dcbf
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8bb9c615
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8b91e27b
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8b881fa8
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8b88201b
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8b9fa0ce
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffff8b9fbc4f
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff8b9fbd83
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff8b9ff2ca
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff8b9f96bd
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x49d
15
? __seccomp_filter
vmlinux 0xffffffff8b95f95b
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x5b
16
tracesys
vmlinux 0xffffffff8bfc562e
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [DOM Worker:77916]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 iosf_mbi snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq aesni_intel snd_seq_device lrw snd_pcm gf128mul glue_helper ablk_helper snd_timer snd ppdev vmw_balloon cryptd soundcore
pcspkr vmw_vmci sg i2c_piix4 joydev parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm nfit mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel mptscsih libata libnvdimm e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 77916 Comm: DOM Worker Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9253e4682100 ti: ffff9255826ec000 task.ti: ffff9255826ec000
RIP: 0010:[<ffffffff8b91dba2>]  [<ffffffff8b91dba2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9255826efbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9255826efbb0 RCX: 0000000000000001
RDX: ffffffff8c0139d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9255826efc30 R08: ffffffff8c0139c8 R09: 00003ffffffff000
R10: ffff9254ec671878 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000300000000 R14: ffff9255e529acc0 R15: ffffffff8b9d36bf
FS:  00007f628970f700(0000) GS:ffff9255e5300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000023ab45663000 CR3: 000000012898a000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8b881b10>] ? leave_mm+0x130/0x130
[<ffffffff8b881b10>] ? leave_mm+0x130/0x130
[<ffffffff8b881b10>] ? leave_mm+0x130/0x130
[<ffffffff8b91dcbf>] smp_call_function_single+0x5f/0xa0
[<ffffffff8bb9c615>] ? cpumask_next_and+0x35/0x50
[<ffffffff8b91e27b>] smp_call_function_many+0x22b/0x270
[<ffffffff8b881fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8b88201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8b9fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff8b9fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8b9fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffff8b9ff2ca>] zap_page_range+0xea/0x140
[<ffffffff8b9f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffff8b95f95b>] ? __seccomp_filter+0x5b/0x310
[<ffffffff8bfc562e>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 c0 19 55 8c 48 89 df e8 b7 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #974402
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffbc116b62
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffbc07ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffbc07ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffbc07ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffbc116c7f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffbc386865
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffbc11722b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffbc07e258
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffbc07e2cb
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffbc1ee6c6
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffbc1f018f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffbc1f02b3
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffbc1f377a
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffbc1edc6d
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x49d
15
? __seccomp_filter
vmlinux 0xffffffffbc1573fb
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x5b
16
? pick_next_entity
vmlinux 0xffffffffbc0e2409
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa9
17
tracesys
vmlinux 0xffffffffbc79a226
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [java:5082]
Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_REDIRECT nf_nat_redirect vxlan ip6_udp_tunnel udp_tunnel xt_mark xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink tun overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event kvm_amd snd_ens1371 kvm snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev irqbypass snd_timer crc32_pclmul snd ghash_clmulni_intel vmw_balloon aesni_intel soundcore lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg joydev parport_pc parport vmw_vmci
i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata nfit libnvdimm mptspi e1000 scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 5082 Comm: java Kdump: loaded Tainted: G               ------------ T 3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffffa12eb55b0000 ti: ffffa125b1b08000 task.ti: ffffa125b1b08000
RIP: 0010:[<ffffffffbc116b62>]  [<ffffffffbc116b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffffa125b1b0bbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffffa125b1b0bbb0 RCX: 0000000000000003
RDX: ffffffffbc813cd0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffffa125b1b0bc30 R08: ffffffffbc813cc8 R09: 00003ffffffff000
R10: ffffa12ecf469b00 R11: 0000000000000206 R12: ffffa12eb55b0000
R13: 00000003b1b0bbe0 R14: 0000000000000002 R15: ffffffffbc1c868f
FS:  00007f1e5e3c4b10(0000) GS:ffffa12eef2c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005555cd87b810 CR3: 0000000854e92000 CR4: 0000000000340fe0
Call Trace:
[<ffffffffbc07ddd0>] ? leave_mm+0x120/0x120
[<ffffffffbc07ddd0>] ? leave_mm+0x120/0x120
[<ffffffffbc07ddd0>] ? leave_mm+0x120/0x120
[<ffffffffbc116c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffffbc386865>] ? cpumask_next_and+0x35/0x50
[<ffffffffbc11722b>] smp_call_function_many+0x22b/0x270
[<ffffffffbc07e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbc07e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbc1ee6c6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbc1f018f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbc1f02b3>] tlb_finish_mmu+0x23/0x30
[<ffffffffbc1f377a>] zap_page_range+0xea/0x130
[<ffffffffbc1edc6d>] SyS_madvise+0x49d/0xac0
[<ffffffffbc1573fb>] ? __seccomp_filter+0x5b/0x300
[<ffffffffbc0e2409>] ? pick_next_entity+0xa9/0x190
[<ffffffffbc79a226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 60 18 d5 bc 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #888616
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffaf316562
Build id: 3.10.0-1160.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffaf27e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffaf27e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffaf27e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffaf31667f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffaf585de5
Build id: 3.10.0-1160.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffaf316c2b
Build id: 3.10.0-1160.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaf27e868
Build id: 3.10.0-1160.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffaf27e8db
Build id: 3.10.0-1160.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffaf3edd06
Build id: 3.10.0-1160.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffaf3ef7cf
Build id: 3.10.0-1160.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffaf3ef8f3
Build id: 3.10.0-1160.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffaf3f2dba
Build id: 3.10.0-1160.el7.x86_64 		0xea
14
? handle_mm_fault
vmlinux 0xffffffffaf3f5e0d
Build id: 3.10.0-1160.el7.x86_64 		0xbfd
15
SyS_madvise
vmlinux 0xffffffffaf3ed2ad
Build id: 3.10.0-1160.el7.x86_64 		0x49d
16
? __seccomp_filter
vmlinux 0xffffffffaf356dfb
Build id: 3.10.0-1160.el7.x86_64 		0x5b
17
tracesys
vmlinux 0xffffffffaf994226
Build id: 3.10.0-1160.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 68s! [scheduler.rb:73:46068]
Modules linked in: ip6table_mangle ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_filter iptable_mangle ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables xt_comment xt_mark xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ppdev ac97_bus snd_seq snd_seq_device snd_pcm crc32_pclmul ghash_clmulni_intel vmw_balloon snd_timer aesni_intel lrw gf128mul glue_helper ablk_helper sg cryptd snd joydev pcspkr soundcore parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod
cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops ttm serio_raw drm ata_piix mptspi e1000 libata scsi_transport_spi nfit mptscsih mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 46068 Comm: scheduler.rb:73 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff965679781080 ti: ffff9657b1244000 task.ti: ffff9657b1244000
RIP: 0010:[<ffffffffaf316562>]  [<ffffffffaf316562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9657b1247be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9657b1247bb0 RCX: 0000000000000002
RDX: ffffffffafa13a10 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9657b1247c30 R08: ffffffffafa13a08 R09: 00003ffffffff000
R10: ffff96583e299878 R11: 0000000000000206 R12: ffffffffaf986e76
R13: ffffffffaf986e82 R14: ffffffffaf986e76 R15: ffffffffaf986e82
FS:  00007fb0c5fff700(0000) GS:ffff96592d600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb0f01ee678 CR3: 00000003084d6000 CR4: 00000000003607f0
Call Trace:
[<ffffffffaf27e6a0>] ? leave_mm+0x120/0x120
[<ffffffffaf27e6a0>] ? leave_mm+0x120/0x120
[<ffffffffaf27e6a0>] ? leave_mm+0x120/0x120
[<ffffffffaf31667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffaf585de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffaf316c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffaf27e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaf27e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffaf3edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffaf3ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffaf3ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffaf3f2dba>] zap_page_range+0xea/0x130
[<ffffffffaf3f5e0d>] ? handle_mm_fault+0xbfd/0xfb0
[<ffffffffaf3ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffaf356dfb>] ? __seccomp_filter+0x5b/0x300
[<ffffffffaf994226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 e0 13 f5 af 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #853902
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9d316ae2
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9d27ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff9d27ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff9d27ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff9d316bff
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9d586845
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9d3171ab
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9d27e258
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9d27e2cb
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9d3ee6a6
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9d3f016f
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9d3f0293
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff9d3f375a
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff9d3edc4d
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x49d
15
? __seccomp_filter
vmlinux 0xffffffff9d35737b
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x5b
16
? system_call_after_swapgs
vmlinux 0xffffffff9d999ed5
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffff9d999ec9
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x96
18
tracesys
vmlinux 0xffffffff9d99a226
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [mysqld:7980]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) sunrpc ppdev joydev virtio_balloon sg parport_pc parport i2c_piix4 pcspkr ip_tables xfs libcrc32c virtio_net virtio_blk net_failover virtio_console virtio_scsi failover sr_mod cdrom ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci virtio_ring drm_panel_orientation_quirks floppy virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 7980 Comm: mysqld Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9f5794be4200 ti: ffff9f5387ae4000 task.ti: ffff9f5387ae4000
RIP: 0010:[<ffffffff9d316ae2>]  [<ffffffff9d316ae2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9f5387ae7be0  EFLAGS: 00000202
RAX: 0000000000000060 RBX: ffff9f5387ae7bb0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 0000000000000060 RDI: 0000000000000282
RBP: ffff9f5387ae7c30 R08: ffffffff9da146c8 R09: 00003ffffffff000
R10: ffff9f569c327950 R11: 00007fe103df1000 R12: ffff9f5387ae7bc8
R13: ffff9f571a2e2000 R14: ffff9f555cdea4f8 R15: ffffffffc077e4da
FS:  00007fe103e31700(0000) GS:ffff9f57ade00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb8aa853000 CR3: 000000008663e000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9d27ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9d27ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9d27ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9d316bff>] smp_call_function_single+0x5f/0xa0
[<ffffffff9d586845>] ? cpumask_next_and+0x35/0x50
[<ffffffff9d3171ab>] smp_call_function_many+0x22b/0x270
[<ffffffff9d27e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9d27e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9d3ee6a6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9d3f016f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9d3f0293>] tlb_finish_mmu+0x23/0x30
[<ffffffff9d3f375a>] zap_page_range+0xea/0x130
[<ffffffff9d3edc4d>] SyS_madvise+0x49d/0xac0
[<ffffffff9d35737b>] ? __seccomp_filter+0x5b/0x300
[<ffffffff9d999ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff9d999ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff9d99a226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 00 19 f5 9d 48 89 df e8 d7 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #756967
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa7b16b62
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa7a7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa7a7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa7a7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa7b16c7f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa7d86865
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa7b1722b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa7a7e258
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa7a7e2cb
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa7bee6c6
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa7bf018f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa7bf02b3
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffa7bf377a
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xea
14
? __slab_free
vmlinux 0xffffffffa7c2680c
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x9c
15
? kmem_cache_free
vmlinux 0xffffffffa7c27042
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x1e2
16
SyS_madvise
vmlinux 0xffffffffa7bedc6d
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x49d
17
? __seccomp_filter
vmlinux 0xffffffffa7b573fb
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x5b
18
tracesys
vmlinux 0xffffffffa819a226
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [ReplCoord-4609:67359]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul vmw_balloon glue_helper ablk_helper cryptd sg joydev pcspkr vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul crct10dif_common drm crc32c_intel libata mptspi nfit scsi_transport_spi e1000 mptscsih serio_raw libnvdimm mptbase drm_panel_orientation_quirks dm_mirror
dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 67359 Comm: ReplCoord-4609 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff903f207e2100 ti: ffff9040eed4c000 task.ti: ffff9040eed4c000
RIP: 0010:[<ffffffffa7b16b62>]  [<ffffffffa7b16b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9040eed4fbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9040eed4fbb0 RCX: 000000000000000b
RDX: ffffffffa82150d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9040eed4fc30 R08: ffffffffa82150c8 R09: 00003ffffffff000
R10: ffff9040da9bd1b8 R11: 00007facd694e000 R12: ffff9040df4ecfc0
R13: ffff9040df4ecfc0 R14: 0000000000000002 R15: ffff9040f575ad40
FS:  00007facd714d700(0000) GS:ffff9040f5740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f98269f9ef8 CR3: 00000002169d2000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa7a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa7a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa7a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa7b16c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa7d86865>] ? cpumask_next_and+0x35/0x50
[<ffffffffa7b1722b>] smp_call_function_many+0x22b/0x270
[<ffffffffa7a7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa7a7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa7bee6c6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa7bf018f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa7bf02b3>] tlb_finish_mmu+0x23/0x30
[<ffffffffa7bf377a>] zap_page_range+0xea/0x130
[<ffffffffa7c2680c>] ? __slab_free+0x9c/0x290
[<ffffffffa7c27042>] ? kmem_cache_free+0x1e2/0x200
[<ffffffffa7bedc6d>] SyS_madvise+0x49d/0xac0
[<ffffffffa7b573fb>] ? __seccomp_filter+0x5b/0x300
[<ffffffffa819a226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 60 18 75 a8 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #546907
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8ef16a32
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8ee7ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8ee7ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8ee7ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8ef16b4f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8f186a45
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8ef170fb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8ee7e258
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8ee7e2cb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8efee676
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff8eff013f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff8eff0263
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff8eff372a
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffff8eedada8
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffff8efedc1d
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x49d
16
? poll_select_copy_remaining
vmlinux 0xffffffff8f064924
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x144
17
system_call_fastpath
vmlinux 0xffffffff8f595f92
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 23s! [8_scheduler:5153]
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [docker-proxy:2926]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ppdev snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec aesni_intel ac97_bus snd_seq lrw vmw_balloon gf128mul glue_helper snd_seq_device ablk_helper cryptd snd_pcm snd_timer joydev pcspkr snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel
drm ata_piix mptspi scsi_transport_spi nfit libata mptscsih serio_raw mptbase libnvdimm e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 7 PID: 2926 Comm: docker-proxy Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff926ab354b180 ti: ffff926a91064000 task.ti: ffff926a91064000
RIP: 0010:[<ffffffff8ef16a32>]  [<ffffffff8ef16a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff926a91067be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff926a91067bb0 RCX: 000000000000000a
RDX: ffffffff8f614e50 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff926a91067c30 R08: ffffffff8f614e48 R09: 00003ffffffff000
R10: ffff9267ea0ed008 R11: 0000000000000246 R12: ffff926af6486f00
R13: ffff926af6486f00 R14: 0000000000000002 R15: ffff926afd7dad40
FS:  00007ff9f94fe700(0000) GS:ffff926afd7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4200cb010 CR3: 0000000f9373a000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8ee7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8ee7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8ee7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8ef16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8f186a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff8ef170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff8ee7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ee7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8efee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8eff013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8eff0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff8eff372a>] zap_page_range+0xea/0x130
[<ffffffff8eedada8>] ? wake_up_q+0x58/0x80
[<ffffffff8efedc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff8f064924>] ? poll_select_copy_remaining+0x144/0x180
[<ffffffff8f595f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 60 15 b5 8f 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #445799
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb0f16562
Build id: 3.10.0-1160.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb0e7e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb0e7e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb0e7e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb0f1667f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb1185de5
Build id: 3.10.0-1160.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb0f16c2b
Build id: 3.10.0-1160.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb0e7e868
Build id: 3.10.0-1160.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb0e7e8db
Build id: 3.10.0-1160.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb0fedd06
Build id: 3.10.0-1160.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb0fef7cf
Build id: 3.10.0-1160.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb0fef8f3
Build id: 3.10.0-1160.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffb0ff2dba
Build id: 3.10.0-1160.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffb0fed2ad
Build id: 3.10.0-1160.el7.x86_64 		0x49d
15
? __seccomp_filter
vmlinux 0xffffffffb0f56dfb
Build id: 3.10.0-1160.el7.x86_64 		0x5b
16
tracesys
vmlinux 0xffffffffb1594226
Build id: 3.10.0-1160.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [popen.rb:42:92530]
Modules linked in: veth br_netfilter tun devlink overlay(T) bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul ac97_bus ghash_clmulni_intel snd_seq snd_seq_device aesni_intel snd_pcm lrw gf128mul ppdev glue_helper ablk_helper cryptd snd_timer vmw_balloon snd soundcore joydev sg pcspkr i2c_piix4 vmw_vmci parport_pc parport xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops ttm nfit libnvdimm drm serio_raw mptspi e1000 scsi_transport_spi ata_piix mptscsih libata mptbase drm_panel_orientation_quirks
fuse [last unloaded: ip_tables]
CPU: 3 PID: 92530 Comm: popen.rb:42 Kdump: loaded Tainted: G    B        L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9c84dfe11080 ti: ffff9c84f1cf8000 task.ti: ffff9c84f1cf8000
RIP: 0010:[<ffffffffb0f16562>]  [<ffffffffb0f16562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c84f1cfbbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9c84f1cfbbb0 RCX: 0000000000000001
RDX: ffffffffb1613790 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9c84f1cfbc30 R08: ffffffffb1613788 R09: 00003ffffffff000
R10: ffff9c85343bf7a0 R11: 0000000000000202 R12: ffffffffb0ee4306
R13: ffff9c84f1cfbb88 R14: ffffffffb0ee4759 R15: ffff9c84f1cfbb60
FS:  00007f4ccdcf0700(0000) GS:ffff9c85b96c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4cca6a2000 CR3: 000000003592e000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb0e7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb0e7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb0e7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb0f1667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb1185de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb0f16c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffb0e7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb0e7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb0fedd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb0fef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb0fef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffb0ff2dba>] zap_page_range+0xea/0x130
[<ffffffffb0fed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffb0f56dfb>] ? __seccomp_filter+0x5b/0x300
[<ffffffffb1594226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 e0 13 b5 b1 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #362546
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9d716a32
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9d67ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff9d67ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff9d67ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff9d716b4f
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9d986a45
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9d7170fb
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9d67e258
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9d67e2cb
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9d7ee676
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9d7f013f
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9d7f0263
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff9d7f372a
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffff9d6dada8
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffff9d7edc1d
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x49d
16
? __seccomp_filter
vmlinux 0xffffffff9d75729b
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x5b
17
? wake_up_state
vmlinux 0xffffffff9d6dadf0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x20
18
tracesys
vmlinux 0xffffffff9dd96226
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [storagenode:31049]
Modules linked in: tcp_diag udp_diag inet_diag veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat ipt_REJECT nf_reject_ipv4 tun bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter devlink ebtables ip6table_filter ip6_tables iptable_filter overlay(T) sunrpc xfs libcrc32c iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel iTCO_wdt iTCO_vendor_support sg aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_i801 lpc_ich joydev virtio_rng virtio_balloon ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci virtio_net libata virtio_console
net_failover virtio_scsi failover crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 31049 Comm: storagenode Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.14.0-1.el8 04/01/2014
task: ffff8d7d952ee300 ti: ffff8d7fc5098000 task.ti: ffff8d7fc5098000
RIP: 0010:[<ffffffff9d716a32>]  [<ffffffff9d716a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8d7fc509bbe0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8d7fc509bbb0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8d7fc509bc30 R08: ffffffff9de13a40 R09: 00003ffffffff000
R10: ffff8d7eba1090d8 R11: 0000000000000202 R12: ffff8d7dda833c00
R13: ffff8d7dda833c00 R14: 0000000000000390 R15: ffff8d7ff6105e00
FS:  00007efbe0db1f38(0000) GS:ffff8d7fffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559b62f25fb8 CR3: 0000000187232000 CR4: 0000000000360ee0
Call Trace:
[<ffffffff9d67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9d67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9d67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9d716b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9d986a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff9d7170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff9d67e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9d67e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9d7ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9d7f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9d7f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff9d7f372a>] zap_page_range+0xea/0x130
[<ffffffff9d6dada8>] ? wake_up_q+0x58/0x80
[<ffffffff9d7edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff9d75729b>] ? __seccomp_filter+0x5b/0x300
[<ffffffff9d6dadf0>] ? wake_up_state+0x20/0x20
[<ffffffff9dd96226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 60 15 35 9e 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #361667
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffaaf16a36
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffaae7ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffaae7ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffaae7ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffaaf16b4f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffab186a45
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffaaf170fb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaae7e258
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffaae7e2cb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffaafee676
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffaaff013f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffaaff0263
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffaaff372a
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffaafedc1d
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x49d
15
? __seccomp_filter
vmlinux 0xffffffffaaf5729b
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x5b
16
tracesys
vmlinux 0xffffffffab596226
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [elasticsearch[_:2270]
Modules linked in: xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop overlay falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13005(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_seq_device snd_pcm lrw gf128mul glue_helper ppdev snd_timer ablk_helper snd cryptd virtio_rng soundcore joydev pcspkr parport_pc parport virtio_balloon i2c_piix4 ip_tables xfs libcrc32c virtio_net virtio_console net_failover failover virtio_blk ata_generic pata_acpi
qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw floppy virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 2270 Comm: elasticsearch[_ Tainted: P            E  ------------   3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.13.0-2.module+el8.3.0+7860+a7792d29 04/01/2014
task: ffff93ef725e5280 ti: ffff93ef729d4000 task.ti: ffff93ef729d4000
RIP: 0010:[<ffffffffaaf16a36>]  [<ffffffffaaf16a36>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff93ef729d7be0  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff93ef729d7bb0 RCX: 000000000000003e
RDX: 0000000000000003 RSI: 0000000000000002 RDI: 0000000000000282
RBP: ffff93ef729d7c30 R08: ffffffffab6137c0 R09: 00003ffffffff000
R10: ffff93ef7269c000 R11: 00007fc6684fc000 R12: ffff93ef725e5280
R13: ffff93ef729d7c18 R14: ffffffffab009efd R15: ffff93ef729d7b70
FS:  00007fc6685f8700(0000) GS:ffff93f03fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000019ed018 CR3: 000000007254e000 CR4: 00000000001606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaae7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffaae7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffaae7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffaaf16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffab186a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffaaf170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffaae7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaae7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffaafee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffaaff013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffaaff0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffaaff372a>] zap_page_range+0xea/0x130
[<ffffffffaafedc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffaaf5729b>] ? __seccomp_filter+0x5b/0x300
[<ffffffffab596226>] tracesys+0xa6/0xcc
Code: 14 c5 60 15 b5 ab 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #276854
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa3b16a32
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa3a7ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa3a7ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa3a7ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa3b16b4f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa3d86a45
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa3b170fb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa3a7e258
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa3a7e2cb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa3bee676
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa3bf013f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa3bf0263
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffa3bf372a
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xea
14
? handle_mm_fault
vmlinux 0xffffffffa3bf677d
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xbfd
15
? do_sync_write
vmlinux 0xffffffffa3c4daa3
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x93
16
SyS_madvise
vmlinux 0xffffffffa3bedc1d
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x49d
17
? __seccomp_filter
vmlinux 0xffffffffa3b5729b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x5b
18
tracesys
vmlinux 0xffffffffa4196226
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [Web Content:4621]
Modules linked in: tcp_lp rfcomm xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter binfmt_misc des_generic md4 nls_utf8 cifs ccm dns_resolver overlay(T) vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event squashfs loop ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device aesni_intel snd_pcm lrw btusb gf128mul glue_helper btrtl ablk_helper btbcm btintel cryptd bluetooth joydev pcspkr sg snd_timer rfkill snd vmw_vmci
i2c_piix4 soundcore parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm nfit ata_piix libnvdimm serio_raw mptspi e1000 libata scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 4621 Comm: Web Content Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff98764be6d280 ti: ffff987255d24000 task.ti: ffff987255d24000
RIP: 0010:[<ffffffffa3b16a32>]  [<ffffffffa3b16a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff987255d27be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff987255d27bb0 RCX: 0000000000000001
RDX: ffffffffa42137d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff987255d27c30 R08: ffffffffa42137c8 R09: 00003ffffffff000
R10: ffff987835a62bd0 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000300000000 R14: ffff987d33a5acc0 R15: ffffffffa3bc862f
FS:  00007f714aaab780(0000) GS:ffff987d33a40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f71072723f0 CR3: 000000079430a000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa3a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa3a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa3a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa3b16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa3d86a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffa3b170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffa3a7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa3a7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa3bee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa3bf013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa3bf0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffa3bf372a>] zap_page_range+0xea/0x130
[<ffffffffa3bf677d>] ? handle_mm_fault+0xbfd/0xfb0
[<ffffffffa3c4daa3>] ? do_sync_write+0x93/0xe0
[<ffffffffa3bedc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffa3b5729b>] ? __seccomp_filter+0x5b/0x300
[<ffffffffa4196226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 60 15 75 a4 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #198925
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff87916842
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8787e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8787e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8787e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8791695f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff87b86395
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff87916f0b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8787e868
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8787e8db
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff879edf56
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff879efa1f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff879efb43
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff879f300a
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xea
14
? do_mmap
vmlinux 0xffffffff879fcc4c
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x39c
15
SyS_madvise
vmlinux 0xffffffff879ed4fd
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x49d
16
? __seccomp_filter
vmlinux 0xffffffff879570ab
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x5b
17
? system_call_after_swapgs
vmlinux 0xffffffff87f95ed5
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffff87f95ec9
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x96
19
tracesys
vmlinux 0xffffffff87f96226
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 21s! [hz.ReferenceDat:31769]
Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_REDIRECT nf_nat_redirect vxlan ip6_udp_tunnel udp_tunnel xt_mark xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM tun devlink rpcsec_gss_krb5 cmac auth_rpcgss nfsv4 arc4 md4 nfs lockd nls_utf8 grace cifs fscache ccm dns_resolver ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash
dm_log dm_mod nfit libnvdimm ppdev snd_intel8x0 iosf_mbi crc32_pclmul snd_ac97_codec ac97_bus ghash_clmulni_intel snd_seq snd_seq_device snd_pcm joydev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg snd_timer parport_pc snd parport i2c_piix4 pcspkr video soundcore ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul e1000 crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks fuse
CPU: 8 PID: 31769 Comm: hz.ReferenceDat Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8ae17f832100 ti: ffff8adfbcac4000 task.ti: ffff8adfbcac4000
RIP: 0010:[<ffffffff87916842>]  [<ffffffff87916842>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8adfbcac7be0  EFLAGS: 00000202
RAX: 000000000000000c RBX: ffff8adfbcac7bb0 RCX: 0000000000000830
RDX: 000000000000000c RSI: 000000000000000c RDI: 0000000000000282
RBP: ffff8adfbcac7c30 R08: ffffffff88015340 R09: 00003ffffffff000
R10: ffff8ad4438dca20 R11: 00007f3e190db000 R12: 00000000c57c7c00
R13: ffffd49b5038eaf0 R14: 00007f3e190db000 R15: 0000000100400030
FS:  00007f3e191d7700(0000) GS:ffff8ae398e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0c24336000 CR3: 0000000167790000 CR4: 00000000000606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8787e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8787e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8787e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8791695f>] smp_call_function_single+0x5f/0xa0
[<ffffffff87b86395>] ? cpumask_next_and+0x35/0x50
[<ffffffff87916f0b>] smp_call_function_many+0x22b/0x270
[<ffffffff8787e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8787e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff879edf56>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff879efa1f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff879efb43>] tlb_finish_mmu+0x23/0x30
[<ffffffff879f300a>] zap_page_range+0xea/0x130
[<ffffffff879fcc4c>] ? do_mmap+0x39c/0x590
[<ffffffff879ed4fd>] SyS_madvise+0x49d/0xac0
[<ffffffff879570ab>] ? __seccomp_filter+0x5b/0x300
[<ffffffff87f95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff87f95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff87f96226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 a0 15 55 88 48 89 df e8 c7 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #172876
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb6d16842
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb6c7e6a0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb6c7e6a0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb6c7e6a0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb6d1695f
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb6f86395
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb6d16f0b
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb6c7e868
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb6c7e8db
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb6dedf56
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb6defa1f
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb6defb43
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffb6df300a
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffb6ded4fd
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x49d
15
? poll_select_copy_remaining
vmlinux 0xffffffffb6e643e4
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x144
16
system_call_fastpath
vmlinux 0xffffffffb7395f92
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [dockerd-current:1311]
Modules linked in: tcp_lp xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 snd_rawmidi ghash_clmulni_intel snd_ac97_codec ac97_bus ppdev snd_seq snd_seq_device aesni_intel lrw vmw_balloon gf128mul glue_helper ablk_helper cryptd snd_pcm joydev pcspkr snd_timer snd soundcore sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper
nfit syscopyarea sysfillrect libnvdimm sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih crct10dif_pclmul crct10dif_common crc32c_intel e1000 libata mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 1311 Comm: dockerd-current Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffffa08170612100 ti: ffffa081f0274000 task.ti: ffffa081f0274000
RIP: 0010:[<ffffffffb6d16842>]  [<ffffffffb6d16842>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffffa081f0277be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffffa081f0277bb0 RCX: 0000000000000001
RDX: ffffffffb74137d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffffa081f0277c30 R08: ffffffffb74137c8 R09: 00003ffffffff000
R10: ffffa081207e1878 R11: 0000000000000246 R12: ffffffffb6dc8cc6
R13: ffffa081f0277bf0 R14: 0000000000020200 R15: 0000000000000000
FS:  00007fb213358700(0000) GS:ffffa081f9680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f855e0b5000 CR3: 00000000b4af4000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb6c7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb6c7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb6c7e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb6d1695f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb6f86395>] ? cpumask_next_and+0x35/0x50
[<ffffffffb6d16f0b>] smp_call_function_many+0x22b/0x270
[<ffffffffb6c7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb6c7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb6dedf56>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb6defa1f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb6defb43>] tlb_finish_mmu+0x23/0x30
[<ffffffffb6df300a>] zap_page_range+0xea/0x130
[<ffffffffb6ded4fd>] SyS_madvise+0x49d/0xac0
[<ffffffffb6e643e4>] ? poll_select_copy_remaining+0x144/0x180
[<ffffffffb7395f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 a0 15 95 b7 48 89 df e8 c7 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #152166
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff89116842
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8907e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8907e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8907e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8911695f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff89386395
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff89116f0b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8907e868
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8907e8db
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff891edf56
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff891efa1f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff891efb43
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff891f300a
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xea
14
? do_mmap
vmlinux 0xffffffff891fcc4c
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x39c
15
SyS_madvise
vmlinux 0xffffffff891ed4fd
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x49d
16
? __seccomp_filter
vmlinux 0xffffffff891570ab
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x5b
17
tracesys
vmlinux 0xffffffff89796226
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [globalEventExec:25734]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw sg virtio_balloon gf128mul glue_helper ablk_helper cryptd parport_pc joydev pcspkr parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console
net_failover virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci floppy virtio_ring serio_raw virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 25734 Comm: globalEventExec Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996)/Standard PC (i440FX + PIIX, 1996), BIOS  
task: ffff9c2f68edd280 ti: ffff9c2f68fbc000 task.ti: ffff9c2f68fbc000
RIP: 0010:[<ffffffff89116842>]  [<ffffffff89116842>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c2f68fbfbe0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9c2f68fbfbb0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff9c2f68fbfc30 R08: ffffffff898146c0 R09: 00003ffffffff000
R10: ffff9c2ff83de288 R11: 00007f11ea177000 R12: 000000007fc9ad40
R13: ffffc8ee02d47b70 R14: 00007f11ea177000 R15: ffff9c2f68edd280
FS:  00007f11ea274700(0000) GS:ffff9c307fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f11ea25f948 CR3: 0000000028c9e000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8907e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8907e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8907e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8911695f>] smp_call_function_single+0x5f/0xa0
[<ffffffff89386395>] ? cpumask_next_and+0x35/0x50
[<ffffffff89116f0b>] smp_call_function_many+0x22b/0x270
[<ffffffff8907e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8907e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff891edf56>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff891efa1f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff891efb43>] tlb_finish_mmu+0x23/0x30
[<ffffffff891f300a>] zap_page_range+0xea/0x130
[<ffffffff891fcc4c>] ? do_mmap+0x39c/0x590
[<ffffffff891ed4fd>] SyS_madvise+0x49d/0xac0
[<ffffffff891570ab>] ? __seccomp_filter+0x5b/0x300
[<ffffffff89796226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 a0 15 d5 89 48 89 df e8 c7 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #81233
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff977167a2
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9767e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff9767e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff9767e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff977168bf
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff979862c5
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff97716e6b
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9767e868
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9767e8db
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff977ede86
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff977ef94f
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff977efa73
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff977f2f3a
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xea
14
? futex_wake
vmlinux 0xffffffff97712280
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x90
15
SyS_madvise
vmlinux 0xffffffff977ed42d
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x49d
16
? __seccomp_filter
vmlinux 0xffffffff97756ffb
Build id: 3.10.0-1160.15.2.el7.x86_64 		0x5b
17
tracesys
vmlinux 0xffffffff97d95226
Build id: 3.10.0-1160.15.2.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [Compositor:15914]
Modules linked in: tcp_lp rfcomm xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc des_generic md4 nls_utf8 cifs ccm dns_resolver overlay(T) vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event squashfs loop snd_ens1371 ppdev iosf_mbi snd_rawmidi snd_ac97_codec crc32_pclmul ac97_bus ghash_clmulni_intel vmw_balloon snd_seq btusb snd_seq_device btrtl btbcm aesni_intel btintel snd_pcm lrw bluetooth gf128mul glue_helper ablk_helper cryptd sg rfkill joydev snd_timer pcspkr
snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit crct10dif_pclmul libnvdimm crct10dif_common crc32c_intel drm ata_piix mptspi libata e1000 serio_raw scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 15914 Comm: Compositor Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffffa0f7e1b84200 ti: ffffa100dabf8000 task.ti: ffffa100dabf8000
RIP: 0010:[<ffffffff977167a2>]  [<ffffffff977167a2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffffa100dabfbbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffffa100dabfbbb0 RCX: 0000000000000006
RDX: ffffffff97e14450 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffffa100dabfbc30 R08: ffffffff97e14448 R09: 00003ffffffff000
R10: ffffa1000b43b878 R11: 0000000000000206 R12: ffffa0f7b5cf8c00
R13: ffffa0f7e1b84268 R14: ffffffff976e4999 R15: ffffa100dabfbb60
FS:  00007fd6d45bf700(0000) GS:ffffa10573b00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb0b791f000 CR3: 000000093a5ce000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9767e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9767e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9767e6a0>] ? leave_mm+0x120/0x120
[<ffffffff977168bf>] smp_call_function_single+0x5f/0xa0
[<ffffffff979862c5>] ? cpumask_next_and+0x35/0x50
[<ffffffff97716e6b>] smp_call_function_many+0x22b/0x270
[<ffffffff9767e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9767e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff977ede86>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff977ef94f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff977efa73>] tlb_finish_mmu+0x23/0x30
[<ffffffff977f2f3a>] zap_page_range+0xea/0x130
[<ffffffff97712280>] ? futex_wake+0x90/0x180
[<ffffffff977ed42d>] SyS_madvise+0x49d/0xac0
[<ffffffff97756ffb>] ? __seccomp_filter+0x5b/0x300
[<ffffffff97d95226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 e0 13 35 98 48 89 df e8 97 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #60038
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff87116802
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8707e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8707e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8707e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8711691f
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff87385fe5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff87116ecb
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8707e868
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8707e8db
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff871edee6
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff871ef9af
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff871efad3
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff871f2f9a
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff871ed48d
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x49d
15
? __seccomp_filter
vmlinux 0xffffffff8715705b
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x5b
16
tracesys
vmlinux 0xffffffff87794226
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [event-12-thread:16014]
Modules linked in: xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop overlay iptable_filter ext4 mbcache jbd2 snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core iosf_mbi snd_hwdep crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel ppdev snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_timer joydev pcspkr virtio_balloon snd parport_pc parport soundcore i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi virtio_console virtio_blk qxl drm_kms_helper ahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libahci ata_piix drm libata
e1000 crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci floppy serio_raw virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 16014 Comm: event-12-thread Kdump: loaded Tainted: G             L ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9c52928b4200 ti: ffff9c510ae90000 task.ti: ffff9c510ae90000
RIP: 0010:[<ffffffff87116802>]  [<ffffffff87116802>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c510ae93be0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff9c510ae93bb0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff9c510ae93c30 R08: ffffffff87813f40 R09: 00003ffffffff000
R10: ffff9c5279b9f798 R11: 00007efd66725000 R12: ffffffff871c8c76
R13: ffff9c510ae93bf0 R14: 0000000000020200 R15: 0000000000000000
FS:  00007efd66822700(0000) GS:ffff9c533fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4d70234504 CR3: 00000000a5c80000 CR4: 00000000003606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8707e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8707e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8707e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8711691f>] smp_call_function_single+0x5f/0xa0
[<ffffffff87385fe5>] ? cpumask_next_and+0x35/0x50
[<ffffffff87116ecb>] smp_call_function_many+0x22b/0x270
[<ffffffff8707e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8707e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff871edee6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff871ef9af>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff871efad3>] tlb_finish_mmu+0x23/0x30
[<ffffffff871f2f9a>] zap_page_range+0xea/0x130
[<ffffffff871ed48d>] SyS_madvise+0x49d/0xac0
[<ffffffff8715705b>] ? __seccomp_filter+0x5b/0x300
[<ffffffff87794226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 e0 13 d5 87 48 89 df e8 57 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1058232
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb091db12
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb0881b10
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x130
3
? leave_mm
vmlinux 0xffffffffb0881b10
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x130
4
? leave_mm
vmlinux 0xffffffffb0881b10
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x130
5
smp_call_function_single
vmlinux 0xffffffffb091dc2f
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb0b9c485
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb091e1eb
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb0881fa8
Build id: 3.10.0-1160.90.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb088201b
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb09fa00e
Build id: 3.10.0-1160.90.1.el7.x86_64 		0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb09fbb8f
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb09fbcc3
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffb09ff20a
Build id: 3.10.0-1160.90.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffb09f95fd
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x49d
15
? putname
vmlinux 0xffffffffb0a6e0ed
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x3d
16
system_call_fastpath
vmlinux 0xffffffffb0fc539a
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [ExterLdrReload:3149]
Modules linked in: xt_CHECKSUM xt_comment ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables cmac arc4 md4 nls_utf8 cifs ccm dns_resolver devlink tun binfmt_misc bridge stp llc ip_set vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon sg pcspkr joydev vmw_vmci i2c_piix4 parport_pc parport
xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci sd_mod crc_t10dif drm libahci crct10dif_generic ata_piix nfit libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks vmxnet3 vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ip_tables]
CPU: 3 PID: 3149 Comm: ExterLdrReload Kdump: loaded Tainted: G             L ------------   3.10.0-1160.90.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9604ba4f6300 ti: ffff9607edd74000 task.ti: ffff9607edd74000
RIP: 0010:[<ffffffffb091db12>]  [<ffffffffb091db12>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9607edd77be0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9607edd77bb0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff9607edd77c30 R08: ffffffffb1013b40 R09: 00003ffffffff000
R10: ffff960434bc11b8 R11: 0000000000000206 R12: ffffffffc07002a9
R13: ffff9607edd77b88 R14: ffff960829f7b840 R15: ffff9607edd77bc8
FS:  00007f2cc3588700(0000) GS:ffff96083fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f99ecd82000 CR3: 0000000410c4e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb0881b10>] ? leave_mm+0x130/0x130
[<ffffffffb0881b10>] ? leave_mm+0x130/0x130
[<ffffffffb0881b10>] ? leave_mm+0x130/0x130
[<ffffffffb091dc2f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb0b9c485>] ? cpumask_next_and+0x35/0x50
[<ffffffffb091e1eb>] smp_call_function_many+0x22b/0x270
[<ffffffffb0881fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb088201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb09fa00e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffb09fbb8f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb09fbcc3>] tlb_finish_mmu+0x23/0x40
[<ffffffffb09ff20a>] zap_page_range+0xea/0x140
[<ffffffffb09f95fd>] SyS_madvise+0x49d/0xac0
[<ffffffffb0a6e0ed>] ? putname+0x3d/0x70
[<ffffffffb0fc539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 55 b1 48 89 df e8 c7 66 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1014254
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8511db12
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff85081b10
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x130
3
? leave_mm
vmlinux 0xffffffff85081b10
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x130
4
? leave_mm
vmlinux 0xffffffff85081b10
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x130
5
smp_call_function_single
vmlinux 0xffffffff8511dc2f
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8539c485
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8511e1eb
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff85081fa8
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8508201b
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff851fa00e
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffff851fbb8f
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff851fbcc3
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff851ff20a
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xea
14
? cp_new_stat
vmlinux 0xffffffff85261c35
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x165
15
SyS_madvise
vmlinux 0xffffffff851f95fd
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x49d
16
system_call_fastpath
vmlinux 0xffffffff857c539a
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [grafana-server:9281]
Modules linked in: tcp_diag inet_diag tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi snd_ens1371 crc32_pclmul snd_rawmidi ppdev snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq vmw_balloon aesni_intel snd_seq_device lrw gf128mul glue_helper ablk_helper snd_pcm cryptd snd_timer vmw_vmci snd joydev pcspkr soundcore sg i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt nfit fb_sys_fops ttm libnvdimm drm ata_piix mptspi libata crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel mptscsih e1000 mptbase serio_raw drm_panel_orientation_quirks fuse
CPU: 3 PID: 9281 Comm: grafana-server Kdump: loaded Tainted: G             L ------------   3.10.0-1160.92.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff88e2c9505280 ti: ffff88dc90088000 task.ti: ffff88dc90088000
RIP: 0010:[<ffffffff8511db12>]  [<ffffffff8511db12>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff88dc9008bbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88dc9008bbb0 RCX: 0000000000000002
RDX: ffffffff85813bd0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff88dc9008bc30 R08: ffffffff85813bc8 R09: 00003ffffffff000
R10: ffff88e237fae798 R11: 0000000000000246 R12: 0000964548b9d6f0
R13: ffffffff850e8cc4 R14: ffff88dc9008bb80 R15: 0000000000000087
FS:  00007f86e35fe700(0000) GS:ffff88e2dd6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe5f4e60000 CR3: 00000007f6694000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff85081b10>] ? leave_mm+0x130/0x130
[<ffffffff85081b10>] ? leave_mm+0x130/0x130
[<ffffffff85081b10>] ? leave_mm+0x130/0x130
[<ffffffff8511dc2f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8539c485>] ? cpumask_next_and+0x35/0x50
[<ffffffff8511e1eb>] smp_call_function_many+0x22b/0x270
[<ffffffff85081fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8508201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff851fa00e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff851fbb8f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff851fbcc3>] tlb_finish_mmu+0x23/0x40
[<ffffffff851ff20a>] zap_page_range+0xea/0x140
[<ffffffff85261c35>] ? cp_new_stat+0x165/0x1a0
[<ffffffff851f95fd>] SyS_madvise+0x49d/0xac0
[<ffffffff857c539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 d5 85 48 89 df e8 c7 66 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #889890
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8a31db82
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8a281b10
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x130
3
? leave_mm
vmlinux 0xffffffff8a281b10
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x130
4
? leave_mm
vmlinux 0xffffffff8a281b10
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x130
5
smp_call_function_single
vmlinux 0xffffffff8a31dc9f
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8a59c5d5
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8a31e25b
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8a281fa8
Build id: 3.10.0-1160.99.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8a28201b
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8a3fa07e
Build id: 3.10.0-1160.99.1.el7.x86_64 		0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffff8a3fbbff
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff8a3fbd33
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff8a3ff27a
Build id: 3.10.0-1160.99.1.el7.x86_64 		0xea
14
? __slab_free
vmlinux 0xffffffff8a4332ac
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x9c
15
? cp_new_stat
vmlinux 0xffffffff8a461ca5
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x165
16
SyS_madvise
vmlinux 0xffffffff8a3f966d
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x49d
17
system_call_fastpath
vmlinux 0xffffffff8a9c539a
Build id: 3.10.0-1160.99.1.el7.x86_64 		0x25 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [promtail-linux-:1046]
Modules linked in: vfat fat ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c iptable_filter dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper parport_pc cryptd pcspkr i2c_piix4 parport virtio_balloon joydev ip_tables ext4 mbcache jbd2 ata_generic pata_acpi virtio_net virtio_console net_failover virtio_blk failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common drm_panel_orientation_quirks crc32c_intel floppy virtio_pci serio_raw virtio_ring virtio
CPU: 0 PID: 1046 Comm: promtail-linux- Tainted: G             L ------------   3.10.0-1160.99.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
task: ffffa0f6b6258000 ti: ffffa0f6b5eb8000 task.ti: ffffa0f6b5eb8000
RIP: 0010:[<ffffffff8a31db82>]  [<ffffffff8a31db82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffffa0f6b5ebbbe0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffffa0f6b5ebbbb0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffffa0f6b5ebbc30 R08: ffffffff8aa13e40 R09: 00003ffffffff000
R10: ffffa0f9b0698510 R11: fff00000f0000000 R12: ffffffff8a434939
R13: ffffa0f6b5ebbc18 R14: ffffeda48b3a53c0 R15: 0000000000000000
FS:  00007f680affd700(0000) GS:ffffa0f9bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa3d9824ce8 CR3: 000000032dda2000 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8a281b10>] ? leave_mm+0x130/0x130
[<ffffffff8a281b10>] ? leave_mm+0x130/0x130
[<ffffffff8a281b10>] ? leave_mm+0x130/0x130
[<ffffffff8a31dc9f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8a59c5d5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8a31e25b>] smp_call_function_many+0x22b/0x270
[<ffffffff8a281fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8a28201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8a3fa07e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff8a3fbbff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8a3fbd33>] tlb_finish_mmu+0x23/0x40
[<ffffffff8a3ff27a>] zap_page_range+0xea/0x140
[<ffffffff8a4332ac>] ? __slab_free+0x9c/0x2b0
[<ffffffff8a461ca5>] ? cp_new_stat+0x165/0x1a0
[<ffffffff8a3f966d>] SyS_madvise+0x49d/0xac0
[<ffffffff8a9c539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 40 19 f5 8a 48 89 df e8 97 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00