Info

Permalink
Function
smp_call_function_many
First occurrence
2020-09-24
Last occurrence
2026-02-14
State
NEW
Components
kernel
Quality
-19

Statistics

Operating system Count
CentOS 7 1,091
Architecture Count
x86_64 1,091
Related packages Count
kernel 1,091
 0:3.10.0-957.el7 286
 0:3.10.0-1062.12.1.el7 271
 0:3.10.0-1062.el7 197
 0:3.10.0-1160.el7 174
 0:3.10.0-1160.36.2.el7 42
 0:3.10.0-1127.19.1.el7 26
 0:3.10.0-1160.108.1.el7 9
 0:3.10.0-1160.88.1.el7 9
 0:3.10.0-1160.2.2.el7 7
 0:3.10.0-1160.119.1.el7 6
 0:3.10.0-1160.81.1.el7 6
 0:3.10.0-1160.90.1.el7 6
 0:3.10.0-862.14.4.el7 6
 0:3.10.0-1127.el7 5
 0:3.10.0-957.27.2.el7 5
 0:3.10.0-1160.102.1.el7 4
 0:3.10.0-957.5.1.el7 4
 0:3.10.0-957.10.1.el7 4
 0:3.10.0-1127.10.1.el7 4
 0:3.10.0-1160.11.1.el7 3
 0:3.10.0-1160.49.1.el7 3
 0:3.10.0-957.21.3.el7 2
 0:3.10.0-1160.92.1.el7 2
 0:3.10.0-1160.42.2.el7 2
 0:3.10.0-1062.18.1.el7 1
 0:3.10.0-1062.4.1.el7 1
 0:3.10.0-1160.71.1.el7 1
 0:3.10.0-1160.59.1.el7 1
 0:3.10.0-1160.66.1.el7 1
 0:3.10.0-1160.6.1.el7 1
 0:3.10.0-1160.15.2.el7 1
 0:3.10.0-1160.25.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #922808
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffaa1170da
Build id: 3.10.0-1160.49.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffaa07e258
Build id: 3.10.0-1160.49.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffaa07e2cb
Build id: 3.10.0-1160.49.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffffaa1feaed
Build id: 3.10.0-1160.49.1.el7.x86_64
0x5dd
5
mprotect_fixup
vmlinux 0xffffffffaa1fecf3
Build id: 3.10.0-1160.49.1.el7.x86_64
0x163
6
? call_rwsem_down_write_failed
vmlinux 0xffffffffaa3981b7
Build id: 3.10.0-1160.49.1.el7.x86_64
0x17
7
do_mprotect_pkey
vmlinux 0xffffffffaa1ff16e
Build id: 3.10.0-1160.49.1.el7.x86_64
0x23e
8
SyS_mprotect
vmlinux 0xffffffffaa1ff2c3
Build id: 3.10.0-1160.49.1.el7.x86_64
0x13
9
tracesys
vmlinux 0xffffffffaa796226
Build id: 3.10.0-1160.49.1.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [VM Thread:7674]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_comment devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter tun vmw_vsock_vmci_transport vsock sunrpc vfat fat ppdev vmw_balloon sb_edac iosf_mbi crc32_pclmul nfit ghash_clmulni_intel libnvdimm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr vmw_vmci i2c_piix4 tpm_crb parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata drm e1000 mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi mptscsih serio_raw crc32c_intel mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ipmi_msghandler]
CPU: 6 PID: 7674 Comm: VM Thread Tainted: G        W    L ------------   3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.16707776.B64.2008070230 08/07/2020
task: ffff98867667c200 ti: ffff9885b6710000 task.ti: ffff9885b6710000
RIP: 0010:[<ffffffffaa1170da>]  [<ffffffffaa1170da>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9885b6713c70  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00080000aa07ddd0 RCX: ffff988b3fca06c8
RDX: 0000000000000002 RSI: 0000000000000007 RDI: 0000000000000000
RBP: ffff9885b6713ca8 R08: ffff988265625c00 R09: ffffffffaa386bf9
R10: ffff988b3fd9f160 R11: fffff1d2c03e7b00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9885b6713c20 R15: 0000000000000006
FS:  00007f5aa585f700(0000) GS:ffff988b3fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0003a9000 CR3: 0000000a11d2e000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaa07e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaa07e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffaa1feaed>] change_protection+0x5dd/0x680
[<ffffffffaa1fecf3>] mprotect_fixup+0x163/0x3a0
[<ffffffffaa3981b7>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffffaa1ff16e>] do_mprotect_pkey+0x23e/0x380
[<ffffffffaa1ff2c3>] SyS_mprotect+0x13/0x20
[<ffffffffaa796226>] tracesys+0xa6/0xcc
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 d5 aa f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #902257
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8611e1ca
Build id: 3.10.0-1160.88.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff86081fa8
Build id: 3.10.0-1160.88.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff8608201b
Build id: 3.10.0-1160.88.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff8620a99d
Build id: 3.10.0-1160.88.1.el7.x86_64
0x5dd
5
? hrtimer_get_res
vmlinux 0xffffffff860cf340
Build id: 3.10.0-1160.88.1.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffff8620aba3
Build id: 3.10.0-1160.88.1.el7.x86_64
0x163
7
? call_rwsem_down_write_failed
vmlinux 0xffffffff863ae417
Build id: 3.10.0-1160.88.1.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffff8620b02e
Build id: 3.10.0-1160.88.1.el7.x86_64
0x23e
9
SyS_mprotect
vmlinux 0xffffffff8620b183
Build id: 3.10.0-1160.88.1.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffff867c539a
Build id: 3.10.0-1160.88.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [java:3249]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq iosf_mbi snd_seq_device crc32_pclmul ghash_clmulni_intel snd_pcm aesni_intel lrw snd_timer gf128mul snd glue_helper soundcore ablk_helper cryptd sg ppdev pcspkr joydev vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod
cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea libata sysfillrect sysimgblt fb_sys_fops ttm nfit drm libnvdimm e1000 mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi mptscsih crc32c_intel mptbase serio_raw drm_panel_orientation_quirks fuse
CPU: 3 PID: 3249 Comm: java Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9b63e6ff6300 ti: ffff9b6610158000 task.ti: ffff9b6610158000
RIP: 0010:[<ffffffff8611e1ca>]  [<ffffffff8611e1ca>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9b661015bc70  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 0008000086081b10 RCX: ffff9b67ed6a0650
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9b661015bca8 R08: ffff9b6535ccb800 R09: ffffffff8639c669
R10: ffff9b67ed6df160 R11: fffff5ba800d6c00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9b661015bc20 R15: 0000000000000001
FS:  00007fc791051700(0000) GS:ffff9b67ed6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e824cda2f8 CR3: 0000000158e28000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff86081fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8608201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8620a99d>] change_protection+0x5dd/0x680
[<ffffffff860cf340>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8620aba3>] mprotect_fixup+0x163/0x3b0
[<ffffffff863ae417>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff8620b02e>] do_mprotect_pkey+0x23e/0x380
[<ffffffff8620b183>] SyS_mprotect+0x13/0x20
[<ffffffff867c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 7e 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 00 19 d5 86 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 4d 18 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #812851
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff92d1e1ca
Build id: 3.10.0-1160.90.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff92c81fa8
Build id: 3.10.0-1160.90.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff92c8201b
Build id: 3.10.0-1160.90.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff92e0a99d
Build id: 3.10.0-1160.90.1.el7.x86_64
0x5dd
5
mprotect_fixup
vmlinux 0xffffffff92e0aba3
Build id: 3.10.0-1160.90.1.el7.x86_64
0x163
6
? __seccomp_filter
vmlinux 0xffffffff92d5f8cb
Build id: 3.10.0-1160.90.1.el7.x86_64
0x5b
7
? call_rwsem_down_write_failed
vmlinux 0xffffffff92fae417
Build id: 3.10.0-1160.90.1.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffff92e0b02e
Build id: 3.10.0-1160.90.1.el7.x86_64
0x23e
9
SyS_mprotect
vmlinux 0xffffffff92e0b183
Build id: 3.10.0-1160.90.1.el7.x86_64
0x13
10
tracesys
vmlinux 0xffffffff933c562e
Build id: 3.10.0-1160.90.1.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [java:33318]
Modules linked in: binfmt_misc xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ppdev joydev virtio_balloon sg parport_pc parport pcspkr i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover virtio_console virtio_scsi virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci virtio_ring serio_raw floppy virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 33318 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.90.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9721b214b180 ti: ffff9720989d0000 task.ti: ffff9720989d0000
RIP: 0010:[<ffffffff92d1e1ca>]  [<ffffffff92d1e1ca>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9720989d3c70  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 0008000092c81b10 RCX: ffff97269ce604f0
RDX: 0000000000000001 RSI: 00000000000000a0 RDI: 0000000000000000
RBP: ffff9720989d3ca8 R08: ffff971fec4d4800 R09: ffffffff92f9c669
R10: ffff97269cfdf160 R11: ffffddbcd326e800 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9720989d3c20 R15: 0000000000000001
FS:  00007f89a090e700(0000) GS:ffff97269cfc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f720ce8e160 CR3: 0000000721094000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff92c81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff92c8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff92e0a99d>] change_protection+0x5dd/0x680
[<ffffffff92e0aba3>] mprotect_fixup+0x163/0x3b0
[<ffffffff92d5f8cb>] ? __seccomp_filter+0x5b/0x310
[<ffffffff92fae417>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff92e0b02e>] do_mprotect_pkey+0x23e/0x380
[<ffffffff92e0b183>] SyS_mprotect+0x13/0x20
[<ffffffff933c562e>] tracesys+0xa6/0xcc
Code: 48 63 35 7e 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 00 19 95 93 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 4d 18 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #610558
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9551173a
Build id: 3.10.0-957.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9547a598
Build id: 3.10.0-957.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff9547a608
Build id: 3.10.0-957.el7.x86_64
0x68
4
change_protection
vmlinux 0xffffffff955f404d
Build id: 3.10.0-957.el7.x86_64
0x5cd
5
mprotect_fixup
vmlinux 0xffffffff955f4262
Build id: 3.10.0-957.el7.x86_64
0x162
6
? call_rwsem_down_write_failed
vmlinux 0xffffffff957869d7
Build id: 3.10.0-957.el7.x86_64
0x17
7
do_mprotect_pkey
vmlinux 0xffffffff955f46b2
Build id: 3.10.0-957.el7.x86_64
0x252
8
SyS_mprotect
vmlinux 0xffffffff955f47d3
Build id: 3.10.0-957.el7.x86_64
0x13
9
system_call_fastpath
vmlinux 0xffffffff95b74ddb
Build id: 3.10.0-957.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 25s! [java:17145]
Modules linked in: vmw_vsock_vmci_transport vsock vfat fat sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon joydev sg pcspkr parport_pc parport tpm_crb vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw e1000e ahci libahci ata_piix drm mptsas scsi_transport_sas mptscsih libata ptp pps_core nfit mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 17145 Comm: java Kdump: loaded Tainted: G        W      ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.17369862.B64.2012240522 12/24/2020
task: ffff97eefea0e180 ti: ffff97ef86bb0000 task.ti: ffff97ef86bb0000
RIP: 0010:[<ffffffff9551173a>]  [<ffffffff9551173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff97ef86bb3c70  EFLAGS: 00000202
RAX: 000000000000000e RBX: 000800009547a3d0 RCX: ffff97ef9d9a06a0
RDX: 000000000000000e RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff97ef86bb3ca8 R08: ffff97e8f450f400 R09: ffffffff957755c9
R10: ffff97ef9d7df160 R11: fffff8ec1f42fa00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff97ef86bb3c20 R15: ffffffff9542a621
FS:  00007f6111fff700(0000) GS:ffff97ef9d7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd2dc32ea48 CR3: 00000007893b2000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9547a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9547a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff955f404d>] change_protection+0x5cd/0x680
[<ffffffff955f4262>] mprotect_fixup+0x162/0x360
[<ffffffff957869d7>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff955f46b2>] do_mprotect_pkey+0x252/0x360
[<ffffffff955f47d3>] SyS_mprotect+0x13/0x20
[<ffffffff95b74ddb>] system_call_fastpath+0x22/0x27
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 14 96 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #595152
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffbbb170da
Build id: 3.10.0-1160.42.2.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffbba7e258
Build id: 3.10.0-1160.42.2.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffbba7e2cb
Build id: 3.10.0-1160.42.2.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffffbbbfeaed
Build id: 3.10.0-1160.42.2.el7.x86_64
0x5dd
5
mprotect_fixup
vmlinux 0xffffffffbbbfecf3
Build id: 3.10.0-1160.42.2.el7.x86_64
0x163
6
? __seccomp_filter
vmlinux 0xffffffffbbb5729b
Build id: 3.10.0-1160.42.2.el7.x86_64
0x5b
7
? call_rwsem_down_write_failed
vmlinux 0xffffffffbbd98197
Build id: 3.10.0-1160.42.2.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffffbbbff16e
Build id: 3.10.0-1160.42.2.el7.x86_64
0x23e
9
SyS_mprotect
vmlinux 0xffffffffbbbff2c3
Build id: 3.10.0-1160.42.2.el7.x86_64
0x13
10
tracesys
vmlinux 0xffffffffbc196226
Build id: 3.10.0-1160.42.2.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [java:3208]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi kvm_intel kvm snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq irqbypass snd_seq_device snd_pcm crc32_pclmul ghash_clmulni_intel snd_timer aesni_intel snd lrw soundcore gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon vmw_vmci sg joydev i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel nfit e1000 mptspi scsi_transport_spi libnvdimm mptscsih serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 3208 Comm: java Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.42.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8a77a26e8000 ti: ffff8a779ca2c000 task.ti: ffff8a779ca2c000
RIP: 0010:[<ffffffffbbb170da>]  [<ffffffffbbb170da>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8a779ca2fc70  EFLAGS: 00000202
RAX: 0000000000000004 RBX: 00080000bba7ddd0 RCX: ffff8a77c9720628
RDX: 0000000000000004 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8a779ca2fca8 R08: ffff8a75b70cd400 R09: ffffffffbbd86bd9
R10: ffff8a77c969f160 R11: ffffc79f012c4400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8a779ca2fc20 R15: ffff8a74f4095800
FS:  00007f3fbcd28700(0000) GS:ffff8a77c9680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fef675dce00 CR3: 0000000365358000 CR4: 00000000001607e0
Call Trace:
[<ffffffffbba7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbba7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbbbfeaed>] change_protection+0x5dd/0x680
[<ffffffffbbbfecf3>] mprotect_fixup+0x163/0x3a0
[<ffffffffbbb5729b>] ? __seccomp_filter+0x5b/0x300
[<ffffffffbbd98197>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffffbbbff16e>] do_mprotect_pkey+0x23e/0x380
[<ffffffffbbbff2c3>] SyS_mprotect+0x13/0x20
[<ffffffffbc196226>] tracesys+0xa6/0xcc
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 75 bc f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #528185
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff83d16eea
Build id: 3.10.0-1160.25.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff83c7e868
Build id: 3.10.0-1160.25.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff83c7e8db
Build id: 3.10.0-1160.25.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff83dfe3cd
Build id: 3.10.0-1160.25.1.el7.x86_64
0x5dd
5
? hrtimer_get_res
vmlinux 0xffffffff83cc99e0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffff83dfe5d3
Build id: 3.10.0-1160.25.1.el7.x86_64
0x163
7
? call_rwsem_down_write_failed
vmlinux 0xffffffff83f97b07
Build id: 3.10.0-1160.25.1.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffff83dfea4e
Build id: 3.10.0-1160.25.1.el7.x86_64
0x23e
9
SyS_mprotect
vmlinux 0xffffffff83dfeba3
Build id: 3.10.0-1160.25.1.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffff84395f92
Build id: 3.10.0-1160.25.1.el7.x86_64
0x25
11
? system_call_after_swapgs
vmlinux 0xffffffff84395ed5
Build id: 3.10.0-1160.25.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 86s! [VM Thread:9275]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink vboxsf(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc vmwgfx ttm snd_intel8x0 drm_kms_helper snd_ac97_codec syscopyarea ac97_bus sysfillrect snd_seq sysimgblt fb_sys_fops snd_seq_device ppdev snd_pcm drm snd_timer joydev virtio_net drm_panel_orientation_quirks parport_pc snd net_failover vboxguest(OE) video pcspkr
parport soundcore i2c_piix4 failover sg ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi serio_raw ahci ata_piix libahci virtio_pci e1000 libata virtio_ring crc32c_intel virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 7 PID: 9275 Comm: VM Thread Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9a7a0ebea100 ti: ffff9a79427d8000 task.ti: ffff9a79427d8000
RIP: 0010:[<ffffffff83d16eea>]  [<ffffffff83d16eea>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9a79427dbc70  EFLAGS: 00010202
RAX: 0000000000000003 RBX: 00000000000000fc RCX: ffff9a7a1fce04f0
RDX: 0000000000000003 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff9a79427dbca8 R08: ffff9a777935a000 R09: ffffffff83f86549
R10: ffff9a7a1fddf160 R11: ffffe4c0903a8200 R12: 0000000000000008
R13: 00000000000008fd R14: ffff9a7a1fddf160 R15: 0000000000000000
FS:  00007fe8b833b700(0000) GS:ffff9a7a1fdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe824c31628 CR3: 00000000a3d5c000 CR4: 00000000000206e0
Call Trace:
[<ffffffff83c7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff83c7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff83dfe3cd>] change_protection+0x5dd/0x680
[<ffffffff83cc99e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff83dfe5d3>] mprotect_fixup+0x163/0x3a0
[<ffffffff83f97b07>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff83dfea4e>] do_mprotect_pkey+0x23e/0x380
[<ffffffff83dfeba3>] SyS_mprotect+0x13/0x20
[<ffffffff84395f92>] system_call_fastpath+0x25/0x2a
[<ffffffff84395ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 48 63 35 fe 87 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 15 95 84 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 cd 87 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #387116
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff92b1185a
Build id: 3.10.0-957.10.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff92a7a5c8
Build id: 3.10.0-957.10.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff92a7a638
Build id: 3.10.0-957.10.1.el7.x86_64
0x68
4
change_protection
vmlinux 0xffffffff92bf44dd
Build id: 3.10.0-957.10.1.el7.x86_64
0x5cd
5
mprotect_fixup
vmlinux 0xffffffff92bf46f2
Build id: 3.10.0-957.10.1.el7.x86_64
0x162
6
? call_rwsem_down_write_failed
vmlinux 0xffffffff92d870a7
Build id: 3.10.0-957.10.1.el7.x86_64
0x17
7
do_mprotect_pkey
vmlinux 0xffffffff92bf4b42
Build id: 3.10.0-957.10.1.el7.x86_64
0x252
8
SyS_mprotect
vmlinux 0xffffffff92bf4c63
Build id: 3.10.0-957.10.1.el7.x86_64
0x13
9
tracesys
vmlinux 0xffffffff9317605b
Build id: 3.10.0-957.10.1.el7.x86_64
0xa3
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 67s! [java:9753]
Modules linked in: xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) ext4 mbcache jbd2 ppdev sg i2c_piix4 virtio_balloon parport_pc parport pcspkr joydev ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_net virtio_scsi bochs_drm floppy drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 9753 Comm: java Kdump: loaded Tainted: G               ------------ T 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff952c10f8d140 ti: ffff952c13e6c000 task.ti: ffff952c13e6c000
RIP: 0010:[<ffffffff92b1185a>]  [<ffffffff92b1185a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff952c13e6fc70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0008000092a7a400 RCX: ffff952c4fc20320
RDX: 0000000000000000 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff952c13e6fca8 R08: ffff9525b49a0400 R09: ffffffff92d75c99
R10: ffff952c4fd1f160 R11: fffffd459d0af800 R12: 0000000000000282
R13: 0000000000000282 R14: ffff952c13e6fc20 R15: ffffffff92a2a621
FS:  00007fb53f7ff700(0000) GS:ffff952c4fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb56da15000 CR3: 00000007cbed6000 CR4: 00000000000006e0
Call Trace:
[<ffffffff92a7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff92a7a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff92bf44dd>] change_protection+0x5cd/0x680
[<ffffffff92bf46f2>] mprotect_fixup+0x162/0x360
[<ffffffff92d870a7>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff92bf4b42>] do_mprotect_pkey+0x252/0x360
[<ffffffff92bf4c63>] SyS_mprotect+0x13/0x20
[<ffffffff9317605b>] tracesys+0xa3/0xc9
Code: 48 63 35 6e 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 b9 74 93 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 82 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #374232
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff87316a7e
Build id: 3.10.0-1062.12.1.el7.x86_64
0x20e
2
native_flush_tlb_others
vmlinux 0xffffffff8727e478
Build id: 3.10.0-1062.12.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff8727e4eb
Build id: 3.10.0-1062.12.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff873fc4dd
Build id: 3.10.0-1062.12.1.el7.x86_64
0x5cd
5
? hrtimer_get_res
vmlinux 0xffffffff872c9e30
Build id: 3.10.0-1062.12.1.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffff873fc6e3
Build id: 3.10.0-1062.12.1.el7.x86_64
0x163
7
? call_rwsem_down_write_failed
vmlinux 0xffffffff87592dc7
Build id: 3.10.0-1062.12.1.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffff873fcb72
Build id: 3.10.0-1062.12.1.el7.x86_64
0x252
9
SyS_mprotect
vmlinux 0xffffffff873fcc93
Build id: 3.10.0-1062.12.1.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffff8798dede
Build id: 3.10.0-1062.12.1.el7.x86_64
0x25
11
? system_call_after_swapgs
vmlinux 0xffffffff8798de21
Build id: 3.10.0-1062.12.1.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 23s! [java:4639]
Modules linked in: tcp_lp fuse kbase(OE) xt_NFQUEUE nfnetlink_log nfnetlink nfring(OE) vmsecmod(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev sg parport_pc pcspkr parport virtio_balloon joydev i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci virtio_ring serio_raw floppy virtio
drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmsec_nfq]
CPU: 11 PID: 4639 Comm: java Kdump: loaded Tainted: GF          OEL ------------   3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9eaebafde2a0 ti: ffff9eaeddff8000 task.ti: ffff9eaeddff8000
RIP: 0010:[<ffffffff87316a7e>]  [<ffffffff87316a7e>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff9eaeddffbc70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800008727e2b0 RCX: ffff9eaf7ce20590
RDX: 0000000000000000 RSI: 00000000000000a0 RDI: 0000000000000000
RBP: ffff9eaeddffbca8 R08: ffff9ea0e95dd400 R09: ffffffff87581839
R10: ffff9eaf7d0df160 R11: ffffdd7de406b400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9eaeddffbc20 R15: ffffffff8722b621
FS:  00007f027512d700(0000) GS:ffff9eaf7d0c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ce0037428 CR3: 0000000c01b80000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8727e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8727e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff873fc4dd>] change_protection+0x5cd/0x670
[<ffffffff872c9e30>] ? hrtimer_get_res+0x50/0x50
[<ffffffff873fc6e3>] mprotect_fixup+0x163/0x3a0
[<ffffffff87592dc7>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff873fcb72>] do_mprotect_pkey+0x252/0x360
[<ffffffff873fcc93>] SyS_mprotect+0x13/0x20
[<ffffffff8798dede>] system_call_fastpath+0x25/0x2a
[<ffffffff8798de21>] ? system_call_after_swapgs+0xae/0x146
Code: 5e c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 e8 f4 87 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 bd 5e c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 
Complete report #315641
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffaf31760a
Build id: 3.10.0-1127.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffaf27e8f8
Build id: 3.10.0-1127.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffaf27e96b
Build id: 3.10.0-1127.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffffaf3fe1dd
Build id: 3.10.0-1127.el7.x86_64
0x5cd
5
? __schedule
vmlinux 0xffffffffaf985922
Build id: 3.10.0-1127.el7.x86_64
0x402
6
mprotect_fixup
vmlinux 0xffffffffaf3fe3e3
Build id: 3.10.0-1127.el7.x86_64
0x163
7
? call_rwsem_down_write_failed
vmlinux 0xffffffffaf596677
Build id: 3.10.0-1127.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffffaf3fe85e
Build id: 3.10.0-1127.el7.x86_64
0x23e
9
SyS_mprotect
vmlinux 0xffffffffaf3fe9b3
Build id: 3.10.0-1127.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffffaf992ed2
Build id: 3.10.0-1127.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#12 stuck for 22s! [node:71368]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm crc32_pclmul ghash_clmulni_intel aesni_intel snd_timer lrw gf128mul snd glue_helper ablk_helper joydev pcspkr soundcore cryptd ppdev vmw_balloon sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper nfit syscopyarea sysfillrect
sysimgblt fb_sys_fops e1000 ata_piix ttm libata mptspi drm scsi_transport_spi mptscsih mptbase crct10dif_pclmul libnvdimm crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 12 PID: 71368 Comm: node Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8a377c0362a0 ti: ffff8a378ec80000 task.ti: ffff8a378ec80000
RIP: 0010:[<ffffffffaf31760a>]  [<ffffffffaf31760a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8a378ec83c70  EFLAGS: 00000202
RAX: 0000000000000005 RBX: 00080000af27e730 RCX: ffff8a37bb760740
RDX: 0000000000000005 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8a378ec83ca8 R08: ffff8a37ba6ddc00 R09: ffffffffaf585099
R10: ffff8a37bb91f160 R11: ffffdcbb80f09e00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8a378ec83c20 R15: ffffffffaf22b621
FS:  00007f3a393b0700(0000) GS:ffff8a37bb900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000519e90 CR3: 000000005e122000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffaf27e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaf27e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffaf3fe1dd>] change_protection+0x5cd/0x670
[<ffffffffaf985922>] ? __schedule+0x402/0x840
[<ffffffffaf3fe3e3>] mprotect_fixup+0x163/0x3a0
[<ffffffffaf596677>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffffaf3fe85e>] do_mprotect_pkey+0x23e/0x380
[<ffffffffaf3fe9b3>] SyS_mprotect+0x13/0x20
[<ffffffffaf992ed2>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 0f f5 af f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 7a c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #262319
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9d5170ba
Build id: 3.10.0-1160.36.2.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9d47e258
Build id: 3.10.0-1160.36.2.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff9d47e2cb
Build id: 3.10.0-1160.36.2.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff9d5feacd
Build id: 3.10.0-1160.36.2.el7.x86_64
0x5dd
5
? hrtimer_get_res
vmlinux 0xffffffff9d4c9a80
Build id: 3.10.0-1160.36.2.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffff9d5fecd3
Build id: 3.10.0-1160.36.2.el7.x86_64
0x163
7
? __seccomp_filter
vmlinux 0xffffffff9d55727b
Build id: 3.10.0-1160.36.2.el7.x86_64
0x5b
8
? call_rwsem_down_write_failed
vmlinux 0xffffffff9d798167
Build id: 3.10.0-1160.36.2.el7.x86_64
0x17
9
do_mprotect_pkey
vmlinux 0xffffffff9d5ff14e
Build id: 3.10.0-1160.36.2.el7.x86_64
0x23e
10
SyS_mprotect
vmlinux 0xffffffff9d5ff2a3
Build id: 3.10.0-1160.36.2.el7.x86_64
0x13
11
tracesys
vmlinux 0xffffffff9db96226
Build id: 3.10.0-1160.36.2.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [java:47926]
Modules linked in: tcp_lp rfcomm xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event crc32_pclmul ext4 ghash_clmulni_intel mbcache jbd2 snd_ens1371 snd_rawmidi ppdev snd_ac97_codec ac97_bus btusb snd_seq btrtl btbcm btintel
aesni_intel snd_seq_device vmw_balloon snd_pcm bluetooth lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr rfkill snd_timer sg snd vmw_vmci soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm ata_piix nfit e1000 mptspi scsi_transport_spi libnvdimm libata mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 47926 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff89f4f49f1080 ti: ffff89f40934c000 task.ti: ffff89f40934c000
RIP: 0010:[<ffffffff9d5170ba>]  [<ffffffff9d5170ba>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff89f40934fc70  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000c00009d47ddd0 RCX: ffff89f4f56a0650
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff89f40934fca8 R08: ffff89f43a88b800 R09: ffffffff9d786ba9
R10: ffff89f4f56df160 R11: ffffd664c52d2200 R12: 0000000000000282
R13: 0000000000000282 R14: ffff89f40934fc20 R15: 0000000000000003
FS:  00007f8fe80afb10(0000) GS:ffff89f4f56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055d31a21c22d CR3: 00000001497c8000 CR4: 0000000000360fe0
Call Trace:
[<ffffffff9d47e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9d47e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9d5feacd>] change_protection+0x5dd/0x680
[<ffffffff9d4c9a80>] ? hrtimer_get_res+0x50/0x50
[<ffffffff9d5fecd3>] mprotect_fixup+0x163/0x3a0
[<ffffffff9d55727b>] ? __seccomp_filter+0x5b/0x300
[<ffffffff9d798167>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff9d5ff14e>] do_mprotect_pkey+0x23e/0x380
[<ffffffff9d5ff2a3>] SyS_mprotect+0x13/0x20
[<ffffffff9db96226>] tracesys+0xa6/0xcc
Code: 48 63 35 ee 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 15 9e f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 bd 85 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #169269
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb0b16eaa
Build id: 3.10.0-1160.11.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffb0a7e868
Build id: 3.10.0-1160.11.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffb0a7e8db
Build id: 3.10.0-1160.11.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffffb0bfe2fd
Build id: 3.10.0-1160.11.1.el7.x86_64
0x5dd
5
? hrtimer_get_res
vmlinux 0xffffffffb0ac9ab0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffffb0bfe503
Build id: 3.10.0-1160.11.1.el7.x86_64
0x163
7
? __seccomp_filter
vmlinux 0xffffffffb0b5705b
Build id: 3.10.0-1160.11.1.el7.x86_64
0x5b
8
? call_rwsem_down_write_failed
vmlinux 0xffffffffb0d97757
Build id: 3.10.0-1160.11.1.el7.x86_64
0x17
9
do_mprotect_pkey
vmlinux 0xffffffffb0bfe97e
Build id: 3.10.0-1160.11.1.el7.x86_64
0x23e
10
SyS_mprotect
vmlinux 0xffffffffb0bfead3
Build id: 3.10.0-1160.11.1.el7.x86_64
0x13
11
tracesys
vmlinux 0xffffffffb1194226
Build id: 3.10.0-1160.11.1.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [VM Thread:63631]
Modules linked in: veth nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache overlay nf_conntrack_netbios_ns nf_conntrack_broadcast xt_nat xt_mark ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash
dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 ppdev aesni_intel snd_rawmidi lrw gf128mul snd_ac97_codec glue_helper ablk_helper ac97_bus cryptd snd_seq snd_seq_device vmw_balloon snd_pcm snd_timer sg snd pcspkr joydev soundcore vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd sch_fq_codel grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix crct10dif_pclmul nfit ttm crct10dif_common crc32c_intel serio_raw e1000 mptspi libnvdimm scsi_transport_spi drm mptscsih libata mptbase drm_panel_orientation_quirks fuse
CPU: 3 PID: 63631 Comm: VM Thread Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff8e584b686300 ti: ffff8e58099a8000 task.ti: ffff8e58099a8000
RIP: 0010:[<ffffffffb0b16eaa>]  [<ffffffffb0b16eaa>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8e58099abc70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00080000b0a7e6a0 RCX: ffff8e5875620650
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8e58099abca8 R08: ffff8e57be34b800 R09: ffffffffb0d86199
R10: ffff8e58756df160 R11: ffffdff1c2984a00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8e58099abc20 R15: ffffffffb0a2b981
FS:  00007f4f649f9700(0000) GS:ffff8e58756c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd15b57d012 CR3: 000000007b728000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb0a7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb0a7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb0bfe2fd>] change_protection+0x5dd/0x680
[<ffffffffb0ac9ab0>] ? hrtimer_get_res+0x50/0x50
[<ffffffffb0bfe503>] mprotect_fixup+0x163/0x3a0
[<ffffffffb0b5705b>] ? __seccomp_filter+0x5b/0x300
[<ffffffffb0d97757>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffffb0bfe97e>] do_mprotect_pkey+0x23e/0x380
[<ffffffffb0bfead3>] SyS_mprotect+0x13/0x20
[<ffffffffb1194226>] tracesys+0xa6/0xcc
Code: 48 63 35 7e 86 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 75 b1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 4d 86 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #168336
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff82d1173a
Build id: 3.10.0-957.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff82c7a598
Build id: 3.10.0-957.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff82c7a608
Build id: 3.10.0-957.el7.x86_64
0x68
4
change_protection
vmlinux 0xffffffff82df404d
Build id: 3.10.0-957.el7.x86_64
0x5cd
5
? hrtimer_get_res
vmlinux 0xffffffff82cc57e0
Build id: 3.10.0-957.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffff82df4262
Build id: 3.10.0-957.el7.x86_64
0x162
7
? call_rwsem_down_write_failed
vmlinux 0xffffffff82f869d7
Build id: 3.10.0-957.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffff82df46b2
Build id: 3.10.0-957.el7.x86_64
0x252
9
SyS_mprotect
vmlinux 0xffffffff82df47d3
Build id: 3.10.0-957.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffff83374ddb
Build id: 3.10.0-957.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [java:2633]
Modules linked in: kbase(OE) udp_diag tcp_diag inet_diag binfmt_misc fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd parport_pc parport virtio_balloon joydev i2c_piix4 pcspkr ip_tables ext4 mbcache jbd2 ata_generic pata_acpi
virtio_blk virtio_console virtio_net cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix floppy libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2633 Comm: java Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
task: ffff9c031c7fc100 ti: ffff9c03dfe70000 task.ti: ffff9c03dfe70000
RIP: 0010:[<ffffffff82d1173a>]  [<ffffffff82d1173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9c03dfe73c70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0008000082c7a3d0 RCX: ffff9c0e3f220320
RDX: 0000000000000000 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9c03dfe73ca8 R08: ffff9bffa94f3800 R09: ffffffff82f755c9
R10: ffff9c0e3f2df160 R11: fffff81ed35ba000 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9c03dfe73c20 R15: ffffffff82c2a621
FS:  00007f12f941b700(0000) GS:ffff9c0e3f2c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b7eabf1a0 CR3: 0000000feae54000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff82c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff82c7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff82df404d>] change_protection+0x5cd/0x680
[<ffffffff82cc57e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff82df4262>] mprotect_fixup+0x162/0x360
[<ffffffff82f869d7>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff82df46b2>] do_mprotect_pkey+0x252/0x360
[<ffffffff82df47d3>] SyS_mprotect+0x13/0x20
[<ffffffff83374ddb>] system_call_fastpath+0x22/0x27
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 83 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #55184
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff85111a8a
Build id: 3.10.0-957.21.3.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8507a6a8
Build id: 3.10.0-957.21.3.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff8507a718
Build id: 3.10.0-957.21.3.el7.x86_64
0x68
4
change_protection
vmlinux 0xffffffff851f465d
Build id: 3.10.0-957.21.3.el7.x86_64
0x5cd
5
? hrtimer_get_res
vmlinux 0xffffffff850c5950
Build id: 3.10.0-957.21.3.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffff851f4872
Build id: 3.10.0-957.21.3.el7.x86_64
0x162
7
? call_rwsem_down_write_failed
vmlinux 0xffffffff85387257
Build id: 3.10.0-957.21.3.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffff851f4cc2
Build id: 3.10.0-957.21.3.el7.x86_64
0x252
9
SyS_mprotect
vmlinux 0xffffffff851f4de3
Build id: 3.10.0-957.21.3.el7.x86_64
0x13
10
tracesys
vmlinux 0xffffffff8577606b
Build id: 3.10.0-957.21.3.el7.x86_64
0xa3
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [java:7503]
Modules linked in: udp_diag tcp_diag inet_diag veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack libcrc32c br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg vmw_vmci i2c_piix4 nfit libnvdimm parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix drm libahci crct10dif_pclmul crct10dif_common libata
crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks
CPU: 15 PID: 7503 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff93e6807d2080 ti: ffff93eb67f38000 task.ti: ffff93eb67f38000
RIP: 0010:[<ffffffff85111a8a>]  [<ffffffff85111a8a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff93eb67f3bc70  EFLAGS: 00000202
RAX: 000000000000000c RBX: 000800008507a4e0 RCX: ffff93eb7d9207e0
RDX: 000000000000000c RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff93eb67f3bca8 R08: ffff93dca9676c00 R09: ffffffff85375e39
R10: ffff93eb7d9df160 R11: ffffe6b23988a800 R12: 0000000000000282
R13: 0000000000000282 R14: ffff93eb67f3bc20 R15: ffffffff8502a981
FS:  00007f2ef4cec700(0000) GS:ffff93eb7d9c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000017e0361c4000 CR3: 0000000036a18000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff8507a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8507a718>] flush_tlb_mm_range+0x68/0x140
[<ffffffff851f465d>] change_protection+0x5cd/0x680
[<ffffffff850c5950>] ? hrtimer_get_res+0x50/0x50
[<ffffffff851f4872>] mprotect_fixup+0x162/0x360
[<ffffffff85387257>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff851f4cc2>] do_mprotect_pkey+0x252/0x360
[<ffffffff851f4de3>] SyS_mprotect+0x13/0x20
[<ffffffff8577606b>] tracesys+0xa3/0xc9
Code: 48 63 35 3e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 bc d4 85 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 0d 83 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #24019
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff86516c5a
Build id: 3.10.0-1160.6.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8647e868
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff8647e8db
Build id: 3.10.0-1160.6.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff865fe15d
Build id: 3.10.0-1160.6.1.el7.x86_64
0x5dd
5
mprotect_fixup
vmlinux 0xffffffff865fe363
Build id: 3.10.0-1160.6.1.el7.x86_64
0x163
6
? call_rwsem_down_write_failed
vmlinux 0xffffffff86797597
Build id: 3.10.0-1160.6.1.el7.x86_64
0x17
7
do_mprotect_pkey
vmlinux 0xffffffff865fe7de
Build id: 3.10.0-1160.6.1.el7.x86_64
0x23e
8
SyS_mprotect
vmlinux 0xffffffff865fe933
Build id: 3.10.0-1160.6.1.el7.x86_64
0x13
9
system_call_fastpath
vmlinux 0xffffffff86b94f92
Build id: 3.10.0-1160.6.1.el7.x86_64
0x25
10
? system_call_after_swapgs
vmlinux 0xffffffff86b94ed5
Build id: 3.10.0-1160.6.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 32s! [reporter-urepor:11742]
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 28s! [moloch-capture:44488]
Modules linked in: tcp_diag inet_diag binfmt_misc ipmi_devintf ipmi_msghandler xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat_ipv4 nf_nat overlay(T) vmw_vsock_vmci_transport vsock iptable_filter coretemp crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon sg joydev pcspkr vmw_vmci parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata mptspi scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror
dm_region_hash dm_log dm_mod fuse nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack libcrc32c br_netfilter bridge stp llc
CPU: 14 PID: 44488 Comm: moloch-capture Tainted: G             L ------------ T 3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff9b6c51d5e300 ti: ffff9b6cf48e8000 task.ti: ffff9b6cf48e8000
RIP: 0010:[<ffffffff86516c5a>]  [<ffffffff86516c5a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9b6cf48ebc70  EFLAGS: 00000202
RAX: 0000000000000008 RBX: 000800008647e6a0 RCX: ffff9b6d1d8207e0
RDX: 0000000000000008 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9b6cf48ebca8 R08: ffff9b6672f56c00 R09: ffffffff86785fd9
R10: ffff9b6d1d99f160 R11: fffff569850afc00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9b6cf48ebc20 R15: ffff9b6cf48ebbe8
FS:  00007f8abcc1a880(0000) GS:ffff9b6d1d980000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5e14b50760 CR3: 00000007f743e000 CR4: 00000000000007e0
Call Trace:
[<ffffffff8647e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8647e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff865fe15d>] change_protection+0x5dd/0x680
[<ffffffff865fe363>] mprotect_fixup+0x163/0x3a0
[<ffffffff86797597>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff865fe7de>] do_mprotect_pkey+0x23e/0x380
[<ffffffff865fe933>] SyS_mprotect+0x13/0x20
[<ffffffff86b94f92>] system_call_fastpath+0x25/0x2a
[<ffffffff86b94ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 48 63 35 ce 88 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 15 87 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 88 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #19388
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff943175ca
Build id: 3.10.0-1127.10.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9427e8f8
Build id: 3.10.0-1127.10.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff9427e96b
Build id: 3.10.0-1127.10.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff943fe23d
Build id: 3.10.0-1127.10.1.el7.x86_64
0x5cd
5
mprotect_fixup
vmlinux 0xffffffff943fe443
Build id: 3.10.0-1127.10.1.el7.x86_64
0x163
6
? call_rwsem_down_write_failed
vmlinux 0xffffffff94596737
Build id: 3.10.0-1127.10.1.el7.x86_64
0x17
7
do_mprotect_pkey
vmlinux 0xffffffff943fe8be
Build id: 3.10.0-1127.10.1.el7.x86_64
0x23e
8
SyS_mprotect
vmlinux 0xffffffff943fea13
Build id: 3.10.0-1127.10.1.el7.x86_64
0x13
9
system_call_fastpath
vmlinux 0xffffffff94992ed2
Build id: 3.10.0-1127.10.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [java:4209]
Modules linked in: veth xt_addrtype br_netfilter xt_CHECKSUM tun devlink xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_mark ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock overlay(T) sunrpc sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd joydev pcspkr sg i2c_piix4 vmw_vmci parport_pc
parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix drm nfit libata mptspi libnvdimm serio_raw scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 4209 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-1127.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff977a93e141c0 ti: ffff977abb71c000 task.ti: ffff977abb71c000
RIP: 0010:[<ffffffff943175ca>]  [<ffffffff943175ca>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff977abb71fc70  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 000800009427e730 RCX: ffff977add6dfc10
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff977abb71fca8 R08: ffff97743fdd7000 R09: ffffffff94585179
R10: ffff977add61f160 R11: fffff3cfc19c9a00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff977abb71fc20 R15: 000000000040003f
FS:  00007f19f84b2700(0000) GS:ffff977add600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcdf30cc5b0 CR3: 00000007d3ef0000 CR4: 00000000003607f0
Call Trace:
[<ffffffff9427e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9427e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff943fe23d>] change_protection+0x5cd/0x670
[<ffffffff943fe443>] mprotect_fixup+0x163/0x3a0
[<ffffffff94596737>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff943fe8be>] do_mprotect_pkey+0x23e/0x380
[<ffffffff943fea13>] SyS_mprotect+0x13/0x20
[<ffffffff94992ed2>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 9e 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 10 f5 94 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 6d 7b c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #18461
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb7f1761e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x20e
2
native_flush_tlb_others
vmlinux 0xffffffffb7e7e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffb7e7e96b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffffb7ffe26d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5dd
5
? __schedule
vmlinux 0xffffffffb8585942
Build id: 3.10.0-1127.19.1.el7.x86_64
0x402
6
mprotect_fixup
vmlinux 0xffffffffb7ffe473
Build id: 3.10.0-1127.19.1.el7.x86_64
0x163
7
? call_rwsem_down_write_failed
vmlinux 0xffffffffb8196777
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffffb7ffe8ee
Build id: 3.10.0-1127.19.1.el7.x86_64
0x23e
9
SyS_mprotect
vmlinux 0xffffffffb7ffea43
Build id: 3.10.0-1127.19.1.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffffb8592ed2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x25
11
? system_call_after_swapgs
vmlinux 0xffffffffb8592e15
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [VM Thread:30200]
Modules linked in: xt_nat veth dcdbas nf_conntrack_netlink nfnetlink br_netfilter bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop overlay vmw_vsock_vmci_transport vsock xt_comment xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ext4 mbcache jbd2 sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi crct10dif_pclmul
nfit crct10dif_common scsi_transport_spi crc32c_intel e1000 mptscsih serio_raw libnvdimm mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 30200 Comm: VM Thread Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff970aa5dee2a0 ti: ffff97085d004000 task.ti: ffff97085d004000
RIP: 0010:[<ffffffffb7f1761e>]  [<ffffffffb7f1761e>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff97085d007c70  EFLAGS: 00000202
RAX: 0000000000000012 RBX: ffffffffb7e7e730 RCX: ffff970b7f49fc10
RDX: 0000000000000012 RSI: 0000000000000014 RDI: 0000000000000000
RBP: ffff97085d007ca8 R08: ffff96fcffd66400 R09: ffffffffb81851b9
R10: ffff970b7f01f160 R11: ffffeded6afc6e00 R12: 0000000000000286
R13: ffff97085d007c20 R14: ffff9707501ac250 R15: ffff970b7f25acc0
FS:  00007f7e8635f700(0000) GS:ffff970b7f000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4f7d882000 CR3: 00000005d5a36000 CR4: 00000000000407f0
Call Trace:
[<ffffffffb7e7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb7e7e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb7ffe26d>] change_protection+0x5dd/0x680
[<ffffffffb8585942>] ? __schedule+0x402/0x840
[<ffffffffb7ffe473>] mprotect_fixup+0x163/0x3a0
[<ffffffffb8196777>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffffb7ffe8ee>] do_mprotect_pkey+0x23e/0x380
[<ffffffffb7ffea43>] SyS_mprotect+0x13/0x20
[<ffffffffb8592ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffb8592e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 10 b5 b8 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 9d 7b c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 
Complete report #14371
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb7516a7a
Build id: 3.10.0-1062.12.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffb747e478
Build id: 3.10.0-1062.12.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffb747e4eb
Build id: 3.10.0-1062.12.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffffb75fc4dd
Build id: 3.10.0-1062.12.1.el7.x86_64
0x5cd
5
? hrtimer_get_res
vmlinux 0xffffffffb74c9e30
Build id: 3.10.0-1062.12.1.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffffb75fc6e3
Build id: 3.10.0-1062.12.1.el7.x86_64
0x163
7
? call_rwsem_down_write_failed
vmlinux 0xffffffffb7792dc7
Build id: 3.10.0-1062.12.1.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffffb75fcb72
Build id: 3.10.0-1062.12.1.el7.x86_64
0x252
9
SyS_mprotect
vmlinux 0xffffffffb75fcc93
Build id: 3.10.0-1062.12.1.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffffb7b8dede
Build id: 3.10.0-1062.12.1.el7.x86_64
0x25
11
? system_call_after_swapgs
vmlinux 0xffffffffb7b8de21
Build id: 3.10.0-1062.12.1.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [VM Thread:6753]
Modules linked in: nf_log_ipv4 nf_log_common xt_LOG xt_recent ip_set_hash_net tcp_diag udp_diag inet_diag tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc fuse ebtable_filter ebtables ip6table_filter ip6_tables devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_set ip_set nfnetlink iptable_filter sunrpc snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd soundcore ppdev sg pcspkr parport_pc parport joydev i2c_piix4 virtio_balloon ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi
virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata floppy serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 6753 Comm: VM Thread Kdump: loaded Tainted: G             L ------------   3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff947f6b6f62a0 ti: ffff947a4b2ec000 task.ti: ffff947a4b2ec000
RIP: 0010:[<ffffffffb7516a7a>]  [<ffffffffb7516a7a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff947a4b2efc70  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffffffffb747e2b0 RCX: ffff94818b920400
RDX: 0000000000000004 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff947a4b2efca8 R08: ffff947121a1ec00 R09: ffffffffb7781839
R10: ffff94818b85f160 R11: fffff08fd0bef200 R12: ffff947a4b2efc40
R13: ffffffffb778178f R14: ffff94818b840000 R15: ffffffffb742b621
FS:  00007f7f7bf8f700(0000) GS:ffff94818b840000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ff828a280 CR3: 0000000c3db5e000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb747e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb747e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb75fc4dd>] change_protection+0x5cd/0x670
[<ffffffffb74c9e30>] ? hrtimer_get_res+0x50/0x50
[<ffffffffb75fc6e3>] mprotect_fixup+0x163/0x3a0
[<ffffffffb7792dc7>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffffb75fcb72>] do_mprotect_pkey+0x252/0x360
[<ffffffffb75fcc93>] SyS_mprotect+0x13/0x20
[<ffffffffb7b8dede>] system_call_fastpath+0x25/0x2a
[<ffffffffb7b8de21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 ee 5e c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 e8 14 b8 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 bd 5e c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #9778
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffaef11a8a
Build id: 3.10.0-957.21.3.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffaee7a6a8
Build id: 3.10.0-957.21.3.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffaee7a718
Build id: 3.10.0-957.21.3.el7.x86_64
0x68
4
change_protection
vmlinux 0xffffffffaeff465d
Build id: 3.10.0-957.21.3.el7.x86_64
0x5cd
5
? __schedule
vmlinux 0xffffffffaf568a7a
Build id: 3.10.0-957.21.3.el7.x86_64
0x42a
6
mprotect_fixup
vmlinux 0xffffffffaeff4872
Build id: 3.10.0-957.21.3.el7.x86_64
0x162
7
? call_rwsem_down_write_failed
vmlinux 0xffffffffaf187257
Build id: 3.10.0-957.21.3.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffffaeff4cc2
Build id: 3.10.0-957.21.3.el7.x86_64
0x252
9
SyS_mprotect
vmlinux 0xffffffffaeff4de3
Build id: 3.10.0-957.21.3.el7.x86_64
0x13
10
tracesys
vmlinux 0xffffffffaf57606b
Build id: 3.10.0-957.21.3.el7.x86_64
0xa3
NMI watchdog: BUG: soft lockup - CPU#12 stuck for 23s! [java:30494]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev sg cryptd virtio_balloon i2c_piix4 joydev pcspkr parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul drm crct10dif_common libata virtio_pci virtio_ring
crc32c_intel drm_panel_orientation_quirks serio_raw virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 12 PID: 30494 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-20180921_191058-build10b184b164b178 04/01/2014
task: ffff9bd5f87b0000 ti: ffff9bd6f4e00000 task.ti: ffff9bd6f4e00000
RIP: 0010:[<ffffffffaef11a8a>]  [<ffffffffaef11a8a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9bd6f4e03c70  EFLAGS: 00000202
RAX: 000000000000000e RBX: 00080000aee7a4e0 RCX: ffff9bd71b9a0768
RDX: 000000000000000e RSI: 00000000000000ff RDI: 0000000000000000
RBP: ffff9bd6f4e03ca8 R08: ffff9bd0745bcc00 R09: ffffffffaf175e39
R10: ffff9bd71b91f160 R11: ffffdeebc1613800 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9bd6f4e03c20 R15: ffffffffaee2a621
FS:  00007f54d0336700(0000) GS:ffff9bd71b900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9e7dac93c0 CR3: 00000006fca30000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaee7a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaee7a718>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaeff465d>] change_protection+0x5cd/0x680
[<ffffffffaf568a7a>] ? __schedule+0x42a/0x860
[<ffffffffaeff4872>] mprotect_fixup+0x162/0x360
[<ffffffffaf187257>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffffaeff4cc2>] do_mprotect_pkey+0x252/0x360
[<ffffffffaeff4de3>] SyS_mprotect+0x13/0x20
[<ffffffffaf57606b>] tracesys+0xa3/0xc9
Code: 48 63 35 3e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 bc b4 af f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 0d 83 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #1364
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8cd1519a
Build id: 3.10.0-1062.4.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8cc7d478
Build id: 3.10.0-1062.4.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff8cc7d4eb
Build id: 3.10.0-1062.4.1.el7.x86_64
0x6b
4
change_protection
vmlinux 0xffffffff8cdfab9d
Build id: 3.10.0-1062.4.1.el7.x86_64
0x5cd
5
? hrtimer_get_res
vmlinux 0xffffffff8ccc8df0
Build id: 3.10.0-1062.4.1.el7.x86_64
0x50
6
mprotect_fixup
vmlinux 0xffffffff8cdfada3
Build id: 3.10.0-1062.4.1.el7.x86_64
0x163
7
? call_rwsem_down_write_failed
vmlinux 0xffffffff8cf91427
Build id: 3.10.0-1062.4.1.el7.x86_64
0x17
8
do_mprotect_pkey
vmlinux 0xffffffff8cdfb232
Build id: 3.10.0-1062.4.1.el7.x86_64
0x252
9
SyS_mprotect
vmlinux 0xffffffff8cdfb353
Build id: 3.10.0-1062.4.1.el7.x86_64
0x13
10
system_call_fastpath
vmlinux 0xffffffff8d38bede
Build id: 3.10.0-1062.4.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [java:11702]
Modules linked in: xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass snd_ens1371 snd_rawmidi snd_ac97_codec ppdev ac97_bus crc32_pclmul snd_seq ghash_clmulni_intel snd_seq_device aesni_intel vmw_balloon snd_pcm lrw gf128mul glue_helper snd_timer ablk_helper cryptd snd soundcore pcspkr joydev nfit i2c_piix4 sg vmw_vmci libnvdimm parport_pc
parport binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix serio_raw libata mptspi scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 11702 Comm: java Kdump: loaded Not tainted 3.10.0-1062.4.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff8b41c740d230 ti: ffff8b43f1bdc000 task.ti: ffff8b43f1bdc000
RIP: 0010:[<ffffffff8cd1519a>]  [<ffffffff8cd1519a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8b43f1bdfc70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800008cc7d2b0 RCX: ffff8b456d6206c8
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8b43f1bdfca8 R08: ffff8b42b9cadc00 R09: ffffffff8cf7fe99
R10: ffff8b456d79f160 R11: ffffe76d40d48e00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8b43f1bdfc20 R15: ffffffff8cc2b621
FS:  00007f18a9660700(0000) GS:ffff8b456d780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f344a9357e0 CR3: 0000000091e54000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8cc7d478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8cc7d4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8cdfab9d>] change_protection+0x5cd/0x670
[<ffffffff8ccc8df0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8cdfada3>] mprotect_fixup+0x163/0x3a0
[<ffffffff8cf91427>] ? call_rwsem_down_write_failed+0x17/0x30
[<ffffffff8cdfb232>] do_mprotect_pkey+0x252/0x360
[<ffffffff8cdfb353>] SyS_mprotect+0x13/0x20
[<ffffffff8d38bede>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 6e 66 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 d6 94 8d f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 66 c4 00 eb b7 0f b6 4d cc 4c 89