Info

Permalink
Function
smp_call_function_many
First occurrence
2020-11-03
Last occurrence
2022-03-31
State
NEW
Components
kernel
Quality
-22

Statistics

Operating system Count
CentOS 7 177
Architecture Count
x86_64 177
Related packages Count
kernel 177
 0:3.10.0-957.el7 99
 0:3.10.0-862.el7 37
 0:3.10.0-862.11.6.el7 14
 0:3.10.0-862.6.3.el7 11
 0:3.10.0-862.14.4.el7 11
 0:3.10.0-957.10.1.el7 2
 0:3.10.0-957.21.2.el7 1
 0:3.10.0-957.21.3.el7 1
 0:3.10.0-957.27.2.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #831725
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9bb12b6a
Build id: 3.10.0-957.27.2.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9ba7b788
Build id: 3.10.0-957.27.2.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff9ba7b7f8
Build id: 3.10.0-957.27.2.el7.x86_64
0x68
4
tlb_flush_mmu.part.76
vmlinux 0xffffffff9bbe57d7
Build id: 3.10.0-957.27.2.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffff9bbe70d5
Build id: 3.10.0-957.27.2.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffff9bbe9f02
Build id: 3.10.0-957.27.2.el7.x86_64
0x142
7
? wake_up_q
vmlinux 0xffffffff9bad7bf8
Build id: 3.10.0-957.27.2.el7.x86_64
0x58
8
SyS_madvise
vmlinux 0xffffffff9bbe4ded
Build id: 3.10.0-957.27.2.el7.x86_64
0x49d
9
? vm_mmap_pgoff
vmlinux 0xffffffff9bbd784b
Build id: 3.10.0-957.27.2.el7.x86_64
0xeb
10
? system_call_after_swapgs
vmlinux 0xffffffff9c176d21
Build id: 3.10.0-957.27.2.el7.x86_64
0xae
11
tracesys
vmlinux 0xffffffff9c17706b
Build id: 3.10.0-957.27.2.el7.x86_64
0xa3
NMI watchdog: BUG: soft lockup - CPU#22 stuck for 22s! [ApplicationServ:35020]
Modules linked in: veth arc4 md4 nls_utf8 cifs ccm dns_resolver xt_nat ip6t_MASQUERADE nf_nat_masquerade_ipv6 nf_conntrack_netlink xt_addrtype br_netfilter binfmt_misc xt_CHECKSUM tun devlink overlay(T) ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack vmhgfs(OE) ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod nouveau mxm_wmi wmi sb_edac video i2c_algo_bit
ttm iosf_mbi crc32_pclmul drm_kms_helper ghash_clmulni_intel syscopyarea sysfillrect sysimgblt aesni_intel fb_sys_fops lrw gf128mul drm glue_helper ppdev ablk_helper drm_panel_orientation_quirks sg cryptd joydev nfit vmw_vmci pcspkr libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi ata_piix crct10dif_pclmul mptspi crct10dif_common scsi_transport_spi libata crc32c_intel mptscsih e1000 mptbase serio_raw
CPU: 22 PID: 35020 Comm: ApplicationServ Kdump: loaded Tainted: G        W  OE  ------------ T 3.10.0-957.27.2.el7.x86_64 #1
Hardware name: ASUS ROG STRIX B560-A/ROG STRIX B560-A, BIOS 6.00 07/22/2020
task: ffff8934df245140 ti: ffff89346eb10000 task.ti: ffff89346eb10000
RIP: 0010:[<ffffffff9bb12b6a>]  [<ffffffff9bb12b6a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff89346eb13c78  EFLAGS: 00000202
RAX: 000000000000001f RBX: 00000000000000fc RCX: ffff89399dde08f8
RDX: 000000000000001f RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff89346eb13cb0 R08: ffff8932f4236c00 R09: ffffffff9bd774a9
R10: ffff89399db9f160 R11: fffff17ea052a600 R12: 0000000000000296
R13: ffff89346eb13c28 R14: 0000000000000001 R15: fffff17e811305c0
FS:  00007f7a196ff640(0000) GS:ffff89399db80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0000e1000 CR3: 00000000509e6000 CR4: 00000000000607e0
Call Trace:
[<ffffffff9ba7b788>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9ba7b7f8>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9bbe57d7>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff9bbe70d5>] tlb_finish_mmu+0x55/0x60
[<ffffffff9bbe9f02>] zap_page_range+0x142/0x190
[<ffffffff9bad7bf8>] ? wake_up_q+0x58/0x80
[<ffffffff9bbe4ded>] SyS_madvise+0x49d/0xac0
[<ffffffff9bbd784b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff9c176d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9c17706b>] tracesys+0xa3/0xc9
Code: 48 63 35 5e 75 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 bf 74 9c f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 2d 75 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #820943
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffaed1173a
Build id: 3.10.0-957.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffaec7a598
Build id: 3.10.0-957.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffaec7a608
Build id: 3.10.0-957.el7.x86_64
0x68
4
tlb_flush_mmu.part.76
vmlinux 0xffffffffaede4077
Build id: 3.10.0-957.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffffaede5975
Build id: 3.10.0-957.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffffaede87c2
Build id: 3.10.0-957.el7.x86_64
0x142
7
? do_mmap
vmlinux 0xffffffffaedf28f8
Build id: 3.10.0-957.el7.x86_64
0x378
8
SyS_madvise
vmlinux 0xffffffffaede368d
Build id: 3.10.0-957.el7.x86_64
0x49d
9
? vm_mmap_pgoff
vmlinux 0xffffffffaedd612b
Build id: 3.10.0-957.el7.x86_64
0xeb
10
? system_call_after_swapgs
vmlinux 0xffffffffaf374d21
Build id: 3.10.0-957.el7.x86_64
0xae
11
? system_call_after_swapgs
vmlinux 0xffffffffaf374d15
Build id: 3.10.0-957.el7.x86_64
0xa2
12
system_call_fastpath
vmlinux 0xffffffffaf374ddb
Build id: 3.10.0-957.el7.x86_64
0x22
13
? system_call_after_swapgs
vmlinux 0xffffffffaf374d21
Build id: 3.10.0-957.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [java:24430]
Modules linked in: binfmt_misc sysmon_edr(OE) netfilter_edr(OE) ppdev sg i2c_piix4 joydev virtio_balloon parport_pc parport pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net serio_raw cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 24430 Comm: java Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff9cfd6927e180 ti: ffff9d00b8a38000 task.ti: ffff9d00b8a38000
RIP: 0010:[<ffffffffaed1173a>]  [<ffffffffaed1173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9d00b8a3bc78  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff9d00dec5f610
RDX: 0000000000000001 RSI: 0000000000000028 RDI: 0000000000000000
RBP: ffff9d00b8a3bcb0 R08: ffff9cfa3fd5f400 R09: ffffffffaef755c9
R10: ffff9d00dec1f160 R11: ffffed4393e09200 R12: 0000000000000296
R13: ffff9d00b8a3bc28 R14: 0000000000000001 R15: ffffed43aaf9c580
FS:  00007fe9008fc700(0000) GS:ffff9d00dec00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe9008e6b68 CR3: 000000078dec8000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaec7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaec7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaede4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffaede5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffaede87c2>] zap_page_range+0x142/0x190
[<ffffffffaedf28f8>] ? do_mmap+0x378/0x530
[<ffffffffaede368d>] SyS_madvise+0x49d/0xac0
[<ffffffffaedd612b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaf374d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaf374ddb>] system_call_fastpath+0x22/0x27
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 af f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #771089
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa3d1173a
Build id: 3.10.0-957.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa3c7a598
Build id: 3.10.0-957.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffa3c7a608
Build id: 3.10.0-957.el7.x86_64
0x68
4
tlb_flush_mmu.part.76
vmlinux 0xffffffffa3de4077
Build id: 3.10.0-957.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffffa3de5975
Build id: 3.10.0-957.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffffa3de87c2
Build id: 3.10.0-957.el7.x86_64
0x142
7
SyS_madvise
vmlinux 0xffffffffa3de368d
Build id: 3.10.0-957.el7.x86_64
0x49d
8
? vm_mmap_pgoff
vmlinux 0xffffffffa3dd612b
Build id: 3.10.0-957.el7.x86_64
0xeb
9
? system_call_after_swapgs
vmlinux 0xffffffffa4374d21
Build id: 3.10.0-957.el7.x86_64
0xae
10
tracesys
vmlinux 0xffffffffa437505b
Build id: 3.10.0-957.el7.x86_64
0xa3
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [java:28342]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel xt_mark nf_conntrack_netlink nfnetlink br_netfilter bridge stp llc overlay(T) ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter iptable_raw iptable_security iptable_mangle xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sg joydev virtio_balloon i2c_piix4 pcspkr ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_console virtio_net virtio_scsi bochs_drm drm_kms_helper syscopyarea sysfillrect serio_raw sysimgblt fb_sys_fops ttm drm floppy ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 28342 Comm: java Kdump: loaded Tainted: G               ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org 04/01/2014
task: ffff9ec007b4e180 ti: ffff9ec025768000 task.ti: ffff9ec025768000
RIP: 0010:[<ffffffffa3d1173a>]  [<ffffffffa3d1173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9ec02576bc78  EFLAGS: 00000202
RAX: 000000000000000b RBX: 00000000000000fc RCX: ffff9ec15f4df610
RDX: 000000000000000b RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9ec02576bcb0 R08: ffff9ebabfd49000 R09: ffffffffa3f755c9
R10: ffff9ec15f21f160 R11: ffffe4d61cf3f600 R12: 0000000000000296
R13: ffff9ec02576bc28 R14: 0000000000000001 R15: ffffe4d6127fe6c0
FS:  00007f3dd61a5700(0000) GS:ffff9ec15f200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f85e9730000 CR3: 000000071b6ee000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa3c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa3c7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa3de4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffa3de5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffa3de87c2>] zap_page_range+0x142/0x190
[<ffffffffa3de368d>] SyS_madvise+0x49d/0xac0
[<ffffffffa3dd612b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffa4374d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa437505b>] tracesys+0xa3/0xc9
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 a4 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #558832
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb050b09a
Build id: 3.10.0-862.11.6.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffb0479e58
Build id: 3.10.0-862.11.6.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffb0479ec8
Build id: 3.10.0-862.11.6.el7.x86_64
0x68
4
tlb_flush_mmu.part.75
vmlinux 0xffffffffb05c33c7
Build id: 3.10.0-862.11.6.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffffb05c4cc5
Build id: 3.10.0-862.11.6.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffffb05c7b12
Build id: 3.10.0-862.11.6.el7.x86_64
0x142
7
? futex_wake
vmlinux 0xffffffffb0506610
Build id: 3.10.0-862.11.6.el7.x86_64
0x90
8
SyS_madvise
vmlinux 0xffffffffb05c2a1d
Build id: 3.10.0-862.11.6.el7.x86_64
0x3cd
9
? system_call_after_swapgs
vmlinux 0xffffffffb0b256e1
Build id: 3.10.0-862.11.6.el7.x86_64
0xae
10
? system_call_after_swapgs
vmlinux 0xffffffffb0b256d5
Build id: 3.10.0-862.11.6.el7.x86_64
0xa2
11
system_call_fastpath
vmlinux 0xffffffffb0b2579b
Build id: 3.10.0-862.11.6.el7.x86_64
0x22
12
? system_call_after_swapgs
vmlinux 0xffffffffb0b256e1
Build id: 3.10.0-862.11.6.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [mariadbd:8289]
Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter bridge stp llc binfmt_misc iptable_nat nf_nat_ipv4 nf_nat ip6table_mangle ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables iptable_mangle ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter overlay(T) dm_mirror dm_region_hash ext4 dm_log dm_mod mbcache jbd2 sg i2c_piix4 ppdev shpchp pcspkr joydev virtio_balloon parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_net virtio_scsi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci i2c_core serio_raw virtio_ring virtio
floppy
CPU: 4 PID: 8289 Comm: mariadbd Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.11.6.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
task: ffff96506006dee0 ti: ffff96503b2e8000 task.ti: ffff96503b2e8000
RIP: 0010:[<ffffffffb050b09a>]  [<ffffffffb050b09a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff96503b2ebc78  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff96550d81cc98
RDX: 0000000000000000 RSI: 000000000000000c RDI: 0000000000000000
RBP: ffff96503b2ebcb0 R08: ffff965177a2e400 R09: ffffffffb074edf9
R10: ffff96550d91bb40 R11: ffffe4fd822bf600 R12: 0000000000000296
R13: ffff96503b2ebc28 R14: 0000000000000001 R15: ffffe4fd880c8b00
FS:  00007f34c3fff700(0000) GS:ffff96550d900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0bdffe36e0 CR3: 00000004d0ee4000 CR4: 00000000000006e0
Call Trace:
[<ffffffffb0479e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb0479ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb05c33c7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffb05c4cc5>] tlb_finish_mmu+0x55/0x60
[<ffffffffb05c7b12>] zap_page_range+0x142/0x190
[<ffffffffb0506610>] ? futex_wake+0x90/0x180
[<ffffffffb05c2a1d>] SyS_madvise+0x3cd/0x9c0
[<ffffffffb0b256e1>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb0b256d5>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffb0b2579b>] system_call_fastpath+0x22/0x27
[<ffffffffb0b256e1>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 ee 84 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 54 13 b1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 bd 84 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #401068
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb0511a8a
Build id: 3.10.0-957.21.3.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffb047a6a8
Build id: 3.10.0-957.21.3.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffb047a718
Build id: 3.10.0-957.21.3.el7.x86_64
0x68
4
tlb_flush_mmu.part.76
vmlinux 0xffffffffb05e4687
Build id: 3.10.0-957.21.3.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffffb05e5f85
Build id: 3.10.0-957.21.3.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffffb05e8dd2
Build id: 3.10.0-957.21.3.el7.x86_64
0x142
7
? do_mmap
vmlinux 0xffffffffb05f2f08
Build id: 3.10.0-957.21.3.el7.x86_64
0x378
8
SyS_madvise
vmlinux 0xffffffffb05e3c9d
Build id: 3.10.0-957.21.3.el7.x86_64
0x49d
9
? vm_mmap_pgoff
vmlinux 0xffffffffb05d677b
Build id: 3.10.0-957.21.3.el7.x86_64
0xeb
10
? system_call_after_swapgs
vmlinux 0xffffffffb0b75d21
Build id: 3.10.0-957.21.3.el7.x86_64
0xae
11
tracesys
vmlinux 0xffffffffb0b7606b
Build id: 3.10.0-957.21.3.el7.x86_64
0xa3
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [java:100046]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop overlay(T) gsch(OE) redirfs(OE) dsa_filter(POE) ppdev sg virtio_balloon i2c_piix4 pcspkr joydev parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix virtio_pci libata virtio_ring virtio drm_panel_orientation_quirks serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 100046 Comm: java Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9cd890551040 ti: ffff9cd6a5228000 task.ti: ffff9cd6a5228000
RIP: 0010:[<ffffffffb0511a8a>]  [<ffffffffb0511a8a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9cd6a522bc78  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff9cdaace1fb80
RDX: 0000000000000000 RSI: 00000000000000a0 RDI: 0000000000000000
RBP: ffff9cd6a522bcb0 R08: ffff9cd7f9c23000 R09: ffffffffb0775e39
R10: ffff9cdaace5f160 R11: fffff648d017b400 R12: 0000000000000296
R13: ffff9cd6a522bc28 R14: 0000000000000001 R15: fffff648c1905940
FS:  00007fb6bdf17700(0000) GS:ffff9cdaace40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f07d1fca050 CR3: 00000003f1f88000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb047a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb047a718>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb05e4687>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffb05e5f85>] tlb_finish_mmu+0x55/0x60
[<ffffffffb05e8dd2>] zap_page_range+0x142/0x190
[<ffffffffb05f2f08>] ? do_mmap+0x378/0x530
[<ffffffffb05e3c9d>] SyS_madvise+0x49d/0xac0
[<ffffffffb05d677b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffb0b75d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb0b7606b>] tracesys+0xa3/0xc9
Code: 48 63 35 3e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 bc 14 b1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 0d 83 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #387174
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9f31185a
Build id: 3.10.0-957.10.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9f27a5c8
Build id: 3.10.0-957.10.1.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff9f27a638
Build id: 3.10.0-957.10.1.el7.x86_64
0x68
4
tlb_flush_mmu.part.76
vmlinux 0xffffffff9f3e4507
Build id: 3.10.0-957.10.1.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffff9f3e5e05
Build id: 3.10.0-957.10.1.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffff9f3e8c52
Build id: 3.10.0-957.10.1.el7.x86_64
0x142
7
SyS_madvise
vmlinux 0xffffffff9f3e3b1d
Build id: 3.10.0-957.10.1.el7.x86_64
0x49d
8
? vm_mmap_pgoff
vmlinux 0xffffffff9f3d65bb
Build id: 3.10.0-957.10.1.el7.x86_64
0xeb
9
? system_call_after_swapgs
vmlinux 0xffffffff9f975d21
Build id: 3.10.0-957.10.1.el7.x86_64
0xae
10
? system_call_after_swapgs
vmlinux 0xffffffff9f975d15
Build id: 3.10.0-957.10.1.el7.x86_64
0xa2
11
system_call_fastpath
vmlinux 0xffffffff9f975ddb
Build id: 3.10.0-957.10.1.el7.x86_64
0x22
12
? system_call_after_swapgs
vmlinux 0xffffffff9f975d21
Build id: 3.10.0-957.10.1.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 67s! [java:1599]
Modules linked in: xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) ext4 mbcache jbd2 ppdev i2c_piix4 sg parport_pc parport virtio_balloon joydev pcspkr ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom virtio_net virtio_scsi ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata serio_raw drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1599 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff8a2439efc100 ti: ffff8a2a7be80000 task.ti: ffff8a2a7be80000
RIP: 0010:[<ffffffff9f31185a>]  [<ffffffff9f31185a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8a2a7be83c78  EFLAGS: 00000202
RAX: 0000000000000006 RBX: 00000000000000fc RCX: ffff8a2b8fd9fb80
RDX: 0000000000000006 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff8a2a7be83cb0 R08: ffff8a24f487f800 R09: ffffffff9f575c99
R10: ffff8a2b8fc5f160 R11: fffffe408fcfcc00 R12: 0000000000000296
R13: 0000000000000018 R14: ffff8a2a7be83be8 R15: 0000000000000296
FS:  00007f79cdb93700(0000) GS:ffff8a2b8fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcd44d8d000 CR3: 000000041be90000 CR4: 00000000000006e0
Call Trace:
[<ffffffff9f27a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9f27a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9f3e4507>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff9f3e5e05>] tlb_finish_mmu+0x55/0x60
[<ffffffff9f3e8c52>] zap_page_range+0x142/0x190
[<ffffffff9f3e3b1d>] SyS_madvise+0x49d/0xac0
[<ffffffff9f3d65bb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff9f975d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9f975d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff9f975ddb>] system_call_fastpath+0x22/0x27
[<ffffffff9f975d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 6e 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 b9 f4 9f f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 82 c4 00 eb b7 0f b6 4d cc 4c 89 
Complete report #373590
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff88107e3a
Build id: 3.10.0-862.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff88077018
Build id: 3.10.0-862.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff88077088
Build id: 3.10.0-862.el7.x86_64
0x68
4
tlb_flush_mmu.part.75
vmlinux 0xffffffff881c0497
Build id: 3.10.0-862.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffff881c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffff881c4a02
Build id: 3.10.0-862.el7.x86_64
0x142
7
? do_mmap
vmlinux 0xffffffff881cdf0c
Build id: 3.10.0-862.el7.x86_64
0x3ac
8
SyS_madvise
vmlinux 0xffffffff881bfafd
Build id: 3.10.0-862.el7.x86_64
0x3cd
9
? vm_mmap_pgoff
vmlinux 0xffffffff881b35eb
Build id: 3.10.0-862.el7.x86_64
0xeb
10
? system_call_after_swapgs
vmlinux 0xffffffff8871f721
Build id: 3.10.0-862.el7.x86_64
0xae
11
? system_call_after_swapgs
vmlinux 0xffffffff8871f715
Build id: 3.10.0-862.el7.x86_64
0xa2
12
system_call_fastpath
vmlinux 0xffffffff8871f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
13
? system_call_after_swapgs
vmlinux 0xffffffff8871f721
Build id: 3.10.0-862.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 21s! [java:17590]
Modules linked in: fuse binfmt_misc af_packet_diag netlink_diag vsock_diag vsock sctp_diag sctp tcp_diag udp_diag inet_diag unix_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc ext4 mbcache jbd2 ppdev sg parport_pc joydev virtio_balloon parport i2c_piix4 pcspkr sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_scsi virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci i2c_core floppy virtio_ring virtio dm_mirror dm_region_hash dm_log
dm_mod
CPU: 5 PID: 17590 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff8a7fdbebaf70 ti: ffff8a7caa20c000 task.ti: ffff8a7caa20c000
RIP: 0010:[<ffffffff88107e3a>]  [<ffffffff88107e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8a7caa20fc78  EFLAGS: 00000202
RAX: 0000000000000007 RBX: 00000000000000fc RCX: ffff8a841dfdcc70
RDX: 0000000000000007 RSI: 0000000000000060 RDI: 0000000000000000
RBP: ffff8a7caa20fcb0 R08: ffff8a7d7443d000 R09: ffffffff8834a3d9
R10: ffff8a841df5bb00 R11: ffffefe5d98cd000 R12: 0000000000000296
R13: ffff8a7caa20fc28 R14: 0000000000000001 R15: ffffefe5d9c3e5c0
FS:  00007f606477d700(0000) GS:ffff8a841df40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6093393500 CR3: 00000007bda60000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff88077018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff88077088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff881c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff881c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff881c4a02>] zap_page_range+0x142/0x190
[<ffffffff881cdf0c>] ? do_mmap+0x3ac/0x4d0
[<ffffffff881bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff881b35eb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff8871f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8871f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8871f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff8871f721>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 d3 88 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #247116
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa0907e3a
Build id: 3.10.0-862.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa0877018
Build id: 3.10.0-862.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffa0877088
Build id: 3.10.0-862.el7.x86_64
0x68
4
tlb_flush_mmu.part.75
vmlinux 0xffffffffa09c0497
Build id: 3.10.0-862.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffffa09c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffffa09c4a02
Build id: 3.10.0-862.el7.x86_64
0x142
7
? do_mmap
vmlinux 0xffffffffa09cdf0c
Build id: 3.10.0-862.el7.x86_64
0x3ac
8
SyS_madvise
vmlinux 0xffffffffa09bfafd
Build id: 3.10.0-862.el7.x86_64
0x3cd
9
? vm_mmap_pgoff
vmlinux 0xffffffffa09b35eb
Build id: 3.10.0-862.el7.x86_64
0xeb
10
? system_call_after_swapgs
vmlinux 0xffffffffa0f1f721
Build id: 3.10.0-862.el7.x86_64
0xae
11
? system_call_after_swapgs
vmlinux 0xffffffffa0f1f715
Build id: 3.10.0-862.el7.x86_64
0xa2
12
system_call_fastpath
vmlinux 0xffffffffa0f1f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
13
? system_call_after_swapgs
vmlinux 0xffffffffa0f1f721
Build id: 3.10.0-862.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:15246]
Modules linked in: gsch(OE) redirfs(OE) dsa_filter(POE) binfmt_misc sunrpc ext4 mbcache jbd2 ppdev joydev sg i2c_piix4 pcspkr parport_pc parport virtio_balloon ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_scsi virtio_console virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci i2c_core virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 15246 Comm: java Kdump: loaded Tainted: P           OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff998b02d5cf10 ti: ffff998d1da94000 task.ti: ffff998d1da94000
RIP: 0010:[<ffffffffa0907e3a>]  [<ffffffffa0907e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff998d1da97c78  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff998eaec1c4b0
RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000000
RBP: ffff998d1da97cb0 R08: ffff998bfa347400 R09: ffffffffa0b4a3d9
R10: ffff998eaec5bb00 R11: ffffef5982ae7000 R12: 0000000000000296
R13: ffff998d1da97c28 R14: 0000000000000001 R15: ffffef598c2f8940
FS:  00007f1a14353700(0000) GS:ffff998eaec40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1a13a395e0 CR3: 000000000bfec000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa0877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa0877088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa09c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa09c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffa09c4a02>] zap_page_range+0x142/0x190
[<ffffffffa09cdf0c>] ? do_mmap+0x3ac/0x4d0
[<ffffffffa09bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa09b35eb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffa0f1f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa0f1f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffa0f1f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffffa0f1f721>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 53 a1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #199344
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa2107e3a
Build id: 3.10.0-862.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa2077018
Build id: 3.10.0-862.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffffa2077088
Build id: 3.10.0-862.el7.x86_64
0x68
4
tlb_flush_mmu.part.75
vmlinux 0xffffffffa21c0497
Build id: 3.10.0-862.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffffa21c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffffa21c4a02
Build id: 3.10.0-862.el7.x86_64
0x142
7
SyS_madvise
vmlinux 0xffffffffa21bfafd
Build id: 3.10.0-862.el7.x86_64
0x3cd
8
? vm_mmap_pgoff
vmlinux 0xffffffffa21b35eb
Build id: 3.10.0-862.el7.x86_64
0xeb
9
? system_call_after_swapgs
vmlinux 0xffffffffa271f721
Build id: 3.10.0-862.el7.x86_64
0xae
10
? system_call_after_swapgs
vmlinux 0xffffffffa271f715
Build id: 3.10.0-862.el7.x86_64
0xa2
11
system_call_fastpath
vmlinux 0xffffffffa271f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
12
? system_call_after_swapgs
vmlinux 0xffffffffa271f721
Build id: 3.10.0-862.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [http-nio-8848-e:6082]
Modules linked in: veth rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vxlan ip6_udp_tunnel udp_tunnel ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_conntrack_netlink br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype xt_set xt_mark xt_conntrack ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ipt_REJECT ip_set_bitmap_port nf_reject_ipv4 ip_set nfnetlink dummy xt_comment tun bridge iptable_nat nf_nat_ipv4 stp llc nf_nat ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper
cryptd pcspkr vmw_balloon ppdev sg joydev parport_pc parport vmw_vmci i2c_piix4 shpchp sch_fq_codel ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm ata_piix libahci mptspi scsi_transport_spi libata mptscsih crct10dif_pclmul crct10dif_common serio_raw crc32c_intel i2c_core mptbase vmxnet3 dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 6082 Comm: http-nio-8848-e Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8face1775ee0 ti: ffff8fae0ab14000 task.ti: ffff8fae0ab14000
RIP: 0010:[<ffffffffa2107e3a>]  [<ffffffffa2107e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8fae0ab17c78  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00000000000000fc RCX: ffff8fae3fcdcf00
RDX: 0000000000000003 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff8fae0ab17cb0 R08: ffff8fab7a34f800 R09: ffffffffa234a3d9
R10: ffffc466ffd20840 R11: fffffbaccc1dac00 R12: 0000000000000296
R13: ffff8fae0ab17c28 R14: 0000000000000001 R15: ffffffffa21f8139
FS:  00007f46238dd700(0000) GS:ffff8fae3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c42090b010 CR3: 000000009d252000 CR4: 00000000000007e0
Call Trace:
[<ffffffffa2077018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2077088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa21c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa21c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffa21c4a02>] zap_page_range+0x142/0x190
[<ffffffffa21bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa21b35eb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffa271f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa271f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffa271f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffffa271f721>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 d3 a2 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #13061
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff82b1173a
Build id: 3.10.0-957.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff82a7a598
Build id: 3.10.0-957.el7.x86_64
0xb8
3
flush_tlb_mm_range
vmlinux 0xffffffff82a7a608
Build id: 3.10.0-957.el7.x86_64
0x68
4
tlb_flush_mmu.part.76
vmlinux 0xffffffff82be4077
Build id: 3.10.0-957.el7.x86_64
0x37
5
tlb_finish_mmu
vmlinux 0xffffffff82be5975
Build id: 3.10.0-957.el7.x86_64
0x55
6
zap_page_range
vmlinux 0xffffffff82be87c2
Build id: 3.10.0-957.el7.x86_64
0x142
7
? do_mmap
vmlinux 0xffffffff82bf28f8
Build id: 3.10.0-957.el7.x86_64
0x378
8
SyS_madvise
vmlinux 0xffffffff82be368d
Build id: 3.10.0-957.el7.x86_64
0x49d
9
? vm_mmap_pgoff
vmlinux 0xffffffff82bd612b
Build id: 3.10.0-957.el7.x86_64
0xeb
10
? system_call_after_swapgs
vmlinux 0xffffffff83174d21
Build id: 3.10.0-957.el7.x86_64
0xae
11
? system_call_after_swapgs
vmlinux 0xffffffff83174d15
Build id: 3.10.0-957.el7.x86_64
0xa2
12
system_call_fastpath
vmlinux 0xffffffff83174ddb
Build id: 3.10.0-957.el7.x86_64
0x22
13
? system_call_after_swapgs
vmlinux 0xffffffff83174d21
Build id: 3.10.0-957.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [java:156675]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop overlay ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ppdev parport_pc sg joydev virtio_balloon parport i2c_piix4 pcspkr nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom virtio_scsi virtio_blk virtio_net
virtio_console ata_generic pata_acpi floppy cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm virtio_pci libata drm_panel_orientation_quirks serio_raw virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 156675 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff978c9ca96180 ti: ffff978c62ff8000 task.ti: ffff978c62ff8000
RIP: 0010:[<ffffffff82b1173a>]  [<ffffffff82b1173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff978c62ffbc78  EFLAGS: 00000202
RAX: 0000000000000005 RBX: 00000000000000fc RCX: ffff978dacf60398
RDX: 0000000000000005 RSI: 00000000000000a0 RDI: 0000000000000000
RBP: ffff978c62ffbcb0 R08: ffff978af9c24800 R09: ffffffff82d755c9
R10: ffff978dacfdf160 R11: fffff666ce51e600 R12: 0000000000000296
R13: ffff978c62ffbc28 R14: 0000000000000001 R15: fffff666c7ccbfc0
FS:  00007fe434437700(0000) GS:ffff978dacfc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe434426630 CR3: 0000000274aee000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff82a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff82a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff82be4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff82be5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff82be87c2>] zap_page_range+0x142/0x190
[<ffffffff82bf28f8>] ? do_mmap+0x378/0x530
[<ffffffff82be368d>] SyS_madvise+0x49d/0xac0
[<ffffffff82bd612b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff83174d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff83174d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff83174ddb>] system_call_fastpath+0x22/0x27
[<ffffffff83174d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 74 83 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89