Info

Permalink
Function
smp_call_function_many
First occurrence
2020-12-31
Last occurrence
2021-05-06
State
NEW
Components
kernel
Quality
-31

Statistics

Operating system Count
CentOS 7 39
Architecture Count
x86_64 39
Related packages Count
kernel 39
 0:3.10.0-693.el7 20
 0:3.10.0-862.3.2.el7 10
 0:3.10.0-862.11.6.el7 3
 0:3.10.0-693.21.1.el7 2
 0:3.10.0-862.el7 2
 0:3.10.0-862.14.4.el7 1
 0:3.10.0-693.11.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #1421451
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f99b2
Build id: 3.10.0-693.11.1.el7.x86_64
0x202
2
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.11.1.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.11.1.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c2494
Build id: 3.10.0-693.11.1.el7.x86_64
0x54
5
try_to_unmap_one
vmlinux 0xffffffff811bc6d2
Build id: 3.10.0-693.11.1.el7.x86_64
0xb2
6
try_to_unmap_anon
vmlinux 0xffffffff811be12d
Build id: 3.10.0-693.11.1.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff811be1ed
Build id: 3.10.0-693.11.1.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff811e6dff
Build id: 3.10.0-693.11.1.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.51
vmlinux 0xffffffff811e42f0
Build id: 3.10.0-693.11.1.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff811e7acc
Build id: 3.10.0-693.11.1.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff811b0a98
Build id: 3.10.0-693.11.1.el7.x86_64
0x138
12
handle_mm_fault
vmlinux 0xffffffff811b1ea7
Build id: 3.10.0-693.11.1.el7.x86_64
0x6b7
13
? thread_group_cputime
vmlinux 0xffffffff810c84cb
Build id: 3.10.0-693.11.1.el7.x86_64
0xdb
14
__do_page_fault
vmlinux 0xffffffff816b0074
Build id: 3.10.0-693.11.1.el7.x86_64
0x154
15
do_page_fault
vmlinux 0xffffffff816b03a5
Build id: 3.10.0-693.11.1.el7.x86_64
0x35
16
page_fault
vmlinux 0xffffffff816ac5c8
Build id: 3.10.0-693.11.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [sidekiq.default:20671]
Modules linked in: tcp_diag inet_diag vmw_vsock_vmci_transport vsock sb_edac ppdev edac_core vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper joydev sg i2c_piix4 nfit shpchp vmw_vmci pcspkr parport_pc cryptd libnvdimm parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi crct10dif_pclmul scsi_transport_spi e1000 crct10dif_common mptscsih crc32c_intel serio_raw mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 13 PID: 20671 Comm: sidekiq.default Tainted: G             L ------------   3.10.0-693.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff88081e278fd0 ti: ffff880146510000 task.ti: ffff880146510000
RIP: 0010:[<ffffffff810f99b2>]  [<ffffffff810f99b2>] smp_call_function_many+0x202/0x260
RSP: 0000:ffff880146513b38  EFLAGS: 00000202
RAX: 000000000000000b RBX: 0000000000017800 RCX: ffff88081fcdb170
RDX: 000000000000000b RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff880146513b70 R08: ffff880174b5b000 R09: ffffffff81322729
R10: ffff88081fd59c80 R11: ffffea002174ee00 R12: ffff880146513ae8
R13: ffff88017fc03700 R14: ffff88016d1e4f00 R15: 00000000e9692cfb
FS:  00007f73909ff700(0000) GS:ffff88081fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f73383f1b40 CR3: 000000015e0f1000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00000001028adf18 ffff880121ce9c48 ffff880121ce9900 00007f73383f1000
0000000000000000 ffff880121ce9c48 0000000000000301 ffff880146513bc0
ffffffff8106e668 ffff880121ce9900 00007f73383f1000 0000000000000000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2494>] ptep_clear_flush+0x54/0x80
[<ffffffff811bc6d2>] try_to_unmap_one+0xb2/0x730
[<ffffffff811be12d>] try_to_unmap_anon+0xbd/0x130
[<ffffffff811be1ed>] try_to_unmap+0x4d/0x60
[<ffffffff811e6dff>] migrate_pages+0x43f/0x7a0
[<ffffffff811e42f0>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[<ffffffff811e7acc>] migrate_misplaced_page+0xcc/0x100
[<ffffffff811b0a98>] do_numa_page+0x138/0x1e0
[<ffffffff811b1ea7>] handle_mm_fault+0x6b7/0xfa0
[<ffffffff810c84cb>] ? thread_group_cputime+0xdb/0x190
[<ffffffff816b0074>] __do_page_fault+0x154/0x450
[<ffffffff816b03a5>] do_page_fault+0x35/0x90
[<ffffffff816ac5c8>] page_fault+0x28/0x30
Code: 48 63 35 16 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 e5 44 a2 00 eb b7 0f b6 4d cc 4c 89 
Complete report #1418236
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9ad07e3e
Build id: 3.10.0-862.el7.x86_64
0x20e
2
native_flush_tlb_others
vmlinux 0xffffffff9ac77018
Build id: 3.10.0-862.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff9ac771b4
Build id: 3.10.0-862.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff9add8837
Build id: 3.10.0-862.el7.x86_64
0x67
5
try_to_unmap_one
vmlinux 0xffffffff9add2478
Build id: 3.10.0-862.el7.x86_64
0x148
6
try_to_unmap_anon
vmlinux 0xffffffff9add3f1d
Build id: 3.10.0-862.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff9add3fdd
Build id: 3.10.0-862.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff9adff15f
Build id: 3.10.0-862.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.53
vmlinux 0xffffffff9adfc040
Build id: 3.10.0-862.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff9adffdfc
Build id: 3.10.0-862.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff9adc4f46
Build id: 3.10.0-862.el7.x86_64
0x156
12
handle_pte_fault
vmlinux 0xffffffff9adc52fe
Build id: 3.10.0-862.el7.x86_64
0x2fe
13
? futex_wake
vmlinux 0xffffffff9ad033a0
Build id: 3.10.0-862.el7.x86_64
0x90
14
handle_mm_fault
vmlinux 0xffffffff9adc747d
Build id: 3.10.0-862.el7.x86_64
0x39d
15
__do_page_fault
vmlinux 0xffffffff9b31a597
Build id: 3.10.0-862.el7.x86_64
0x197
16
do_page_fault
vmlinux 0xffffffff9b31a925
Build id: 3.10.0-862.el7.x86_64
0x35
17
? error_swapgs
vmlinux 0xffffffff9b316ab6
Build id: 3.10.0-862.el7.x86_64
0xa7
18
page_fault
vmlinux 0xffffffff9b316768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [java:15554]
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [java:12740]
Modules linked in: tcp_diag inet_diag veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ppdev sg joydev vmw_balloon parport_pc parport i2c_piix4 shpchp vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm
ata_piix mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common libata mptscsih serio_raw crc32c_intel mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 12740 Comm: java Kdump: loaded Tainted: G        W      ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff8f3e20df6eb0 ti: ffff8f3e89868000 task.ti: ffff8f3e89868000
RIP: 0010:[<ffffffff9ad07e3e>]  [<ffffffff9ad07e3e>] smp_call_function_many+0x20e/0x270
RSP: 0000:ffff8f3e8986ba88  EFLAGS: 00000202
RAX: 0000000000000005 RBX: 0000000000019680 RCX: ffff8f4035d5ce48
RDX: 0000000000000005 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff8f3e8986bac0 R08: ffff8f3efbacb400 R09: ffffffff9af4a3d9
R10: ffff8f4035cdbb00 R11: ffffd72a409eec00 R12: ffff8f3e8986ba38
R13: 000000009ac541b2 R14: ffff8f403ffed228 R15: ffffffff9ac5f2ee
FS:  00007fc897f8a700(0000) GS:ffff8f4035cc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000f194b000 CR3: 000000010beb0000 CR4: 00000000000407e0
Call Trace:
[<ffffffff9ac77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9ac771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff9add8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff9add2478>] try_to_unmap_one+0x148/0x7c0
[<ffffffff9add3f1d>] try_to_unmap_anon+0xbd/0x130
[<ffffffff9add3fdd>] try_to_unmap+0x4d/0x60
[<ffffffff9adff15f>] migrate_pages+0x43f/0x7a0
[<ffffffff9adfc040>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffff9adffdfc>] migrate_misplaced_page+0xcc/0x100
[<ffffffff9adc4f46>] do_numa_page+0x156/0x210
[<ffffffff9adc52fe>] handle_pte_fault+0x2fe/0xc30
[<ffffffff9ad033a0>] ? futex_wake+0x90/0x180
[<ffffffff9adc747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9b31a597>] __do_page_fault+0x197/0x4f0
[<ffffffff9b31a925>] do_page_fault+0x35/0x90
[<ffffffff9b316ab6>] ? error_swapgs+0xa7/0xbd
[<ffffffff9b316768>] page_fault+0x28/0x30
Code: b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 93 9b f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 
Complete report #1225472
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffaaf0b0ca
Build id: 3.10.0-862.14.4.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffaae79e58
Build id: 3.10.0-862.14.4.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffffaae79ff4
Build id: 3.10.0-862.14.4.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffffaafdc0f7
Build id: 3.10.0-862.14.4.el7.x86_64
0x67
5
try_to_unmap_one
vmlinux 0xffffffffaafd5c98
Build id: 3.10.0-862.14.4.el7.x86_64
0x148
6
try_to_unmap_anon
vmlinux 0xffffffffaafd77bd
Build id: 3.10.0-862.14.4.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffffaafd787d
Build id: 3.10.0-862.14.4.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffffab002fdf
Build id: 3.10.0-862.14.4.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffaafffd60
Build id: 3.10.0-862.14.4.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffffab003c9c
Build id: 3.10.0-862.14.4.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffffaafc819d
Build id: 3.10.0-862.14.4.el7.x86_64
0x19d
12
handle_pte_fault
vmlinux 0xffffffffaafc8566
Build id: 3.10.0-862.14.4.el7.x86_64
0x316
13
handle_mm_fault
vmlinux 0xffffffffaafcaefd
Build id: 3.10.0-862.14.4.el7.x86_64
0x39d
14
__do_page_fault
vmlinux 0xffffffffab520547
Build id: 3.10.0-862.14.4.el7.x86_64
0x197
15
do_page_fault
vmlinux 0xffffffffab5208d5
Build id: 3.10.0-862.14.4.el7.x86_64
0x35
16
page_fault
vmlinux 0xffffffffab51c758
Build id: 3.10.0-862.14.4.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [java:28937]
Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 sb_edac iosf_mbi ppdev vmwgfx crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul ttm glue_helper ablk_helper cryptd drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm pcnet32 pcspkr joydev vmw_vmci mii i2c_piix4 vmw_pvscsi sg i2c_core nfit shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi ata_piix
libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 8 PID: 28937 Comm: java Kdump: loaded Tainted: G           OEL ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8d43b3246eb0 ti: ffff8d3798780000 task.ti: ffff8d3798780000
RIP: 0010:[<ffffffffaaf0b0ca>]  [<ffffffffaaf0b0ca>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8d3798783a88  EFLAGS: 00000202
RAX: 000000000000000c RBX: 00000000000000fc RCX: ffff8d477f91d078
RDX: 000000000000000c RSI: 0000000000000014 RDI: 0000000000000000
RBP: ffff8d3798783ac0 R08: ffff8d2929b53c00 R09: ffffffffab14ef19
R10: ffff8d37a041bb40 R11: fffff00af28aac00 R12: 0000000000000292
R13: ffff8d3798783a38 R14: 00000000004c4b3f R15: ffff8d3798783b58
FS:  00007f59a0842700(0000) GS:ffff8d37a0400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f59feaf45b0 CR3: 0000000d01774000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaae79e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaae79ff4>] flush_tlb_page+0x54/0xa0
[<ffffffffaafdc0f7>] ptep_clear_flush+0x67/0xa0
[<ffffffffaafd5c98>] try_to_unmap_one+0x148/0x810
[<ffffffffaafd77bd>] try_to_unmap_anon+0xbd/0x130
[<ffffffffaafd787d>] try_to_unmap+0x4d/0x60
[<ffffffffab002fdf>] migrate_pages+0x43f/0x7a0
[<ffffffffaafffd60>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffab003c9c>] migrate_misplaced_page+0xcc/0x100
[<ffffffffaafc819d>] do_numa_page+0x19d/0x250
[<ffffffffaafc8566>] handle_pte_fault+0x316/0xd10
[<ffffffffaafcaefd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffab520547>] __do_page_fault+0x197/0x4f0
[<ffffffffab5208d5>] do_page_fault+0x35/0x90
[<ffffffffab51c758>] page_fault+0x28/0x30
Code: 48 63 35 fe 84 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 54 b3 ab f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 cd 84 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #1219761
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f99a6
Build id: 3.10.0-693.el7.x86_64
0x206
2
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c2524
Build id: 3.10.0-693.el7.x86_64
0x54
5
try_to_unmap_one
vmlinux 0xffffffff811bc762
Build id: 3.10.0-693.el7.x86_64
0xb2
6
try_to_unmap_anon
vmlinux 0xffffffff811be1bd
Build id: 3.10.0-693.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff811be27d
Build id: 3.10.0-693.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff811e6e6f
Build id: 3.10.0-693.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.51
vmlinux 0xffffffff811e4360
Build id: 3.10.0-693.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff811e7b3c
Build id: 3.10.0-693.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff811b0b28
Build id: 3.10.0-693.el7.x86_64
0x138
12
handle_mm_fault
vmlinux 0xffffffff811b1f37
Build id: 3.10.0-693.el7.x86_64
0x6b7
13
? dentry_kill
vmlinux 0xffffffff812186b4
Build id: 3.10.0-693.el7.x86_64
0x154
14
__do_page_fault
vmlinux 0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
0x154
15
do_page_fault
vmlinux 0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
0x35
16
page_fault
vmlinux 0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mysqld:7466]
Modules linked in: tcp_lp binfmt_misc arc4 md4 fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nls_utf8 cifs dns_resolver vmw_vsock_vmci_transport vsock edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc shpchp parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix
crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 7466 Comm: mysqld Not tainted 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880101b61fa0 ti: ffff880687e80000 task.ti: ffff880687e80000
RIP: 0010:[<ffffffff810f99a6>]  [<ffffffff810f99a6>] smp_call_function_many+0x206/0x260
RSP: 0000:ffff880687e83b38  EFLAGS: 00000202
RAX: 0000000000000007 RBX: 00000000000000fc RCX: ffff88042fddaf90
RDX: 0000000000000007 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff880687e83b70 R08: ffff88017a1fb800 R09: ffffffff813227d9
R10: ffff88042fc59c80 R11: ffffea0010b4cc00 R12: 0000000000000292
R13: ffff880687e83ae8 R14: ffff880687e83ac8 R15: ffffffff812361f2
FS:  00007f24dc71f700(0000) GS:ffff88042fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f319a753fe8 CR3: 000000083fd67000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
000000012fc5a1d8 ffff88042dbbc1c8 ffff88042dbbbe80 00007f319a753000
0000000000000000 ffff88042dbbc1c8 0000000000000301 ffff880687e83bc0
ffffffff8106e668 ffff88042dbbbe80 00007f319a753000 0000000000000000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2524>] ptep_clear_flush+0x54/0x80
[<ffffffff811bc762>] try_to_unmap_one+0xb2/0x730
[<ffffffff811be1bd>] try_to_unmap_anon+0xbd/0x130
[<ffffffff811be27d>] try_to_unmap+0x4d/0x60
[<ffffffff811e6e6f>] migrate_pages+0x43f/0x7a0
[<ffffffff811e4360>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[<ffffffff811e7b3c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff811b0b28>] do_numa_page+0x138/0x1e0
[<ffffffff811b1f37>] handle_mm_fault+0x6b7/0xfa0
[<ffffffff812186b4>] ? dentry_kill+0x154/0x1b0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 
Complete report #1181325
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa0d0816a
Build id: 3.10.0-862.3.2.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa0c772d8
Build id: 3.10.0-862.3.2.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffffa0c77474
Build id: 3.10.0-862.3.2.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffffa0dd8b77
Build id: 3.10.0-862.3.2.el7.x86_64
0x67
5
try_to_unmap_one
vmlinux 0xffffffffa0dd27b8
Build id: 3.10.0-862.3.2.el7.x86_64
0x148
6
try_to_unmap_anon
vmlinux 0xffffffffa0dd425d
Build id: 3.10.0-862.3.2.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffffa0dd431d
Build id: 3.10.0-862.3.2.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffffa0dff49f
Build id: 3.10.0-862.3.2.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffa0dfc380
Build id: 3.10.0-862.3.2.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffffa0e0013c
Build id: 3.10.0-862.3.2.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffffa0dc5286
Build id: 3.10.0-862.3.2.el7.x86_64
0x156
12
handle_pte_fault
vmlinux 0xffffffffa0dc563e
Build id: 3.10.0-862.3.2.el7.x86_64
0x2fe
13
? futex_wake
vmlinux 0xffffffffa0d036d0
Build id: 3.10.0-862.3.2.el7.x86_64
0x90
14
handle_mm_fault
vmlinux 0xffffffffa0dc77bd
Build id: 3.10.0-862.3.2.el7.x86_64
0x39d
15
__do_page_fault
vmlinux 0xffffffffa131b5c7
Build id: 3.10.0-862.3.2.el7.x86_64
0x197
16
do_page_fault
vmlinux 0xffffffffa131b955
Build id: 3.10.0-862.3.2.el7.x86_64
0x35
17
? error_swapgs
vmlinux 0xffffffffa1317af0
Build id: 3.10.0-862.3.2.el7.x86_64
0xc1
18
page_fault
vmlinux 0xffffffffa1317788
Build id: 3.10.0-862.3.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#10 stuck for 32s! [ContainerBackgr:13211]
Modules linked in: binfmt_misc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache fuse sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev joydev vmw_balloon pcspkr sg nfit parport_pc parport libnvdimm shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel mptscsih serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 10 PID: 13211 Comm: ContainerBackgr Kdump: loaded Not tainted 3.10.0-862.3.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
task: ffff8b6c80a9eeb0 ti: ffff8b7310e9c000 task.ti: ffff8b7310e9c000
RIP: 0010:[<ffffffffa0d0816a>]  [<ffffffffa0d0816a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8b7310e9fa88  EFLAGS: 00000202
RAX: 0000000000000005 RBX: 0000000000019680 RCX: ffff8b6ff349d150
RDX: 0000000000000005 RSI: 000000000000000e RDI: 0000000000000000
RBP: ffff8b7310e9fac0 R08: ffff8b707b6f4c00 R09: ffffffffa0f4a819
R10: ffff8b731351bb00 R11: ffffcdf9ed5e6a00 R12: ffff8b7310e9fa38
R13: 00000000a0df8505 R14: ffff8b731ffed228 R15: ffffffffc04de397
FS:  00007fd5dd731700(0000) GS:ffff8b7313500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd598004648 CR3: 000000060339a000 CR4: 00000000000407e0
Call Trace:
[<ffffffffa0c772d8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa0c77474>] flush_tlb_page+0x54/0xa0
[<ffffffffa0dd8b77>] ptep_clear_flush+0x67/0xa0
[<ffffffffa0dd27b8>] try_to_unmap_one+0x148/0x7c0
[<ffffffffa0dd425d>] try_to_unmap_anon+0xbd/0x130
[<ffffffffa0dd431d>] try_to_unmap+0x4d/0x60
[<ffffffffa0dff49f>] migrate_pages+0x43f/0x7a0
[<ffffffffa0dfc380>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffa0e0013c>] migrate_misplaced_page+0xcc/0x100
[<ffffffffa0dc5286>] do_numa_page+0x156/0x210
[<ffffffffa0dc563e>] handle_pte_fault+0x2fe/0xc30
[<ffffffffa0d036d0>] ? futex_wake+0x90/0x180
[<ffffffffa0dc77bd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa131b5c7>] __do_page_fault+0x197/0x4f0
[<ffffffffa131b955>] do_page_fault+0x35/0x90
[<ffffffffa1317af0>] ? error_swapgs+0xc1/0xd7
[<ffffffffa1317788>] page_fault+0x28/0x30
Code: 48 63 35 9e b2 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 52 93 a1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 6d b2 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #1015622
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff86907e3a
Build id: 3.10.0-862.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff86877018
Build id: 3.10.0-862.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff868771b4
Build id: 3.10.0-862.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff869d8837
Build id: 3.10.0-862.el7.x86_64
0x67
5
try_to_unmap_one
vmlinux 0xffffffff869d2478
Build id: 3.10.0-862.el7.x86_64
0x148
6
try_to_unmap_anon
vmlinux 0xffffffff869d3f1d
Build id: 3.10.0-862.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff869d3fdd
Build id: 3.10.0-862.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff869ff15f
Build id: 3.10.0-862.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.53
vmlinux 0xffffffff869fc040
Build id: 3.10.0-862.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff869ffdfc
Build id: 3.10.0-862.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff869c4f46
Build id: 3.10.0-862.el7.x86_64
0x156
12
handle_pte_fault
vmlinux 0xffffffff869c52fe
Build id: 3.10.0-862.el7.x86_64
0x2fe
13
handle_mm_fault
vmlinux 0xffffffff869c747d
Build id: 3.10.0-862.el7.x86_64
0x39d
14
__do_page_fault
vmlinux 0xffffffff86f1a597
Build id: 3.10.0-862.el7.x86_64
0x197
15
do_page_fault
vmlinux 0xffffffff86f1a925
Build id: 3.10.0-862.el7.x86_64
0x35
16
page_fault
vmlinux 0xffffffff86f16768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [java:2995]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache tcp_diag inet_diag nfnetlink_queue nfnetlink_log bluetooth rfkill xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul vmw_balloon glue_helper ablk_helper
cryptd joydev pcspkr sg nfit parport_pc libnvdimm vmw_vmci parport shpchp i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 2995 Comm: java Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9ce28b2ccf10 ti: ffff9ce289b18000 task.ti: ffff9ce289b18000
RIP: 0010:[<ffffffff86907e3a>]  [<ffffffff86907e3a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9ce289b1ba88  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff9ce35fc1d000
RDX: 0000000000000000 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9ce289b1bac0 R08: ffff9cdcb4b38800 R09: ffffffff86b4a3d9
R10: ffff9ce35fddbb00 R11: ffffe4261faef000 R12: 0000000000000292
R13: ffff9ce289b1ba38 R14: ffff9ce37ffed228 R15: 0000000000001ffa
FS:  00007f61ba163700(0000) GS:ffff9ce35fdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f61800020e0 CR3: 0000001007a24000 CR4: 00000000001607e0
Call Trace:
[<ffffffff86877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff868771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff869d8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff869d2478>] try_to_unmap_one+0x148/0x7c0
[<ffffffff869d3f1d>] try_to_unmap_anon+0xbd/0x130
[<ffffffff869d3fdd>] try_to_unmap+0x4d/0x60
[<ffffffff869ff15f>] migrate_pages+0x43f/0x7a0
[<ffffffff869fc040>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffff869ffdfc>] migrate_misplaced_page+0xcc/0x100
[<ffffffff869c4f46>] do_numa_page+0x156/0x210
[<ffffffff869c52fe>] handle_pte_fault+0x2fe/0xc30
[<ffffffff869c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff86f1a597>] __do_page_fault+0x197/0x4f0
[<ffffffff86f1a925>] do_page_fault+0x35/0x90
[<ffffffff86f16768>] page_fault+0x28/0x30
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 53 87 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #898899
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffad70816a
Build id: 3.10.0-862.3.2.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffad6772d8
Build id: 3.10.0-862.3.2.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffffad677474
Build id: 3.10.0-862.3.2.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffffad7d8b77
Build id: 3.10.0-862.3.2.el7.x86_64
0x67
5
try_to_unmap_one
vmlinux 0xffffffffad7d27b8
Build id: 3.10.0-862.3.2.el7.x86_64
0x148
6
try_to_unmap_anon
vmlinux 0xffffffffad7d425d
Build id: 3.10.0-862.3.2.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffffad7d431d
Build id: 3.10.0-862.3.2.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffffad7ff49f
Build id: 3.10.0-862.3.2.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffad7fc380
Build id: 3.10.0-862.3.2.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffffad80013c
Build id: 3.10.0-862.3.2.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffffad7c5286
Build id: 3.10.0-862.3.2.el7.x86_64
0x156
12
handle_pte_fault
vmlinux 0xffffffffad7c563e
Build id: 3.10.0-862.3.2.el7.x86_64
0x2fe
13
handle_mm_fault
vmlinux 0xffffffffad7c77bd
Build id: 3.10.0-862.3.2.el7.x86_64
0x39d
14
__do_page_fault
vmlinux 0xffffffffadd1b5c7
Build id: 3.10.0-862.3.2.el7.x86_64
0x197
15
do_page_fault
vmlinux 0xffffffffadd1b955
Build id: 3.10.0-862.3.2.el7.x86_64
0x35
16
? error_swapgs
vmlinux 0xffffffffadd17af0
Build id: 3.10.0-862.3.2.el7.x86_64
0xc1
17
page_fault
vmlinux 0xffffffffadd17788
Build id: 3.10.0-862.3.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#12 stuck for 23s! [java:9913]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache fuse sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr nfit sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi mptscsih drm ata_piix mptbase crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 12 PID: 9913 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-862.3.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
task: ffff905cfffaeeb0 ti: ffff9062d7bbc000 task.ti: ffff9062d7bbc000
RIP: 0010:[<ffffffffad70816a>]  [<ffffffffad70816a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9062d7bbfa88  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0000000000019680 RCX: ffff9059d341d1a0
RDX: 0000000000000000 RSI: 000000000000000e RDI: 0000000000000000
RBP: ffff9062d7bbfac0 R08: ffff90609b728000 R09: ffffffffad94a819
R10: ffff90633fc1bb00 R11: ffffca126aa9ac00 R12: ffff9062d7bbfa38
R13: 00000000d7bbfa28 R14: ffff90633ffbf228 R15: ffff9062d7bbfa80
FS:  00007fada68f7700(0000) GS:ffff90633fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fada68f4528 CR3: 000000066b35e000 CR4: 00000000000407e0
Call Trace:
[<ffffffffad6772d8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffad677474>] flush_tlb_page+0x54/0xa0
[<ffffffffad7d8b77>] ptep_clear_flush+0x67/0xa0
[<ffffffffad7d27b8>] try_to_unmap_one+0x148/0x7c0
[<ffffffffad7d425d>] try_to_unmap_anon+0xbd/0x130
[<ffffffffad7d431d>] try_to_unmap+0x4d/0x60
[<ffffffffad7ff49f>] migrate_pages+0x43f/0x7a0
[<ffffffffad7fc380>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffad80013c>] migrate_misplaced_page+0xcc/0x100
[<ffffffffad7c5286>] do_numa_page+0x156/0x210
[<ffffffffad7c563e>] handle_pte_fault+0x2fe/0xc30
[<ffffffffad7c77bd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffadd1b5c7>] __do_page_fault+0x197/0x4f0
[<ffffffffadd1b955>] do_page_fault+0x35/0x90
[<ffffffffadd17af0>] ? error_swapgs+0xc1/0xd7
[<ffffffffadd17788>] page_fault+0x28/0x30
Code: 48 63 35 9e b2 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 52 33 ae f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 6d b2 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #833695
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f9901
Build id: 3.10.0-693.el7.x86_64
0x161
2
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c2524
Build id: 3.10.0-693.el7.x86_64
0x54
5
try_to_unmap_one
vmlinux 0xffffffff811bc762
Build id: 3.10.0-693.el7.x86_64
0xb2
6
try_to_unmap_anon
vmlinux 0xffffffff811be1bd
Build id: 3.10.0-693.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff811be27d
Build id: 3.10.0-693.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff811e6e6f
Build id: 3.10.0-693.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.51
vmlinux 0xffffffff811e4360
Build id: 3.10.0-693.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff811e7b3c
Build id: 3.10.0-693.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff811b0b28
Build id: 3.10.0-693.el7.x86_64
0x138
12
handle_mm_fault
vmlinux 0xffffffff811b1f37
Build id: 3.10.0-693.el7.x86_64
0x6b7
13
__do_page_fault
vmlinux 0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
0x154
14
do_page_fault
vmlinux 0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
0x35
15
page_fault
vmlinux 0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#83 stuck for 23s! [R:22368]
Modules linked in: fuse vfat fat isofs ext4 mbcache jbd2 ses enclosure scsi_transport_sas uas usb_storage nfsv3 nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support dell_smbios dcdbas sparse_keymap intel_powerclamp coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif pcspkr sg i2c_i801 mei_me mei shpchp wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
mgag200 i2c_algo_bit drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops ttm drm ahci libahci tg3 libata bnxt_en megaraid_sas ptp i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 83 PID: 22368 Comm: R Tainted: G        W    L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R750/06V45N, BIOS 1.3.8 08/31/2021
task: ffff88010a449fa0 ti: ffff8801baf48000 task.ti: ffff8801baf48000
RIP: 0010:[<ffffffff810f9901>]  [<ffffffff810f9901>] smp_call_function_many+0x161/0x260
RSP: 0000:ffff8801baf4bb38  EFLAGS: 00000286
RAX: 0000000000000044 RBX: ffffffffffffffb3 RCX: 0000000000000010
RDX: 0000000000000000 RSI: 0000000000000030 RDI: ffff883f7fe9bc78
RBP: ffff8801baf4bb70 R08: ffff8840b64f0408 R09: 0000000000000001
R10: ffff8840b64f0400 R11: ffffffffffffff83 R12: ffff88707fbc2c80
R13: ffffffffffffff83 R14: ffff8801baf4bb00 R15: ffff8801baf4bc10
FS:  00002b5cee491700(0000) GS:ffff886fbf240000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002b4ded5eb000 CR3: 0000004090821000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00000001bf25a240 ffff8863eb3daf08 ffff8863eb3dabc0 00002b4ded5eb000
0000000000000000 ffff8863eb3daf08 0000000000000301 ffff8801baf4bbc0
ffffffff8106e668 ffff8863eb3dabc0 00002b4ded5eb000 0000000000000000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2524>] ptep_clear_flush+0x54/0x80
[<ffffffff811bc762>] try_to_unmap_one+0xb2/0x730
[<ffffffff811be1bd>] try_to_unmap_anon+0xbd/0x130
[<ffffffff811be27d>] try_to_unmap+0x4d/0x60
[<ffffffff811e6e6f>] migrate_pages+0x43f/0x7a0
[<ffffffff811e4360>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[<ffffffff811e7b3c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff811b0b28>] do_numa_page+0x138/0x1e0
[<ffffffff811b1f37>] handle_mm_fault+0x6b7/0xfa0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 00 49 8b 7f 08 83 c3 01 48 63 d3 e8 da d1 23 00 3b 05 b8 45 a2 00 89 c3 7d 52 48 98 49 8b 3f 48 03 3c c5 e0 fd b0 81 0f b7 57 20 <f6> c2 01 74 15 66 2e 0f 1f 84 00 00 00 00 00 f3 90 0f b7 57 20 
Complete report #676753
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa650b09a
Build id: 3.10.0-862.11.6.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa6479e58
Build id: 3.10.0-862.11.6.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffffa6479ff4
Build id: 3.10.0-862.11.6.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffffa65dc037
Build id: 3.10.0-862.11.6.el7.x86_64
0x67
5
try_to_unmap_one
vmlinux 0xffffffffa65d5bd8
Build id: 3.10.0-862.11.6.el7.x86_64
0x148
6
try_to_unmap_anon
vmlinux 0xffffffffa65d76fd
Build id: 3.10.0-862.11.6.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffffa65d77bd
Build id: 3.10.0-862.11.6.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffffa6602f1f
Build id: 3.10.0-862.11.6.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffa65ffca0
Build id: 3.10.0-862.11.6.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffffa6603bdc
Build id: 3.10.0-862.11.6.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffffa65c80dd
Build id: 3.10.0-862.11.6.el7.x86_64
0x19d
12
handle_pte_fault
vmlinux 0xffffffffa65c84a6
Build id: 3.10.0-862.11.6.el7.x86_64
0x316
13
? futex_wait_queue_me
vmlinux 0xffffffffa65067a4
Build id: 3.10.0-862.11.6.el7.x86_64
0xa4
14
handle_mm_fault
vmlinux 0xffffffffa65cae3d
Build id: 3.10.0-862.11.6.el7.x86_64
0x39d
15
__do_page_fault
vmlinux 0xffffffffa6b20557
Build id: 3.10.0-862.11.6.el7.x86_64
0x197
16
do_page_fault
vmlinux 0xffffffffa6b208e5
Build id: 3.10.0-862.11.6.el7.x86_64
0x35
17
page_fault
vmlinux 0xffffffffa6b1c758
Build id: 3.10.0-862.11.6.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [HikariPool-1 ho:12171]
Modules linked in: tcp_lp fuse xt_nat veth xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache overlay(T) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul vmw_balloon glue_helper ablk_helper cryptd joydev pcspkr sg pcnet32 nfit vmw_vmci mii i2c_piix4 shpchp libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel mptscsih serio_raw mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 10 PID: 12171 Comm: HikariPool-1 ho Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8a5c20334f10 ti: ffff8a5a7a264000 task.ti: ffff8a5a7a264000
RIP: 0010:[<ffffffffa650b09a>]  [<ffffffffa650b09a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8a5a7a267a88  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff8a549fa1d0c8
RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff8a5a7a267ac0 R08: ffff8a4df4bae800 R09: ffffffffa674edf9
R10: ffff8a549fc9bb40 R11: ffffe2fe9e07ee00 R12: 0000000000000292
R13: ffff8a5a7a267a38 R14: 0000000100000001 R15: 0000000000003ff9
FS:  00007f42845c5700(0000) GS:ffff8a549fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f42845c5a08 CR3: 0000000809f30000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa6479e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa6479ff4>] flush_tlb_page+0x54/0xa0
[<ffffffffa65dc037>] ptep_clear_flush+0x67/0xa0
[<ffffffffa65d5bd8>] try_to_unmap_one+0x148/0x810
[<ffffffffa65d76fd>] try_to_unmap_anon+0xbd/0x130
[<ffffffffa65d77bd>] try_to_unmap+0x4d/0x60
[<ffffffffa6602f1f>] migrate_pages+0x43f/0x7a0
[<ffffffffa65ffca0>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffa6603bdc>] migrate_misplaced_page+0xcc/0x100
[<ffffffffa65c80dd>] do_numa_page+0x19d/0x250
[<ffffffffa65c84a6>] handle_pte_fault+0x316/0xd10
[<ffffffffa65067a4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffa65cae3d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa6b20557>] __do_page_fault+0x197/0x4f0
[<ffffffffa6b208e5>] do_page_fault+0x35/0x90
[<ffffffffa6b1c758>] page_fault+0x28/0x30
Code: 48 63 35 ee 84 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 54 13 a7 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 bd 84 c3 00 eb b7 0f b6 4d cc 4c 89 
Complete report #542950
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f99a2
Build id: 3.10.0-693.el7.x86_64
0x202
2
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c2524
Build id: 3.10.0-693.el7.x86_64
0x54
5
try_to_unmap_one
vmlinux 0xffffffff811bc762
Build id: 3.10.0-693.el7.x86_64
0xb2
6
try_to_unmap_anon
vmlinux 0xffffffff811be1bd
Build id: 3.10.0-693.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff811be27d
Build id: 3.10.0-693.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff811e6e6f
Build id: 3.10.0-693.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.51
vmlinux 0xffffffff811e4360
Build id: 3.10.0-693.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff811e7b3c
Build id: 3.10.0-693.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff811b0b28
Build id: 3.10.0-693.el7.x86_64
0x138
12
handle_mm_fault
vmlinux 0xffffffff811b1f37
Build id: 3.10.0-693.el7.x86_64
0x6b7
13
__do_page_fault
vmlinux 0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
0x154
14
do_page_fault
vmlinux 0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
0x35
15
page_fault
vmlinux 0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 30s! [java:4273]
Modules linked in: sctp_diag sctp dccp_diag dccp udp_diag unix_diag af_packet_diag netlink_diag tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock ext4 sb_edac edac_core mbcache iosf_mbi jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon sg joydev pcspkr nfit parport_pc parport libnvdimm shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod ata_generic
sysimgblt fb_sys_fops crc_t10dif pata_acpi ttm crct10dif_generic drm ahci ata_piix libahci crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw i2c_core vmxnet3 vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 8 PID: 4273 Comm: java Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8809cd24cf10 ti: ffff880bd0388000 task.ti: ffff880bd0388000
RIP: 0010:[<ffffffff810f99a2>]  [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0000:ffff880bd038bb38  EFLAGS: 00000202
RAX: 000000000000000e RBX: 00000000000000fc RCX: ffff88183fd9b0a8
RDX: 000000000000000e RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff880bd038bb70 R08: ffff880cadf68000 R09: ffffffff813227d9
R10: ffff88183fc19c80 R11: ffffea005ccfb200 R12: 0000000000000292
R13: ffff880bd038bae8 R14: ffff8809cd24cf10 R15: ffff88180ab7a6c0
FS:  00007f3fbcace700(0000) GS:ffff88183fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f40d8ab7f06 CR3: 0000000bf02be000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00000001d038bb78 ffff880c0d053b88 ffff880c0d053840 00007f40d8ab7000
0000000000000000 ffff880c0d053b88 0000000000000301 ffff880bd038bbc0
ffffffff8106e668 ffff880c0d053840 00007f40d8ab7000 0000000000000000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2524>] ptep_clear_flush+0x54/0x80
[<ffffffff811bc762>] try_to_unmap_one+0xb2/0x730
[<ffffffff811be1bd>] try_to_unmap_anon+0xbd/0x130
[<ffffffff811be27d>] try_to_unmap+0x4d/0x60
[<ffffffff811e6e6f>] migrate_pages+0x43f/0x7a0
[<ffffffff811e4360>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[<ffffffff811e7b3c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff811b0b28>] do_numa_page+0x138/0x1e0
[<ffffffff811b1f37>] handle_mm_fault+0x6b7/0xfa0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 
Complete report #309996
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f99a2
Build id: 3.10.0-693.el7.x86_64
0x202
2
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c2524
Build id: 3.10.0-693.el7.x86_64
0x54
5
try_to_unmap_one
vmlinux 0xffffffff811bc762
Build id: 3.10.0-693.el7.x86_64
0xb2
6
try_to_unmap_anon
vmlinux 0xffffffff811be1bd
Build id: 3.10.0-693.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff811be27d
Build id: 3.10.0-693.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff811e6e6f
Build id: 3.10.0-693.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.51
vmlinux 0xffffffff811e4360
Build id: 3.10.0-693.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff811e7b3c
Build id: 3.10.0-693.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff811b0b28
Build id: 3.10.0-693.el7.x86_64
0x138
12
handle_mm_fault
vmlinux 0xffffffff811b1f37
Build id: 3.10.0-693.el7.x86_64
0x6b7
13
__do_page_fault
vmlinux 0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
0x154
14
do_page_fault
vmlinux 0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
0x35
15
page_fault
vmlinux 0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 21s! [dockerd:1327]
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 21s! [java:20483]
Modules linked in: vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vmw_vsock_vmci_transport vsock vboxdrv(OE) sb_edac edac_core coretemp iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nfit ppdev vmw_balloon joydev sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod vmwgfx crc_t10dif crct10dif_generic drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 8 PID: 20483 Comm: java Tainted: G           OEL ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff88101de85ee0 ti: ffff880a19be8000 task.ti: ffff880a19be8000
RIP: 0010:[<ffffffff810f99a2>]  [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0000:ffff880a19bebb38  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff88081fc5b0a8
RDX: 0000000000000001 RSI: 000000000000000c RDI: 0000000000000000
RBP: ffff880a19bebb70 R08: ffff8808b4e88c00 R09: ffffffff813227d9
R10: ffff88103fc99c80 R11: ffffea003df64600 R12: 0000000000000292
R13: ffff880a19bebae8 R14: ffff880a19bebab0 R15: ffffffff81098a48
FS:  00007f39ef0f0700(0000) GS:ffff88103fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f39ef0f0a08 CR3: 0000000717b86000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000119bebb78 ffff88081d87bb88 ffff88081d87b840 00007f39ef0f0000
0000000000000000 ffff88081d87bb88 0000000000000301 ffff880a19bebbc0
ffffffff8106e668 ffff88081d87b840 00007f39ef0f0000 0000000000000000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2524>] ptep_clear_flush+0x54/0x80
[<ffffffff811bc762>] try_to_unmap_one+0xb2/0x730
[<ffffffff811be1bd>] try_to_unmap_anon+0xbd/0x130
[<ffffffff811be27d>] try_to_unmap+0x4d/0x60
[<ffffffff811e6e6f>] migrate_pages+0x43f/0x7a0
[<ffffffff811e4360>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[<ffffffff811e7b3c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff811b0b28>] do_numa_page+0x138/0x1e0
[<ffffffff811b1f37>] handle_mm_fault+0x6b7/0xfa0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 
Complete report #236677
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f99a2
Build id: 3.10.0-693.el7.x86_64
0x202
2
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c2524
Build id: 3.10.0-693.el7.x86_64
0x54
5
try_to_unmap_one
vmlinux 0xffffffff811bc762
Build id: 3.10.0-693.el7.x86_64
0xb2
6
try_to_unmap_anon
vmlinux 0xffffffff811be1bd
Build id: 3.10.0-693.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff811be27d
Build id: 3.10.0-693.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff811e6e6f
Build id: 3.10.0-693.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.51
vmlinux 0xffffffff811e4360
Build id: 3.10.0-693.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff811e7b3c
Build id: 3.10.0-693.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff811b0b28
Build id: 3.10.0-693.el7.x86_64
0x138
12
handle_mm_fault
vmlinux 0xffffffff811b1f37
Build id: 3.10.0-693.el7.x86_64
0x6b7
13
? wake_up_q
vmlinux 0xffffffff810c47cb
Build id: 3.10.0-693.el7.x86_64
0x5b
14
__do_page_fault
vmlinux 0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
0x154
15
do_page_fault
vmlinux 0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
0x35
16
page_fault
vmlinux 0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [java:15419]
Modules linked in: tcp_diag udp_diag inet_diag nf_conntrack_ftp tcp_lp nls_utf8 isofs fuse xt_nat veth nf_conntrack_netlink xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter tun binfmt_misc overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock vfat fat sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 vmw_vmci shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptspi crct10dif_pclmul scsi_transport_spi crct10dif_common libata crc32c_intel mptscsih serio_raw vmxnet3 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 15419 Comm: java Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1410210136 10/21/2014
task: ffff880409b46eb0 ti: ffff8803fbe38000 task.ti: ffff8803fbe38000
RIP: 0010:[<ffffffff810f99a2>]  [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0000:ffff8803fbe3bb38  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff880237c1b080
RDX: 0000000000000000 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff8803fbe3bb70 R08: ffff88017cd30800 R09: ffffffff813227d9
R10: ffff880237dd9c80 R11: ffffea0008aeb800 R12: 0000000000000292
R13: ffff8803fbe3bae8 R14: 0000000800000005 R15: 0000000000001ffb
FS:  00007fbd5d085b10(0000) GS:ffff880237dc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbd7a6d7a80 CR3: 0000000432fda000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000100002d2e ffff8803f0315ac8 ffff8803f0315780 00007fbd7a6d7000
0000000000000000 ffff8803f0315ac8 0000000000000301 ffff8803fbe3bbc0
ffffffff8106e668 ffff8803f0315780 00007fbd7a6d7000 0000000000000000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2524>] ptep_clear_flush+0x54/0x80
[<ffffffff811bc762>] try_to_unmap_one+0xb2/0x730
[<ffffffff811be1bd>] try_to_unmap_anon+0xbd/0x130
[<ffffffff811be27d>] try_to_unmap+0x4d/0x60
[<ffffffff811e6e6f>] migrate_pages+0x43f/0x7a0
[<ffffffff811e4360>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[<ffffffff811e7b3c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff811b0b28>] do_numa_page+0x138/0x1e0
[<ffffffff811b1f37>] handle_mm_fault+0x6b7/0xfa0
[<ffffffff810c47cb>] ? wake_up_q+0x5b/0x80
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 
Complete report #23835
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810fd5fa
Build id: 3.10.0-693.21.1.el7.x86_64
0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff81070b98
Build id: 3.10.0-693.21.1.el7.x86_64
0xb8
3
flush_tlb_page
vmlinux 0xffffffff81070d94
Build id: 3.10.0-693.21.1.el7.x86_64
0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c6db4
Build id: 3.10.0-693.21.1.el7.x86_64
0x54
5
try_to_unmap_one
vmlinux 0xffffffff811c0fe2
Build id: 3.10.0-693.21.1.el7.x86_64
0xb2
6
try_to_unmap_anon
vmlinux 0xffffffff811c2a3d
Build id: 3.10.0-693.21.1.el7.x86_64
0xbd
7
try_to_unmap
vmlinux 0xffffffff811c2afd
Build id: 3.10.0-693.21.1.el7.x86_64
0x4d
8
migrate_pages
vmlinux 0xffffffff811eb87f
Build id: 3.10.0-693.21.1.el7.x86_64
0x43f
9
? migrate_vma_collect.constprop.51
vmlinux 0xffffffff811e8d50
Build id: 3.10.0-693.21.1.el7.x86_64
0xe0
10
migrate_misplaced_page
vmlinux 0xffffffff811ec54c
Build id: 3.10.0-693.21.1.el7.x86_64
0xcc
11
do_numa_page
vmlinux 0xffffffff811b5398
Build id: 3.10.0-693.21.1.el7.x86_64
0x138
12
handle_mm_fault
vmlinux 0xffffffff811b67a7
Build id: 3.10.0-693.21.1.el7.x86_64
0x6b7
13
? do_futex
vmlinux 0xffffffff810fb77a
Build id: 3.10.0-693.21.1.el7.x86_64
0x12a
14
? update_curr
vmlinux 0xffffffff810d11c4
Build id: 3.10.0-693.21.1.el7.x86_64
0x104
15
__do_page_fault
vmlinux 0xffffffff816bb504
Build id: 3.10.0-693.21.1.el7.x86_64
0x154
16
do_page_fault
vmlinux 0xffffffff816bb835
Build id: 3.10.0-693.21.1.el7.x86_64
0x35
17
page_fault
vmlinux 0xffffffff816b7768
Build id: 3.10.0-693.21.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:1390]
Modules linked in: xt_multiport xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock nls_utf8 isofs sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper vmw_balloon cryptd joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic vmwgfx drm_kms_helper pata_acpi syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1390 Comm: java Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff88076ca9eeb0 ti: ffff8808286cc000 task.ti: ffff8808286cc000
RIP: 0010:[<ffffffff810fd5fa>]  [<ffffffff810fd5fa>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8808286cfb38  EFLAGS: 00000202
RAX: 000000000000000b RBX: 0000000000019580 RCX: ffff880abfd1c6a0
RDX: 000000000000000b RSI: 000000000000000c RDI: 0000000000000000
RBP: ffff8808286cfb70 R08: ffff88017b47d000 R09: ffffffff81328989
R10: ffff8802d549ba00 R11: ffffea00041e6800 R12: ffff8808286cfae8
R13: ffff8808286cfb48 R14: ffff880ab437dee0 R15: ffff8802d5519368
FS:  00007f57f866d700(0000) GS:ffff8802d5480000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5820250a70 CR3: 00000002cdeaa000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff81070b98>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81070d94>] flush_tlb_page+0x54/0xa0
[<ffffffff811c6db4>] ptep_clear_flush+0x54/0x80
[<ffffffff811c0fe2>] try_to_unmap_one+0xb2/0x730
[<ffffffff811c2a3d>] try_to_unmap_anon+0xbd/0x130
[<ffffffff811c2afd>] try_to_unmap+0x4d/0x60
[<ffffffff811eb87f>] migrate_pages+0x43f/0x7a0
[<ffffffff811e8d50>] ? migrate_vma_collect.constprop.51+0xe0/0xe0
[<ffffffff811ec54c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff811b5398>] do_numa_page+0x138/0x1e0
[<ffffffff811b67a7>] handle_mm_fault+0x6b7/0xfa0
[<ffffffff810fb77a>] ? do_futex+0x12a/0x5a0
[<ffffffff810d11c4>] ? update_curr+0x104/0x190
[<ffffffff816bb504>] __do_page_fault+0x154/0x450
[<ffffffff816bb835>] do_page_fault+0x35/0x90
[<ffffffff816b7768>] page_fault+0x28/0x30
Code: 48 63 35 8e 1b a3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 10 b2 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d 1b a3 00 eb b7 0f b6 4d cc 4c 89