Info

Permalink
Function
smp_call_function_many
First occurrence
2020-11-14
Last occurrence
2026-01-31
State
NEW
Components
kernel
Quality
-47

Graphs

Statistics

Operating system Count
CentOS 7 519
CentOS 8 4
Architecture Count
x86_64 523
Related packages Count
kernel 516
 0:3.10.0-1127.el7 192
 0:3.10.0-1160.108.1.el7 94
 0:3.10.0-1160.59.1.el7 63
 0:3.10.0-957.el7 61
 0:3.10.0-1160.49.1.el7 37
 0:3.10.0-957.1.3.el7 25
 0:3.10.0-1127.19.1.el7 7
 0:3.10.0-1160.21.1.el7 6
 0:3.10.0-1160.88.1.el7 5
 0:3.10.0-1160.118.1.el7 4
 0:3.10.0-1160.53.1.el7 3
 0:3.10.0-1160.114.2.el7 2
 0:3.10.0-1160.45.1.el7 2
 0:3.10.0-1160.41.1.el7 2
 0:3.10.0-1062.12.1.el7 1
 0:3.10.0-1062.9.1.el7 1
 0:3.10.0-1160.71.1.el7 1
 0:3.10.0-1062.el7 1
 0:3.10.0-1160.76.1.el7 1
 0:3.10.0-1127.13.1.el7 1
 0:3.10.0-957.21.2.el7 1
 0:3.10.0-1160.81.1.el7 1
 0:3.10.0-1160.66.1.el7 1
 0:3.10.0-1160.el7 1
 0:3.10.0-957.5.1.el7 1
 0:3.10.0-957.21.3.el7 1
 0:3.10.0-1062.18.1.el7 1
kernel-plus 7
 0:3.10.0-957.el7.centos.plus 7

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1385520
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8191181a
Build id: 3.10.0-957.1.3.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8187a598
Build id: 3.10.0-957.1.3.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8187a734
Build id: 3.10.0-957.1.3.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff819fd7f7
Build id: 3.10.0-957.1.3.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffff819f74a8
Build id: 3.10.0-957.1.3.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff819f8d66
Build id: 3.10.0-957.1.3.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff819f90cb
Build id: 3.10.0-957.1.3.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff819f7350
Build id: 3.10.0-957.1.3.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff819f64c0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff819f7ba0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffff819f89f0
Build id: 3.10.0-957.1.3.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffff819f6490
Build id: 3.10.0-957.1.3.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff81a2490f
Build id: 3.10.0-957.1.3.el7.x86_64 		0x43f
14
? migrate_vma_collect.constprop.53
vmlinux 0xffffffff81a215d0
Build id: 3.10.0-957.1.3.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffff81a255bc
Build id: 3.10.0-957.1.3.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffff819e8ffd
Build id: 3.10.0-957.1.3.el7.x86_64 		0x19d
17
handle_pte_fault
vmlinux 0xffffffff819e93c6
Build id: 3.10.0-957.1.3.el7.x86_64 		0x316
18
? futex_wait_queue_me
vmlinux 0xffffffff8190cf14
Build id: 3.10.0-957.1.3.el7.x86_64 		0xa4
19
handle_mm_fault
vmlinux 0xffffffff819ebedd
Build id: 3.10.0-957.1.3.el7.x86_64 		0x39d
20
__do_page_fault
vmlinux 0xffffffff81f6f5e3
Build id: 3.10.0-957.1.3.el7.x86_64 		0x203
21
do_page_fault
vmlinux 0xffffffff81f6f915
Build id: 3.10.0-957.1.3.el7.x86_64 		0x35
22
? error_swapgs
vmlinux 0xffffffff81f6ba96
Build id: 3.10.0-957.1.3.el7.x86_64 		0xa7
23
page_fault
vmlinux 0xffffffff81f6b758
Build id: 3.10.0-957.1.3.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 23s! [s1-agent:22217]
Modules linked in: rpcsec_gss_krb5 nfsv4 nfs fscache cmac arc4 md4 nls_utf8 cifs ccm dns_resolver xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter dgrp(OE) vmw_vsock_vmci_transport vsock ppdev sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc nfit parport libnvdimm vmw_vmci i2c_piix4 binfmt_misc nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm ahci mptspi libahci crct10dif_pclmul crct10dif_common drm ata_piix scsi_transport_spi crc32c_intel e1000 mptscsih libata mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 13 PID: 22217 Comm: s1-agent Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9f691bfcc100 ti: ffff9f69c0cb0000 task.ti: ffff9f69c0cb0000
RIP: 0010:[<ffffffff8191181a>]  [<ffffffff8191181a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9f69c0cb3a28  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff9f691fa207f0
RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff9f69c0cb3a60 R08: ffff9f6274456400 R09: ffffffff81b75839
R10: ffff9f691fd5f160 R11: ffffe1c25f5c8a00 R12: 0000000000000292
R13: ffff9f69c0cb39d8 R14: 00000001c0cb39a0 R15: ffffffff818e3ce4
FS:  00007f89422d8700(0000) GS:ffff9f691fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f89418a2038 CR3: 00000002406c0000 CR4: 00000000000607e0
Call Trace:
[<ffffffff8187a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8187a734>] flush_tlb_page+0x54/0xa0
[<ffffffff819fd7f7>] ptep_clear_flush+0x67/0xa0
[<ffffffff819f74a8>] try_to_unmap_one+0x158/0x850
[<ffffffff819f8d66>] rmap_walk+0x266/0x360
[<ffffffff819f90cb>] try_to_unmap+0x8b/0xe0
[<ffffffff819f7350>] ? page_remove_rmap+0x160/0x160
[<ffffffff819f64c0>] ? invalid_migration_vma+0x30/0x30
[<ffffffff819f7ba0>] ? try_to_unmap_one+0x850/0x850
[<ffffffff819f89f0>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffff819f6490>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff81a2490f>] migrate_pages+0x43f/0x790
[<ffffffff81a215d0>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffff81a255bc>] migrate_misplaced_page+0xcc/0x100
[<ffffffff819e8ffd>] do_numa_page+0x19d/0x250
[<ffffffff819e93c6>] handle_pte_fault+0x316/0xd10
[<ffffffff8190cf14>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff819ebedd>] handle_mm_fault+0x39d/0x9b0
[<ffffffff81f6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff81f6f915>] do_page_fault+0x35/0x90
[<ffffffff81f6ba96>] ? error_swapgs+0xa7/0xbd
[<ffffffff81f6b758>] page_fault+0x28/0x30
Code: 48 63 35 6e 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 54 82 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 82 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1257504
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff98316cba
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9827e608
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff9827e7a4
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff98405638
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff983ff2b8
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff98400b66
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff98400ecb
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff983ff160
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff983fe270
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff983ff9c0
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x860
11
? page_get_anon_vma
vmlinux 0xffffffff98400360
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x80
12
? invalid_mkclean_vma
vmlinux 0xffffffff983fe240
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff9842dbaf
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x35f
14
? __switch_to
vmlinux 0xffffffff9822b621
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x151
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff9842a860
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffff9842e9ac
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffff983f131d
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x19d
18
handle_mm_fault
vmlinux 0xffffffff983f3c8b
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xadb
19
__do_page_fault
vmlinux 0xffffffff98988653
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x213
20
do_page_fault
vmlinux 0xffffffff98988975
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x35
21
page_fault
vmlinux 0xffffffff98984778
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [SyscomIgnite62-:9613]
Modules linked in: cmac arc4 md4 xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nls_utf8 cifs ccm dns_resolver vmw_vsock_vmci_transport vsock sunrpc ppdev sb_edac vmw_balloon coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm ata_piix libahci nfit crct10dif_pclmul libnvdimm
libata crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 9613 Comm: SyscomIgnite62- Kdump: loaded Tainted: G        W    L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8d08d6165230 ti: ffff8d08d6188000 task.ti: ffff8d08d6188000
RIP: 0010:[<ffffffff98316cba>]  [<ffffffff98316cba>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8d08d618ba98  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff8d08efa205a8
RDX: 0000000000000000 RSI: 0000000000000018 RDI: 0000000000000000
RBP: ffff8d08d618bad0 R08: ffff8d063a399000 R09: ffffffff985819f9
R10: ffff8d08efadf160 R11: fffff3ef20bac400 R12: 0000000000000292
R13: ffff8d08d618ba48 R14: ffff8d08d618ba08 R15: ffffffff98425e1a
FS:  00007f2250aca700(0000) GS:ffff8d08efac0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2250ac3770 CR3: 00000004215ce000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9827e608>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9827e7a4>] flush_tlb_page+0x54/0xa0
[<ffffffff98405638>] ptep_clear_flush+0x68/0xa0
[<ffffffff983ff2b8>] try_to_unmap_one+0x158/0x860
[<ffffffff98400b66>] rmap_walk+0x266/0x360
[<ffffffff98400ecb>] try_to_unmap+0x8b/0xe0
[<ffffffff983ff160>] ? page_remove_rmap+0x160/0x160
[<ffffffff983fe270>] ? invalid_migration_vma+0x30/0x30
[<ffffffff983ff9c0>] ? try_to_unmap_one+0x860/0x860
[<ffffffff98400360>] ? page_get_anon_vma+0x80/0x80
[<ffffffff983fe240>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff9842dbaf>] migrate_pages+0x35f/0x7f0
[<ffffffff9822b621>] ? __switch_to+0x151/0x580
[<ffffffff9842a860>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff9842e9ac>] migrate_misplaced_page+0xcc/0x100
[<ffffffff983f131d>] do_numa_page+0x19d/0x250
[<ffffffff983f3c8b>] handle_mm_fault+0xadb/0xfb0
[<ffffffff98988653>] __do_page_fault+0x213/0x500
[<ffffffff98988975>] do_page_fault+0x35/0x90
[<ffffffff98984778>] page_fault+0x28/0x30
Code: 48 63 35 ee 5c c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 e8 f4 98 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 bd 5c c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1170974
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff815170da
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8147e258
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8147e3f4
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff81608108
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff81601d78
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x158
6
? __check_object_size
vmlinux 0xffffffff8164b13a
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x1ca
7
rmap_walk
vmlinux 0xffffffff81603626
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x266
8
try_to_unmap
vmlinux 0xffffffff8160398b
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x8b
9
? page_remove_rmap
vmlinux 0xffffffff81601c20
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x160
10
? invalid_migration_vma
vmlinux 0xffffffff81600d30
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x30
11
? try_to_unmap_one
vmlinux 0xffffffff81602480
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x860
12
? page_get_anon_vma
vmlinux 0xffffffff81602e20
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x80
13
? invalid_mkclean_vma
vmlinux 0xffffffff81600d00
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x20
14
migrate_pages
vmlinux 0xffffffff81630807
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x447
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff8162d3d0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffff8163151c
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffff815f3ced
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x19d
18
handle_mm_fault
vmlinux 0xffffffff815f665b
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xadb
19
? sock_aio_read
vmlinux 0xffffffff81a35731
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x21
20
? do_sync_read
vmlinux 0xffffffff8164d9c3
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x93
21
__do_page_fault
vmlinux 0xffffffff81b90653
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x213
22
do_page_fault
vmlinux 0xffffffff81b90975
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x35
23
page_fault
vmlinux 0xffffffff81b8c778
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [prometheus:12356]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink tun vmw_vsock_vmci_transport vsock sunrpc vfat fat sb_edac nfit ppdev iosf_mbi libnvdimm crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport tpm_crb i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi mptscsih e1000 crct10dif_pclmul crct10dif_common crc32c_intel mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ipmi_msghandler]
CPU: 0 PID: 12356 Comm: prometheus Tainted: G        W    L ------------   3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.16707776.B64.2008070230 08/07/2020
task: ffff944f8914b180 ti: ffff944f8f034000 task.ti: ffff944f8f034000
RIP: 0010:[<ffffffff815170da>]  [<ffffffff815170da>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff944f8f037aa0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: 000800008147ddd0 RCX: ffff944b2bd1fc10
RDX: 0000000000000004 RSI: 000000000000000c RDI: 0000000000000000
RBP: ffff944f8f037ad8 R08: ffff94477fd49400 R09: ffffffff81786bf9
R10: ffff944b2bc1f160 R11: ffffebfe660bea00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff944f8f037a50 R15: ffffffff814a4077
FS:  000000c04219a490(0000) GS:ffff944b2bc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c069ceaaa0 CR3: 0000000a21aea000 CR4: 00000000001607f0
Call Trace:
[<ffffffff8147e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8147e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffff81608108>] ptep_clear_flush+0x68/0xa0
[<ffffffff81601d78>] try_to_unmap_one+0x158/0x860
[<ffffffff8164b13a>] ? __check_object_size+0x1ca/0x250
[<ffffffff81603626>] rmap_walk+0x266/0x360
[<ffffffff8160398b>] try_to_unmap+0x8b/0xe0
[<ffffffff81601c20>] ? page_remove_rmap+0x160/0x160
[<ffffffff81600d30>] ? invalid_migration_vma+0x30/0x30
[<ffffffff81602480>] ? try_to_unmap_one+0x860/0x860
[<ffffffff81602e20>] ? page_get_anon_vma+0x80/0x80
[<ffffffff81600d00>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff81630807>] migrate_pages+0x447/0x7f0
[<ffffffff8162d3d0>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff8163151c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff815f3ced>] do_numa_page+0x19d/0x250
[<ffffffff815f665b>] handle_mm_fault+0xadb/0xfb0
[<ffffffff81a35731>] ? sock_aio_read+0x21/0x30
[<ffffffff8164d9c3>] ? do_sync_read+0x93/0xe0
[<ffffffff81b90653>] __do_page_fault+0x213/0x500
[<ffffffff81b90975>] do_page_fault+0x35/0x90
[<ffffffff81b8c778>] page_fault+0x28/0x30
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 15 82 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1146587
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff96f1e1ca
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff96e81fa8
Build id: 3.10.0-1160.88.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff96e82144
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff970141b8
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff9700dc80
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x160
6
? put_prev_entity
vmlinux 0xffffffff96eea1ab
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x8b
7
rmap_walk
vmlinux 0xffffffff9700f586
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x276
8
try_to_unmap
vmlinux 0xffffffff9700f8eb
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x8b
9
? page_remove_rmap
vmlinux 0xffffffff9700db20
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x160
10
? invalid_migration_vma
vmlinux 0xffffffff9700cc00
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x30
11
? try_to_unmap_one
vmlinux 0xffffffff9700e380
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x860
12
? page_get_anon_vma
vmlinux 0xffffffff9700ed50
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x80
13
? invalid_mkclean_vma
vmlinux 0xffffffff9700cbd0
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x20
14
migrate_pages
vmlinux 0xffffffff9703d497
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x447
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff97039ff0
Build id: 3.10.0-1160.88.1.el7.x86_64 		0xf0
16
migrate_misplaced_page
vmlinux 0xffffffff9703e1dc
Build id: 3.10.0-1160.88.1.el7.x86_64 		0xdc
17
do_numa_page
vmlinux 0xffffffff96fff7dd
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x19d
18
handle_mm_fault
vmlinux 0xffffffff970021d3
Build id: 3.10.0-1160.88.1.el7.x86_64 		0xaf3
19
__do_page_fault
vmlinux 0xffffffff975bf7d3
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x213
20
? fput
vmlinux 0xffffffff9705e493
Build id: 3.10.0-1160.88.1.el7.x86_64 		0x13 
NMI watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [conn3422:11833]
Modules linked in: binfmt_misc tcp_diag udp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun xt_comment bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc vfat fat sb_edac intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr vmxnet3 sg i2c_piix4 vmw_vmci parport_pc parport tpm_crb ip_tables xfs libcrc32c vmwgfx ata_generic pata_acpi drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix
drm nfit libata mptsas crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm scsi_transport_sas serio_raw mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: rpm_deathwatch]
CPU: 10 PID: 11833 Comm: conn3422 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.21100432.B64.2301110304 01/11/2023
task: ffff9719e6176300 ti: ffff96ede272c000 task.ti: ffff96ede272c000
RIP: 0010:[<ffffffff96f1e1ca>]  [<ffffffff96f1e1ca>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff96ede272faa0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 0008000096e81b10 RCX: ffff97021dc607c8
RDX: 0000000000000001 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff96ede272fad8 R08: ffff9702c2a01c00 R09: ffffffff9719c669
R10: ffff971affc9f160 R11: ffffd786616eb200 R12: 0000000000000282
R13: 0000000000000282 R14: ffff96ede272fa50 R15: 000000000020001f
FS:  00007f2ab42c9700(0000) GS:ffff971affc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe25febe000 CR3: 00000017f5832000 CR4: 00000000000607e0
Call Trace:
[<ffffffff96e81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff96e82144>] flush_tlb_page+0x54/0xa0
[<ffffffff970141b8>] ptep_clear_flush+0x68/0xa0
[<ffffffff9700dc80>] try_to_unmap_one+0x160/0x860
[<ffffffff96eea1ab>] ? put_prev_entity+0x8b/0x400
[<ffffffff9700f586>] rmap_walk+0x276/0x370
[<ffffffff9700f8eb>] try_to_unmap+0x8b/0xf0
[<ffffffff9700db20>] ? page_remove_rmap+0x160/0x160
[<ffffffff9700cc00>] ? invalid_migration_vma+0x30/0x30
[<ffffffff9700e380>] ? try_to_unmap_one+0x860/0x860
[<ffffffff9700ed50>] ? page_get_anon_vma+0x80/0x80
[<ffffffff9700cbd0>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff9703d497>] migrate_pages+0x447/0x7f0
[<ffffffff97039ff0>] ? migrate_vma_collect.constprop.52+0xf0/0xf0
[<ffffffff9703e1dc>] migrate_misplaced_page+0xdc/0x120
[<ffffffff96fff7dd>] do_numa_page+0x19d/0x260
[<ffffffff970021d3>] handle_mm_fault+0xaf3/0x1190
[<ffffffff975bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff9705e493>] ? fput+0x13/0x20
Complete report #1018052
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb7d1e23a
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffb7c81fa8
Build id: 3.10.0-1160.108.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffb7c82144
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffb7e14228
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffffb7e0dcf0
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x160
6
rmap_walk
vmlinux 0xffffffffb7e0f5f6
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x276
7
try_to_unmap
vmlinux 0xffffffffb7e0f95b
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffffb7e0db90
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffffb7e0cc70
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffffb7e0e3f0
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x860
11
? page_get_anon_vma
vmlinux 0xffffffffb7e0edc0
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x80
12
? invalid_mkclean_vma
vmlinux 0xffffffffb7e0cc40
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffffb7e3d507
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x447
14
? migrate_vma_collect.constprop.52
vmlinux 0xffffffffb7e3a060
Build id: 3.10.0-1160.108.1.el7.x86_64 		0xf0
15
migrate_misplaced_page
vmlinux 0xffffffffb7e3e24c
Build id: 3.10.0-1160.108.1.el7.x86_64 		0xdc
16
do_numa_page
vmlinux 0xffffffffb7dff84d
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x19d
17
handle_mm_fault
vmlinux 0xffffffffb7e02243
Build id: 3.10.0-1160.108.1.el7.x86_64 		0xaf3
18
? cp_new_stat
vmlinux 0xffffffffb7e61ca5
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x165
19
__do_page_fault
vmlinux 0xffffffffb83bf7d3
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x213
20
do_page_fault
vmlinux 0xffffffffb83bfb05
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x35
21
page_fault
vmlinux 0xffffffffb83bb7b8
Build id: 3.10.0-1160.108.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [promtail:22246]
Modules linked in: xt_owner xt_REDIRECT nf_nat_redirect xt_multiport ipt_rpfilter iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel rpcsec_gss_krb5 veth auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipportip ip_set_hash_ipport dummy nf_tables ip_set ip_vs_sh ip_vs_wrr ip_vs_rr ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_filter ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6_tables xt_mark xt_comment iptable_mangle xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype iptable_filter scini(POE) ip_vs nf_conntrack binfmt_misc vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon
sg parport_pc joydev parport pcspkr vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi scsi_transport_spi mptscsih drm libata crc32c_intel serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc overlay(T)
CPU: 9 PID: 22246 Comm: promtail Tainted: P        W  OEL ------------ T 3.10.0-1160.108.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffffa05ed2fcd280 ti: ffffa05ee8d44000 task.ti: ffffa05ee8d44000
RIP: 0010:[<ffffffffb7d1e23a>]  [<ffffffffb7d1e23a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffffa05ee8d47aa0  EFLAGS: 00000202
RAX: 000000000000000b RBX: 00080000b7c81b10 RCX: ffffa05fffda07f0
RDX: 000000000000000b RSI: 000000000000000c RDI: 0000000000000000
RBP: ffffa05ee8d47ad8 R08: ffffa05f7d538000 R09: ffffffffb7f9c7b9
R10: ffffa05fffc9f160 R11: fffff0f308d4fc00 R12: 0000000000000282
R13: 0000000000000282 R14: ffffa05ee8d47a50 R15: 0000000000000003
FS:  00007f212a7fc700(0000) GS:ffffa05fffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0040df000 CR3: 00000002e3c3c000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb7c81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb7c82144>] flush_tlb_page+0x54/0xa0
[<ffffffffb7e14228>] ptep_clear_flush+0x68/0xa0
[<ffffffffb7e0dcf0>] try_to_unmap_one+0x160/0x860
[<ffffffffb7e0f5f6>] rmap_walk+0x276/0x370
[<ffffffffb7e0f95b>] try_to_unmap+0x8b/0xf0
[<ffffffffb7e0db90>] ? page_remove_rmap+0x160/0x160
[<ffffffffb7e0cc70>] ? invalid_migration_vma+0x30/0x30
[<ffffffffb7e0e3f0>] ? try_to_unmap_one+0x860/0x860
[<ffffffffb7e0edc0>] ? page_get_anon_vma+0x80/0x80
[<ffffffffb7e0cc40>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffffb7e3d507>] migrate_pages+0x447/0x7f0
[<ffffffffb7e3a060>] ? migrate_vma_collect.constprop.52+0xf0/0xf0
[<ffffffffb7e3e24c>] migrate_misplaced_page+0xdc/0x120
[<ffffffffb7dff84d>] do_numa_page+0x19d/0x260
[<ffffffffb7e02243>] handle_mm_fault+0xaf3/0x1190
[<ffffffffb7e61ca5>] ? cp_new_stat+0x165/0x1a0
[<ffffffffb83bf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb83bfb05>] do_page_fault+0x35/0x90
[<ffffffffb83bb7b8>] page_fault+0x28/0x30
Code: 48 63 35 8e 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 80 19 95 b8 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d 18 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #968255
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa3311a8a
Build id: 3.10.0-957.21.3.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa327a6a8
Build id: 3.10.0-957.21.3.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffa327a844
Build id: 3.10.0-957.21.3.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffa33fdb97
Build id: 3.10.0-957.21.3.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffffa33f7848
Build id: 3.10.0-957.21.3.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffffa33f9106
Build id: 3.10.0-957.21.3.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffffa33f946b
Build id: 3.10.0-957.21.3.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffffa33f76f0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffffa33f6860
Build id: 3.10.0-957.21.3.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffffa33f7f40
Build id: 3.10.0-957.21.3.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffffa33f8d90
Build id: 3.10.0-957.21.3.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffffa33f6830
Build id: 3.10.0-957.21.3.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffffa3424d2f
Build id: 3.10.0-957.21.3.el7.x86_64 		0x43f
14
? iomap_write_end
vmlinux 0xffffffffa34ac9b0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x80
15
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffa34219f0
Build id: 3.10.0-957.21.3.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffffa34259dc
Build id: 3.10.0-957.21.3.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffffa33e939d
Build id: 3.10.0-957.21.3.el7.x86_64 		0x19d
18
handle_pte_fault
vmlinux 0xffffffffa33e9766
Build id: 3.10.0-957.21.3.el7.x86_64 		0x316
19
? kmem_cache_free
vmlinux 0xffffffffa341bdf2
Build id: 3.10.0-957.21.3.el7.x86_64 		0x1e2
20
handle_mm_fault
vmlinux 0xffffffffa33ec27d
Build id: 3.10.0-957.21.3.el7.x86_64 		0x39d
21
? __schedule
vmlinux 0xffffffffa396897c
Build id: 3.10.0-957.21.3.el7.x86_64 		0x32c
22
? __schedule
vmlinux 0xffffffffa3968988
Build id: 3.10.0-957.21.3.el7.x86_64 		0x338
23
__do_page_fault
vmlinux 0xffffffffa3970603
Build id: 3.10.0-957.21.3.el7.x86_64 		0x203
24
do_page_fault
vmlinux 0xffffffffa3970925
Build id: 3.10.0-957.21.3.el7.x86_64 		0x35
25
page_fault
vmlinux 0xffffffffa396c768
Build id: 3.10.0-957.21.3.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 2395s! [mysqld:21308]
Modules linked in: gsch(OE) redirfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter dsa_filter(POE) vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common serio_raw crc32c_intel vmxnet3 ahci libahci vmw_pvscsi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm
drm ata_piix libata nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 14 PID: 21308 Comm: mysqld Kdump: loaded Tainted: P           OEL ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9e24a0b24100 ti: ffff9e1889474000 task.ti: ffff9e1889474000
RIP: 0010:[<ffffffffa3311a8a>]  [<ffffffffa3311a8a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9e1889477a28  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff9e1ea7c60818
RDX: 0000000000000001 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9e1889477a60 R08: ffff9e1f36e15c00 R09: ffffffffa3575e39
R10: ffff9e24bfd9f160 R11: fffff19fc0079a00 R12: 0000000000000292
R13: ffff9e18894779d8 R14: ffff9e249fa303b0 R15: ffff9e24a58ab6c8
FS:  00007f4d97873700(0000) GS:ffff9e24bfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f658d67d080 CR3: 0000000624bf2000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa327a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa327a844>] flush_tlb_page+0x54/0xa0
[<ffffffffa33fdb97>] ptep_clear_flush+0x67/0xa0
[<ffffffffa33f7848>] try_to_unmap_one+0x158/0x850
[<ffffffffa33f9106>] rmap_walk+0x266/0x360
[<ffffffffa33f946b>] try_to_unmap+0x8b/0xe0
[<ffffffffa33f76f0>] ? page_remove_rmap+0x160/0x160
[<ffffffffa33f6860>] ? invalid_migration_vma+0x30/0x30
[<ffffffffa33f7f40>] ? try_to_unmap_one+0x850/0x850
[<ffffffffa33f8d90>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffffa33f6830>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffffa3424d2f>] migrate_pages+0x43f/0x790
[<ffffffffa34ac9b0>] ? iomap_write_end+0x80/0x80
[<ffffffffa34219f0>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffa34259dc>] migrate_misplaced_page+0xcc/0x100
[<ffffffffa33e939d>] do_numa_page+0x19d/0x250
[<ffffffffa33e9766>] handle_pte_fault+0x316/0xd10
[<ffffffffa341bdf2>] ? kmem_cache_free+0x1e2/0x200
[<ffffffffa33ec27d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa396897c>] ? __schedule+0x32c/0x860
[<ffffffffa3968988>] ? __schedule+0x338/0x860
[<ffffffffa3970603>] __do_page_fault+0x203/0x4f0
[<ffffffffa3970925>] do_page_fault+0x35/0x90
[<ffffffffa396c768>] page_fault+0x28/0x30
Code: 48 63 35 3e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 bc f4 a3 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 0d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #885428
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa111173a
Build id: 3.10.0-957.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa107a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffa107a734
Build id: 3.10.0-957.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffa11fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffffa11f7238
Build id: 3.10.0-957.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffffa11f8af6
Build id: 3.10.0-957.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffffa11f8e5b
Build id: 3.10.0-957.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffffa11f70e0
Build id: 3.10.0-957.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffffa11f6250
Build id: 3.10.0-957.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffffa11f7930
Build id: 3.10.0-957.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffffa11f8780
Build id: 3.10.0-957.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffffa11f6220
Build id: 3.10.0-957.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffffa122469f
Build id: 3.10.0-957.el7.x86_64 		0x43f
14
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffa1221360
Build id: 3.10.0-957.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffffa122534c
Build id: 3.10.0-957.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffffa11e8d8d
Build id: 3.10.0-957.el7.x86_64 		0x19d
17
handle_pte_fault
vmlinux 0xffffffffa11e9156
Build id: 3.10.0-957.el7.x86_64 		0x316
18
? get_futex_key
vmlinux 0xffffffffa110cb18
Build id: 3.10.0-957.el7.x86_64 		0x1c8
19
handle_mm_fault
vmlinux 0xffffffffa11ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
20
__do_page_fault
vmlinux 0xffffffffa176f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
21
do_page_fault
vmlinux 0xffffffffa176f915
Build id: 3.10.0-957.el7.x86_64 		0x35
22
page_fault
vmlinux 0xffffffffa176b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#16 stuck for 22s! [DeploymentScann:23088]
Modules linked in: bmhook(OE) tmhook(OE) dsa_filter(POE) dsa_filter_hook(OE) nfsv3 nfs_acl ip6table_filter ip6_tables tcp_diag inet_diag iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect sysimgblt crc32c_intel fb_sys_fops ttm serio_raw ahci libahci drm ata_piix libata vmxnet3 nfit vmw_pvscsi libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log
dm_mod [last unloaded: bmsensor]
CPU: 16 PID: 23088 Comm: DeploymentScann Tainted: P           OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9f6fe47ec100 ti: ffff9f6fe937c000 task.ti: ffff9f6fe937c000
RIP: 0010:[<ffffffffa111173a>]  [<ffffffffa111173a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9f6fe937fa28  EFLAGS: 00000202
RAX: 000000000000000e RBX: 00000000000000fc RCX: ffff9f6fefca0940
RDX: 000000000000000e RSI: 0000000000000018 RDI: 0000000000000000
RBP: ffff9f6fe937fa60 R08: ffff9f6c7a4ef000 R09: ffffffffa13755c9
R10: ffff9f6fefd1f160 R11: ffffe7fa70adf400 R12: 0000000000000292
R13: ffff9f6fe937f9d8 R14: 00000000955388ac R15: ffffffffa124ce98
FS:  00007f65b35b8700(0000) GS:ffff9f6fefd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6711d7a000 CR3: 0000000c2e2f8000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa107a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa107a734>] flush_tlb_page+0x54/0xa0
[<ffffffffa11fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffffa11f7238>] try_to_unmap_one+0x158/0x850
[<ffffffffa11f8af6>] rmap_walk+0x266/0x360
[<ffffffffa11f8e5b>] try_to_unmap+0x8b/0xe0
[<ffffffffa11f70e0>] ? page_remove_rmap+0x160/0x160
[<ffffffffa11f6250>] ? invalid_migration_vma+0x30/0x30
[<ffffffffa11f7930>] ? try_to_unmap_one+0x850/0x850
[<ffffffffa11f8780>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffffa11f6220>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffffa122469f>] migrate_pages+0x43f/0x790
[<ffffffffa1221360>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffa122534c>] migrate_misplaced_page+0xcc/0x100
[<ffffffffa11e8d8d>] do_numa_page+0x19d/0x250
[<ffffffffa11e9156>] handle_pte_fault+0x316/0xd10
[<ffffffffa110cb18>] ? get_futex_key+0x1c8/0x2c0
[<ffffffffa11ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa176f5e3>] __do_page_fault+0x203/0x500
[<ffffffffa176f915>] do_page_fault+0x35/0x90
[<ffffffffa176b758>] page_fault+0x28/0x30
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 d4 a1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #778968
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8d71185e
Build id: 3.10.0-957.5.1.el7.x86_64 		0x20e
2
native_flush_tlb_others
vmlinux 0xffffffff8d67a5c8
Build id: 3.10.0-957.5.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8d67a764
Build id: 3.10.0-957.5.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff8d7fd937
Build id: 3.10.0-957.5.1.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffff8d7f75e8
Build id: 3.10.0-957.5.1.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff8d7f8ea6
Build id: 3.10.0-957.5.1.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff8d7f920b
Build id: 3.10.0-957.5.1.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff8d7f7490
Build id: 3.10.0-957.5.1.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff8d7f6600
Build id: 3.10.0-957.5.1.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff8d7f7ce0
Build id: 3.10.0-957.5.1.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffff8d7f8b30
Build id: 3.10.0-957.5.1.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffff8d7f65d0
Build id: 3.10.0-957.5.1.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff8d824acf
Build id: 3.10.0-957.5.1.el7.x86_64 		0x43f
14
? migrate_vma_collect.constprop.53
vmlinux 0xffffffff8d821790
Build id: 3.10.0-957.5.1.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffff8d82577c
Build id: 3.10.0-957.5.1.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffff8d7e913d
Build id: 3.10.0-957.5.1.el7.x86_64 		0x19d
17
handle_pte_fault
vmlinux 0xffffffff8d7e9506
Build id: 3.10.0-957.5.1.el7.x86_64 		0x316
18
? schedule_hrtimeout_range_clock
vmlinux 0xffffffff8dd66e0e
Build id: 3.10.0-957.5.1.el7.x86_64 		0xbe
19
handle_mm_fault
vmlinux 0xffffffff8d7ec01d
Build id: 3.10.0-957.5.1.el7.x86_64 		0x39d
20
__do_page_fault
vmlinux 0xffffffff8dd6f5e3
Build id: 3.10.0-957.5.1.el7.x86_64 		0x203
21
do_page_fault
vmlinux 0xffffffff8dd6f915
Build id: 3.10.0-957.5.1.el7.x86_64 		0x35
22
page_fault
vmlinux 0xffffffff8dd6b758
Build id: 3.10.0-957.5.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#20 stuck for 25s! [migration/20:109]
NMI watchdog: BUG: soft lockup - CPU#17 stuck for 28s! [java:3424]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables xt_owner iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc nfit libnvdimm ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptspi
crct10dif_pclmul scsi_transport_spi libata crct10dif_common mptscsih crc32c_intel serio_raw vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 17 PID: 3424 Comm: java Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff90731d7ba080 ti: ffff9072f03a4000 task.ti: ffff9072f03a4000
RIP: 0010:[<ffffffff8d71185e>]  [<ffffffff8d71185e>] smp_call_function_many+0x20e/0x270
RSP: 0000:ffff9072f03a7a28  EFLAGS: 00000202
RAX: 000000000000002d RBX: 00000000000000fc RCX: ffff907b1f560890
RDX: 000000000000002d RSI: 0000000000000030 RDI: 0000000000000000
RBP: ffff9072f03a7a60 R08: ffff906c74738400 R09: ffffffff8d975ba9
R10: ffff90731fc5f160 R11: ffffc4430d4ee600 R12: 0000000000000292
R13: ffff9072f03a79d8 R14: 00000001f03a7998 R15: ffffffff8d6e3d24
FS:  00007f1fbc9ab700(0000) GS:ffff90731fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1fbc9a9828 CR3: 0000001013ec4000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff8d67a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8d67a764>] flush_tlb_page+0x54/0xa0
[<ffffffff8d7fd937>] ptep_clear_flush+0x67/0xa0
[<ffffffff8d7f75e8>] try_to_unmap_one+0x158/0x850
[<ffffffff8d7f8ea6>] rmap_walk+0x266/0x360
[<ffffffff8d7f920b>] try_to_unmap+0x8b/0xe0
[<ffffffff8d7f7490>] ? page_remove_rmap+0x160/0x160
[<ffffffff8d7f6600>] ? invalid_migration_vma+0x30/0x30
[<ffffffff8d7f7ce0>] ? try_to_unmap_one+0x850/0x850
[<ffffffff8d7f8b30>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffff8d7f65d0>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff8d824acf>] migrate_pages+0x43f/0x790
[<ffffffff8d821790>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffff8d82577c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff8d7e913d>] do_numa_page+0x19d/0x250
[<ffffffff8d7e9506>] handle_pte_fault+0x316/0xd10
[<ffffffff8dd66e0e>] ? schedule_hrtimeout_range_clock+0xbe/0x150
[<ffffffff8d7ec01d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8dd6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff8dd6f915>] do_page_fault+0x35/0x90
[<ffffffff8dd6b758>] page_fault+0x28/0x30
Code: 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 34 8e f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 fd 81 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #598071
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffbc71181a
Build id: 3.10.0-957.1.3.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffbc67a598
Build id: 3.10.0-957.1.3.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffbc67a734
Build id: 3.10.0-957.1.3.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffbc7fd7f7
Build id: 3.10.0-957.1.3.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffffbc7f74a8
Build id: 3.10.0-957.1.3.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffffbc7f8d66
Build id: 3.10.0-957.1.3.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffffbc7f90cb
Build id: 3.10.0-957.1.3.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffffbc7f7350
Build id: 3.10.0-957.1.3.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffffbc7f64c0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffffbc7f7ba0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffffbc7f89f0
Build id: 3.10.0-957.1.3.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffffbc7f6490
Build id: 3.10.0-957.1.3.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffffbc82490f
Build id: 3.10.0-957.1.3.el7.x86_64 		0x43f
14
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffbc8215d0
Build id: 3.10.0-957.1.3.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffffbc8255bc
Build id: 3.10.0-957.1.3.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffffbc7e8ffd
Build id: 3.10.0-957.1.3.el7.x86_64 		0x19d
17
handle_pte_fault
vmlinux 0xffffffffbc7e93c6
Build id: 3.10.0-957.1.3.el7.x86_64 		0x316
18
? schedule
vmlinux 0xffffffffbcd67c49
Build id: 3.10.0-957.1.3.el7.x86_64 		0x29
19
handle_mm_fault
vmlinux 0xffffffffbc7ebedd
Build id: 3.10.0-957.1.3.el7.x86_64 		0x39d
20
__do_page_fault
vmlinux 0xffffffffbcd6f5e3
Build id: 3.10.0-957.1.3.el7.x86_64 		0x203
21
do_page_fault
vmlinux 0xffffffffbcd6f915
Build id: 3.10.0-957.1.3.el7.x86_64 		0x35
22
? error_swapgs
vmlinux 0xffffffffbcd6ba96
Build id: 3.10.0-957.1.3.el7.x86_64 		0xa7
23
page_fault
vmlinux 0xffffffffbcd6b758
Build id: 3.10.0-957.1.3.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 23s! [val_auto_daemon:54090]
Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter dgrp(OE) vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev pcspkr sg i2c_piix4 vmw_vmci nfit parport_pc parport libnvdimm nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm mptspi ahci scsi_transport_spi crct10dif_pclmul crct10dif_common libahci ata_piix mptscsih drm serio_raw crc32c_intel libata mptbase e1000 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: redirfs]
CPU: 9 PID: 54090 Comm: val_auto_daemon Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9b242db32080 ti: ffff9b1ac396c000 task.ti: ffff9b1ac396c000
RIP: 0010:[<ffffffffbc71181a>]  [<ffffffffbc71181a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9b1ac396fa28  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000000000001b700 RCX: ffff9b19948207c8
RDX: 0000000000000000 RSI: 0000000000000040 RDI: 0000000000000000
RBP: ffff9b1ac396fa60 R08: ffff9b103089c400 R09: ffffffffbc975839
R10: ffff9b1994a5f160 R11: fffffd7f46501e00 R12: ffff9b1ac396f9d8
R13: 0000000010ffb980 R14: 00ff9b11a145cf00 R15: ffff9b19901d1000
FS:  00007f59eed54700(0000) GS:ffff9b1994a40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f59d81b7000 CR3: 00000011f5898000 CR4: 00000000000607e0
Call Trace:
[<ffffffffbc67a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbc67a734>] flush_tlb_page+0x54/0xa0
[<ffffffffbc7fd7f7>] ptep_clear_flush+0x67/0xa0
[<ffffffffbc7f74a8>] try_to_unmap_one+0x158/0x850
[<ffffffffbc7f8d66>] rmap_walk+0x266/0x360
[<ffffffffbc7f90cb>] try_to_unmap+0x8b/0xe0
[<ffffffffbc7f7350>] ? page_remove_rmap+0x160/0x160
[<ffffffffbc7f64c0>] ? invalid_migration_vma+0x30/0x30
[<ffffffffbc7f7ba0>] ? try_to_unmap_one+0x850/0x850
[<ffffffffbc7f89f0>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffffbc7f6490>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffffbc82490f>] migrate_pages+0x43f/0x790
[<ffffffffbc8215d0>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffbc8255bc>] migrate_misplaced_page+0xcc/0x100
[<ffffffffbc7e8ffd>] do_numa_page+0x19d/0x250
[<ffffffffbc7e93c6>] handle_pte_fault+0x316/0xd10
[<ffffffffbcd67c49>] ? schedule+0x29/0x70
[<ffffffffbc7ebedd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffbcd6f5e3>] __do_page_fault+0x203/0x500
[<ffffffffbcd6f915>] do_page_fault+0x35/0x90
[<ffffffffbcd6ba96>] ? error_swapgs+0xa7/0xbd
[<ffffffffbcd6b758>] page_fault+0x28/0x30
Code: 48 63 35 6e 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 34 bd f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 82 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #540256
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff88d1173a
Build id: 3.10.0-957.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff88c7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff88c7a734
Build id: 3.10.0-957.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff88dfd587
Build id: 3.10.0-957.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffff88df7238
Build id: 3.10.0-957.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff88df8af6
Build id: 3.10.0-957.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff88df8e5b
Build id: 3.10.0-957.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff88df70e0
Build id: 3.10.0-957.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff88df6250
Build id: 3.10.0-957.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff88df7930
Build id: 3.10.0-957.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffff88df8780
Build id: 3.10.0-957.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffff88df6220
Build id: 3.10.0-957.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff88e2469f
Build id: 3.10.0-957.el7.x86_64 		0x43f
14
? migrate_vma_collect.constprop.53
vmlinux 0xffffffff88e21360
Build id: 3.10.0-957.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffff88e2534c
Build id: 3.10.0-957.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffff88de8d8d
Build id: 3.10.0-957.el7.x86_64 		0x19d
17
handle_pte_fault
vmlinux 0xffffffff88de9156
Build id: 3.10.0-957.el7.x86_64 		0x316
18
? wake_up_q
vmlinux 0xffffffff88cd676b
Build id: 3.10.0-957.el7.x86_64 		0x5b
19
handle_mm_fault
vmlinux 0xffffffff88debc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
20
__do_page_fault
vmlinux 0xffffffff8936f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
21
do_page_fault
vmlinux 0xffffffff8936f915
Build id: 3.10.0-957.el7.x86_64 		0x35
22
page_fault
vmlinux 0xffffffff8936b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [influxd:32216]
Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common ahci crc32c_intel serio_raw libahci vmxnet3 vmw_pvscsi vmwgfx ata_piix drm_kms_helper libata syscopyarea sysfillrect sysimgblt fb_sys_fops ttm
drm nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 32216 Comm: influxd Kdump: loaded Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff88885de530c0 ti: ffff8885c7eb0000 task.ti: ffff8885c7eb0000
RIP: 0010:[<ffffffff88d1173a>]  [<ffffffff88d1173a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8885c7eb3a28  EFLAGS: 00000202
RAX: 0000000000000007 RBX: 00000000000000fc RCX: ffff88886fde0598
RDX: 0000000000000007 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff8885c7eb3a60 R08: ffff8885ba2c2800 R09: ffffffff88f755c9
R10: ffff88886fd5f160 R11: ffffd353d815f800 R12: 0000000000000292
R13: ffff8885c7eb39d8 R14: ffff8885c7eb3990 R15: ffffffff88c61253
FS:  00007fdc14d5c700(0000) GS:ffff88886fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdbb17a2d8c CR3: 000000030aec8000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff88c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff88c7a734>] flush_tlb_page+0x54/0xa0
[<ffffffff88dfd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff88df7238>] try_to_unmap_one+0x158/0x850
[<ffffffff88df8af6>] rmap_walk+0x266/0x360
[<ffffffff88df8e5b>] try_to_unmap+0x8b/0xe0
[<ffffffff88df70e0>] ? page_remove_rmap+0x160/0x160
[<ffffffff88df6250>] ? invalid_migration_vma+0x30/0x30
[<ffffffff88df7930>] ? try_to_unmap_one+0x850/0x850
[<ffffffff88df8780>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffff88df6220>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff88e2469f>] migrate_pages+0x43f/0x790
[<ffffffff88e21360>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffff88e2534c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff88de8d8d>] do_numa_page+0x19d/0x250
[<ffffffff88de9156>] handle_pte_fault+0x316/0xd10
[<ffffffff88cd676b>] ? wake_up_q+0x5b/0x80
[<ffffffff88debc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8936f5e3>] __do_page_fault+0x203/0x500
[<ffffffff8936f915>] do_page_fault+0x35/0x90
[<ffffffff8936b758>] page_fault+0x28/0x30
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 89 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #505472
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff89d1173e
Build id: 3.10.0-957.el7.x86_64 		0x20e
2
native_flush_tlb_others
vmlinux 0xffffffff89c7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff89c7a734
Build id: 3.10.0-957.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff89dfd587
Build id: 3.10.0-957.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffff89df7238
Build id: 3.10.0-957.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff89df8af6
Build id: 3.10.0-957.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff89df8e5b
Build id: 3.10.0-957.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff89df70e0
Build id: 3.10.0-957.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff89df6250
Build id: 3.10.0-957.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff89df7930
Build id: 3.10.0-957.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffff89df8780
Build id: 3.10.0-957.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffff89df6220
Build id: 3.10.0-957.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff89e2469f
Build id: 3.10.0-957.el7.x86_64 		0x43f
14
? migrate_vma_collect.constprop.53
vmlinux 0xffffffff89e21360
Build id: 3.10.0-957.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffff89e2534c
Build id: 3.10.0-957.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffff89de8d8d
Build id: 3.10.0-957.el7.x86_64 		0x19d
17
handle_pte_fault
vmlinux 0xffffffff89de9156
Build id: 3.10.0-957.el7.x86_64 		0x316
18
? futex_wait_queue_me
vmlinux 0xffffffff89d0ce34
Build id: 3.10.0-957.el7.x86_64 		0xa4
19
handle_mm_fault
vmlinux 0xffffffff89debc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
20
__do_page_fault
vmlinux 0xffffffff8a36f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
21
do_page_fault
vmlinux 0xffffffff8a36f915
Build id: 3.10.0-957.el7.x86_64 		0x35
22
page_fault
vmlinux 0xffffffff8a36b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [srepd:522624]
Modules linked in: dattobd(OE) sfs(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc nfit libnvdimm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom
crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm mptspi scsi_transport_spi mptscsih ahci vmxnet3 libahci mptbase drm_panel_orientation_quirks ata_piix libata dm_mirror dm_region_hash dm_log dm_mod [last unloaded: dattobd]
CPU: 6 PID: 522624 Comm: srepd Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff98678b38b0c0 ti: ffff9866f75f0000 task.ti: ffff9866f75f0000
RIP: 0010:[<ffffffff89d1173e>]  [<ffffffff89d1173e>] smp_call_function_many+0x20e/0x270
RSP: 0000:ffff9866f75f3a28  EFLAGS: 00000202
RAX: 0000000000000007 RBX: 00000000000000fc RCX: ffff9869fc920a28
RDX: 0000000000000007 RSI: 0000000000000014 RDI: 0000000000000000
RBP: ffff9866f75f3a60 R08: ffff9869afc08800 R09: ffffffff89f755c9
R10: ffff9869fc81f160 R11: fffffb850dac9a00 R12: 0000000000000292
R13: ffff9866f75f39d8 R14: 00000001f75f3ad8 R15: ffffffff89ce3c04
FS:  00007f6c43646700(0000) GS:ffff9869fc800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6a7c000024 CR3: 00000005182cc000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff89c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff89c7a734>] flush_tlb_page+0x54/0xa0
[<ffffffff89dfd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff89df7238>] try_to_unmap_one+0x158/0x850
[<ffffffff89df8af6>] rmap_walk+0x266/0x360
[<ffffffff89df8e5b>] try_to_unmap+0x8b/0xe0
[<ffffffff89df70e0>] ? page_remove_rmap+0x160/0x160
[<ffffffff89df6250>] ? invalid_migration_vma+0x30/0x30
[<ffffffff89df7930>] ? try_to_unmap_one+0x850/0x850
[<ffffffff89df8780>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffff89df6220>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff89e2469f>] migrate_pages+0x43f/0x790
[<ffffffff89e21360>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffff89e2534c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff89de8d8d>] do_numa_page+0x19d/0x250
[<ffffffff89de9156>] handle_pte_fault+0x316/0xd10
[<ffffffff89d0ce34>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff89debc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8a36f5e3>] __do_page_fault+0x203/0x500
[<ffffffff8a36f915>] do_page_fault+0x35/0x90
[<ffffffff8a36b758>] page_fault+0x28/0x30
Code: 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 8a f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #403765
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9951760a
Build id: 3.10.0-1127.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9947e8f8
Build id: 3.10.0-1127.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff9947ea94
Build id: 3.10.0-1127.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff996077f8
Build id: 3.10.0-1127.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff99601478
Build id: 3.10.0-1127.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff99602d26
Build id: 3.10.0-1127.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff9960308b
Build id: 3.10.0-1127.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff99601320
Build id: 3.10.0-1127.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff99600430
Build id: 3.10.0-1127.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff99601b80
Build id: 3.10.0-1127.el7.x86_64 		0x860
11
? page_get_anon_vma
vmlinux 0xffffffff99602520
Build id: 3.10.0-1127.el7.x86_64 		0x80
12
? invalid_mkclean_vma
vmlinux 0xffffffff99600400
Build id: 3.10.0-1127.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff9962fd6f
Build id: 3.10.0-1127.el7.x86_64 		0x35f
14
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff9962ca20
Build id: 3.10.0-1127.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffff99630b6c
Build id: 3.10.0-1127.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffff995f344d
Build id: 3.10.0-1127.el7.x86_64 		0x19d
17
handle_mm_fault
vmlinux 0xffffffff995f5dbb
Build id: 3.10.0-1127.el7.x86_64 		0xadb
18
__do_page_fault
vmlinux 0xffffffff99b8d653
Build id: 3.10.0-1127.el7.x86_64 		0x213
19
do_page_fault
vmlinux 0xffffffff99b8d975
Build id: 3.10.0-1127.el7.x86_64 		0x35
20
page_fault
vmlinux 0xffffffff99b89778
Build id: 3.10.0-1127.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [StreamLis1:2536]
Modules linked in: xfs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_umad(OE) mlx5_fpga_tools(OE) mlx4_en(OE) mlx4_ib(OE) mlx4_core(OE) sunrpc dm_mirror dm_region_hash dm_log dm_mod iTCO_wdt iTCO_vendor_support vfat fat skx_edac coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ses enclosure pcspkr scsi_transport_sas sg joydev mei_me i2c_i801 lpc_ich mei wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter
knem(OE) ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci i40e mlx5_core(OE) drm libahci megaraid_sas crct10dif_pclmul crct10dif_common crc32c_intel libata mlxfw(OE) devlink mlx_compat(OE) ptp pps_core drm_panel_orientation_quirks nfit libnvdimm
CPU: 3 PID: 2536 Comm: StreamLis1 Kdump: loaded Tainted: G           OE  ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: Inspur NF5466M5/YZMB-00882-104, BIOS 4.1.16 06/23/2020
task: ffff9cfa64cd5230 ti: ffff9cfa617c4000 task.ti: ffff9cfa617c4000
RIP: 0010:[<ffffffff9951760a>]  [<ffffffff9951760a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9cfa617c7a98  EFLAGS: 00000202
RAX: 000000000000000e RBX: 0000007000000003 RCX: ffff9cfa7f7a05c8
RDX: 000000000000000e RSI: 0000000000000070 RDI: 0000000000000000
RBP: ffff9cfa617c7ad0 R08: ffff9ceba93fa000 R09: ffffffff99785099
R10: ffff9cfa7f4df160 R11: ffffdefbc4ec7200 R12: 000000fc99784fef
R13: 0000000000000292 R14: 0000000000000292 R15: ffff9cfa617c7a48
FS:  00007ff343fff700(0000) GS:ffff9cfa7f4c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff2abed5018 CR3: 000000202805a000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff9947e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9947ea94>] flush_tlb_page+0x54/0xa0
[<ffffffff996077f8>] ptep_clear_flush+0x68/0xa0
[<ffffffff99601478>] try_to_unmap_one+0x158/0x860
[<ffffffff99602d26>] rmap_walk+0x266/0x360
[<ffffffff9960308b>] try_to_unmap+0x8b/0xe0
[<ffffffff99601320>] ? page_remove_rmap+0x160/0x160
[<ffffffff99600430>] ? invalid_migration_vma+0x30/0x30
[<ffffffff99601b80>] ? try_to_unmap_one+0x860/0x860
[<ffffffff99602520>] ? page_get_anon_vma+0x80/0x80
[<ffffffff99600400>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff9962fd6f>] migrate_pages+0x35f/0x7f0
[<ffffffff9962ca20>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff99630b6c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff995f344d>] do_numa_page+0x19d/0x250
[<ffffffff995f5dbb>] handle_mm_fault+0xadb/0xfb0
[<ffffffff99b8d653>] __do_page_fault+0x213/0x500
[<ffffffff99b8d975>] do_page_fault+0x35/0x90
[<ffffffff99b89778>] page_fault+0x28/0x30
Code: 48 63 35 1e 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 0f 15 9a f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 7a c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #370612
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8c9170de
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x20e
2
native_flush_tlb_others
vmlinux 0xffffffff8c87e258
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8c87e3f4
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff8ca08108
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff8ca01d78
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x158
6
? tcp_write_xmit
vmlinux 0xffffffff8cec0f83
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x353
7
rmap_walk
vmlinux 0xffffffff8ca03626
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x266
8
try_to_unmap
vmlinux 0xffffffff8ca0398b
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x8b
9
? page_remove_rmap
vmlinux 0xffffffff8ca01c20
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x160
10
? invalid_migration_vma
vmlinux 0xffffffff8ca00d30
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x30
11
? try_to_unmap_one
vmlinux 0xffffffff8ca02480
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x860
12
? page_get_anon_vma
vmlinux 0xffffffff8ca02e20
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x80
13
? invalid_mkclean_vma
vmlinux 0xffffffff8ca00d00
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x20
14
migrate_pages
vmlinux 0xffffffff8ca30807
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x447
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff8ca2d3d0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffff8ca3151c
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffff8c9f3ced
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x19d
18
handle_mm_fault
vmlinux 0xffffffff8c9f665b
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xadb
19
__do_page_fault
vmlinux 0xffffffff8cf90653
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x213
20
do_page_fault
vmlinux 0xffffffff8cf90975
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x35
21
page_fault
vmlinux 0xffffffff8cf8c778
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#16 stuck for 23s! [migration/16:90]
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [splunkd:2637]
Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc i2c_piix4 parport vmw_vmci ip_tables xfs libcrc32c ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sd_mod sysfillrect
sysimgblt fb_sys_fops crc_t10dif ttm crct10dif_generic ata_piix drm libata nfit crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 2637 Comm: splunkd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffffa0e7c7e16300 ti: ffffa0e7c7e28000 task.ti: ffffa0e7c7e28000
RIP: 0010:[<ffffffff8c9170de>]  [<ffffffff8c9170de>] smp_call_function_many+0x20e/0x270
RSP: 0000:ffffa0e7c7e2baa0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: 000800008c87ddd0 RCX: ffffa0e7dfb20700
RDX: 0000000000000004 RSI: 0000000000000018 RDI: 0000000000000000
RBP: ffffa0e7c7e2bad8 R08: ffffa0e12cb44c00 R09: ffffffff8cb86bf9
R10: ffffa0e7dfb5f160 R11: ffffde79800fae00 R12: 0000000000000282
R13: 0000000000000282 R14: ffffa0e7c7e2ba50 R15: ffffffff8c8a4077
FS:  00007f28d25ff700(0000) GS:ffffa0e7dfb40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f287cc36000 CR3: 0000000810af0000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8c87e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c87e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffff8ca08108>] ptep_clear_flush+0x68/0xa0
[<ffffffff8ca01d78>] try_to_unmap_one+0x158/0x860
[<ffffffff8cec0f83>] ? tcp_write_xmit+0x353/0xd40
[<ffffffff8ca03626>] rmap_walk+0x266/0x360
[<ffffffff8ca0398b>] try_to_unmap+0x8b/0xe0
[<ffffffff8ca01c20>] ? page_remove_rmap+0x160/0x160
[<ffffffff8ca00d30>] ? invalid_migration_vma+0x30/0x30
[<ffffffff8ca02480>] ? try_to_unmap_one+0x860/0x860
[<ffffffff8ca02e20>] ? page_get_anon_vma+0x80/0x80
[<ffffffff8ca00d00>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff8ca30807>] migrate_pages+0x447/0x7f0
[<ffffffff8ca2d3d0>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff8ca3151c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff8c9f3ced>] do_numa_page+0x19d/0x250
[<ffffffff8c9f665b>] handle_mm_fault+0xadb/0xfb0
[<ffffffff8cf90653>] __do_page_fault+0x213/0x500
[<ffffffff8cf90975>] do_page_fault+0x35/0x90
[<ffffffff8cf8c778>] page_fault+0x28/0x30
Code: 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 55 8d f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #342154
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8a5170da
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8a47e258
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8a47e3f4
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff8a608108
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff8a601d78
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x158
6
? rmap_walk
vmlinux 0xffffffff8a603679
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x2b9
7
rmap_walk
vmlinux 0xffffffff8a603626
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x266
8
try_to_unmap
vmlinux 0xffffffff8a60398b
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x8b
9
? page_remove_rmap
vmlinux 0xffffffff8a601c20
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x160
10
? invalid_migration_vma
vmlinux 0xffffffff8a600d30
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x30
11
? try_to_unmap_one
vmlinux 0xffffffff8a602480
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x860
12
? page_get_anon_vma
vmlinux 0xffffffff8a602e20
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x80
13
? invalid_mkclean_vma
vmlinux 0xffffffff8a600d00
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x20
14
migrate_pages
vmlinux 0xffffffff8a630807
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x447
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff8a62d3d0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffff8a63151c
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffff8a5f3ced
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x19d
18
handle_mm_fault
vmlinux 0xffffffff8a5f665b
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xadb
19
__do_page_fault
vmlinux 0xffffffff8ab90653
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x213
20
do_page_fault
vmlinux 0xffffffff8ab90975
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x35
21
? error_swapgs
vmlinux 0xffffffff8ab8cac9
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xaa
22
page_fault
vmlinux 0xffffffff8ab8c778
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 23s! [C2 CompilerThre:9259]
Modules linked in: nf_log_ipv4 nf_log_common xt_LOG xt_comment xt_limit xt_multiport ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic
pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm ata_piix crct10dif_pclmul crct10dif_common mptspi crc32c_intel libnvdimm scsi_transport_spi libata mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 11 PID: 9259 Comm: C2 CompilerThre Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff90b4d95e9080 ti: ffff90b4ab134000 task.ti: ffff90b4ab134000
RIP: 0010:[<ffffffff8a5170da>]  [<ffffffff8a5170da>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff90b4ab137aa0  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800008a47ddd0 RCX: ffff90b4e0a207f0
RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff90b4ab137ad8 R08: ffff90ae6d3afc00 R09: ffffffff8a786bf9
R10: ffff90b4e0cdf160 R11: fffff97e006ea600 R12: 0000000000000282
R13: 0000000000000282 R14: ffff90b4ab137a50 R15: ffff90ae6cf96c00
FS:  00007f3fd1b04700(0000) GS:ffff90b4e0cc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3fd1b02f08 CR3: 000000101a4a2000 CR4: 00000000000607e0
Call Trace:
[<ffffffff8a47e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8a47e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffff8a608108>] ptep_clear_flush+0x68/0xa0
[<ffffffff8a601d78>] try_to_unmap_one+0x158/0x860
[<ffffffff8a603679>] ? rmap_walk+0x2b9/0x360
[<ffffffff8a603626>] rmap_walk+0x266/0x360
[<ffffffff8a60398b>] try_to_unmap+0x8b/0xe0
[<ffffffff8a601c20>] ? page_remove_rmap+0x160/0x160
[<ffffffff8a600d30>] ? invalid_migration_vma+0x30/0x30
[<ffffffff8a602480>] ? try_to_unmap_one+0x860/0x860
[<ffffffff8a602e20>] ? page_get_anon_vma+0x80/0x80
[<ffffffff8a600d00>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff8a630807>] migrate_pages+0x447/0x7f0
[<ffffffff8a62d3d0>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff8a63151c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff8a5f3ced>] do_numa_page+0x19d/0x250
[<ffffffff8a5f665b>] handle_mm_fault+0xadb/0xfb0
[<ffffffff8ab90653>] __do_page_fault+0x213/0x500
[<ffffffff8ab90975>] do_page_fault+0x35/0x90
[<ffffffff8ab8cac9>] ? error_swapgs+0xaa/0xc0
[<ffffffff8ab8c778>] page_fault+0x28/0x30
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 15 8b f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #211548
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb6111a8a
Build id: 3.10.0-957.21.2.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffb607a6a8
Build id: 3.10.0-957.21.2.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffb607a844
Build id: 3.10.0-957.21.2.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffb61fdb97
Build id: 3.10.0-957.21.2.el7.x86_64 		0x67
5
try_to_unmap_one
vmlinux 0xffffffffb61f7848
Build id: 3.10.0-957.21.2.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffffb61f9106
Build id: 3.10.0-957.21.2.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffffb61f946b
Build id: 3.10.0-957.21.2.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffffb61f76f0
Build id: 3.10.0-957.21.2.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffffb61f6860
Build id: 3.10.0-957.21.2.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffffb61f7f40
Build id: 3.10.0-957.21.2.el7.x86_64 		0x850
11
? page_get_anon_vma
vmlinux 0xffffffffb61f8d90
Build id: 3.10.0-957.21.2.el7.x86_64 		0xa0
12
? invalid_mkclean_vma
vmlinux 0xffffffffb61f6830
Build id: 3.10.0-957.21.2.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffffb6224d2f
Build id: 3.10.0-957.21.2.el7.x86_64 		0x43f
14
? migrate_vma_collect.constprop.53
vmlinux 0xffffffffb62219f0
Build id: 3.10.0-957.21.2.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffffb62259dc
Build id: 3.10.0-957.21.2.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffffb61e939d
Build id: 3.10.0-957.21.2.el7.x86_64 		0x19d
17
handle_pte_fault
vmlinux 0xffffffffb61e9766
Build id: 3.10.0-957.21.2.el7.x86_64 		0x316
18
handle_mm_fault
vmlinux 0xffffffffb61ec27d
Build id: 3.10.0-957.21.2.el7.x86_64 		0x39d
19
__do_page_fault
vmlinux 0xffffffffb6770603
Build id: 3.10.0-957.21.2.el7.x86_64 		0x203
20
do_page_fault
vmlinux 0xffffffffb6770925
Build id: 3.10.0-957.21.2.el7.x86_64 		0x35
21
? error_swapgs
vmlinux 0xffffffffb676cab6
Build id: 3.10.0-957.21.2.el7.x86_64 		0xa7
22
page_fault
vmlinux 0xffffffffb676c768
Build id: 3.10.0-957.21.2.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#31 stuck for 22s! [R:22321]
Modules linked in: binfmt_misc iptable_filter ext4 mbcache jbd2 iscsi_tcp libiscsi_tcp rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm iTCO_wdt gpio_ich iTCO_vendor_support intel_powerclamp coretemp kvm_intel kvm irqbypass crc32_pclmul vfat fat ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif ocrdma(T) cdc_ether pcspkr usbnet ib_core mii lpc_ich ioatdma sg i2c_i801 i7core_edac dca ipmi_si ipmi_devintf wmi ipmi_msghandler nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi mgag200 i2c_algo_bit drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common serio_raw crc32c_intel drm ata_piix libata drm_panel_orientation_quirks be2net megaraid_sas bnx2 dm_mirror dm_region_hash dm_log dm_mod
CPU: 31 PID: 22321 Comm: R Kdump: loaded Tainted: G          I  L ------------ T 3.10.0-957.21.2.el7.x86_64 #1
Hardware name: IBM System x3850 X5 -[7143B7U]-/Node 1, Processor Card, BIOS -[G0E188BUS-1.88]- 05/29/2018
task: ffff953ef6ba0000 ti: ffff957ee5790000 task.ti: ffff957ee5790000
RIP: 0010:[<ffffffffb6111a8a>]  [<ffffffffb6111a8a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff957ee5793a28  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff953eff860b98
RDX: 0000000000000001 RSI: 0000000000000050 RDI: 0000000000000000
RBP: ffff957ee5793a60 R08: ffff957fd3b68c00 R09: ffffffffb6375e39
R10: ffff959eff05f160 R11: ffffe125ffcad000 R12: 0000000000000292
R13: 0000000000000000 R14: ffff957ee5793998 R15: 0000000000000292
FS:  00007f078e8c97c0(0000) GS:ffff959eff040000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000013701b8 CR3: 0000001f8a3a8000 CR4: 00000000000207e0
Call Trace:
[<ffffffffb607a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb607a844>] flush_tlb_page+0x54/0xa0
[<ffffffffb61fdb97>] ptep_clear_flush+0x67/0xa0
[<ffffffffb61f7848>] try_to_unmap_one+0x158/0x850
[<ffffffffb61f9106>] rmap_walk+0x266/0x360
[<ffffffffb61f946b>] try_to_unmap+0x8b/0xe0
[<ffffffffb61f76f0>] ? page_remove_rmap+0x160/0x160
[<ffffffffb61f6860>] ? invalid_migration_vma+0x30/0x30
[<ffffffffb61f7f40>] ? try_to_unmap_one+0x850/0x850
[<ffffffffb61f8d90>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffffb61f6830>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffffb6224d2f>] migrate_pages+0x43f/0x790
[<ffffffffb62219f0>] ? migrate_vma_collect.constprop.53+0xe0/0xe0
[<ffffffffb62259dc>] migrate_misplaced_page+0xcc/0x100
[<ffffffffb61e939d>] do_numa_page+0x19d/0x250
[<ffffffffb61e9766>] handle_pte_fault+0x316/0xd10
[<ffffffffb61ec27d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb6770603>] __do_page_fault+0x203/0x4f0
[<ffffffffb6770925>] do_page_fault+0x35/0x90
[<ffffffffb676cab6>] ? error_swapgs+0xa7/0xbd
[<ffffffffb676c768>] page_fault+0x28/0x30
Code: 48 63 35 fe 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 bc d4 b6 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 cd 82 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #174948
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9371760a
Build id: 3.10.0-1127.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9367e8f8
Build id: 3.10.0-1127.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff9367ea94
Build id: 3.10.0-1127.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff938077f8
Build id: 3.10.0-1127.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff93801478
Build id: 3.10.0-1127.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff93802d26
Build id: 3.10.0-1127.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff9380308b
Build id: 3.10.0-1127.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff93801320
Build id: 3.10.0-1127.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff93800430
Build id: 3.10.0-1127.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff93801b80
Build id: 3.10.0-1127.el7.x86_64 		0x860
11
? page_get_anon_vma
vmlinux 0xffffffff93802520
Build id: 3.10.0-1127.el7.x86_64 		0x80
12
? invalid_mkclean_vma
vmlinux 0xffffffff93800400
Build id: 3.10.0-1127.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff9382fd6f
Build id: 3.10.0-1127.el7.x86_64 		0x35f
14
? generic_file_aio_read
vmlinux 0xffffffff937c0371
Build id: 3.10.0-1127.el7.x86_64 		0x601
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff9382ca20
Build id: 3.10.0-1127.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffff93830b6c
Build id: 3.10.0-1127.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffff937f344d
Build id: 3.10.0-1127.el7.x86_64 		0x19d
18
handle_mm_fault
vmlinux 0xffffffff937f5dbb
Build id: 3.10.0-1127.el7.x86_64 		0xadb
19
? do_sync_read
vmlinux 0xffffffff9384c583
Build id: 3.10.0-1127.el7.x86_64 		0x93
20
__do_page_fault
vmlinux 0xffffffff93d8d653
Build id: 3.10.0-1127.el7.x86_64 		0x213
21
do_page_fault
vmlinux 0xffffffff93d8d975
Build id: 3.10.0-1127.el7.x86_64 		0x35
22
page_fault
vmlinux 0xffffffff93d89778
Build id: 3.10.0-1127.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#16 stuck for 22s! [Worker6-2:2673]
Modules linked in: xfs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_umad(OE) mlx5_fpga_tools(OE) mlx4_en(OE) mlx4_ib(OE) mlx4_core(OE) sunrpc dm_mirror dm_region_hash dm_log dm_mod iTCO_wdt iTCO_vendor_support vfat fat skx_edac coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ses enclosure scsi_transport_sas sg joydev mei_me mei i2c_i801 lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter
knem(OE) ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mlx5_core(OE) drm ahci i40e libahci crct10dif_pclmul crct10dif_common mlxfw(OE) megaraid_sas devlink crc32c_intel libata vfio_mdev(OE) vfio_iommu_type1 vfio mdev(OE) mlx_compat(OE) ptp pps_core drm_panel_orientation_quirks nfit libnvdimm
CPU: 16 PID: 2673 Comm: Worker6-2 Kdump: loaded Tainted: G           OE  ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: Inspur NF5466M5/YZMB-00882-104, BIOS 4.1.16 06/23/2020
task: ffff89919de162a0 ti: ffff89919de54000 task.ti: ffff89919de54000
RIP: 0010:[<ffffffff9371760a>]  [<ffffffff9371760a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff89919de57a98  EFLAGS: 00000202
RAX: 000000000000000e RBX: 0000007000000010 RCX: ffff8991bf7a08e0
RDX: 000000000000000e RSI: 0000000000000070 RDI: 0000000000000000
RBP: ffff89919de57ad0 R08: ffff899269200000 R09: ffffffff93985099
R10: ffff89a1be81f160 R11: fffffce404202a00 R12: 000000fc93984fef
R13: 0000000000000292 R14: 0000000000000292 R15: ffff89919de57a48
FS:  00007f5f997f4700(0000) GS:ffff89a1be800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5ff4068ff0 CR3: 0000002030b2c000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff9367e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9367ea94>] flush_tlb_page+0x54/0xa0
[<ffffffff938077f8>] ptep_clear_flush+0x68/0xa0
[<ffffffff93801478>] try_to_unmap_one+0x158/0x860
[<ffffffff93802d26>] rmap_walk+0x266/0x360
[<ffffffff9380308b>] try_to_unmap+0x8b/0xe0
[<ffffffff93801320>] ? page_remove_rmap+0x160/0x160
[<ffffffff93800430>] ? invalid_migration_vma+0x30/0x30
[<ffffffff93801b80>] ? try_to_unmap_one+0x860/0x860
[<ffffffff93802520>] ? page_get_anon_vma+0x80/0x80
[<ffffffff93800400>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff9382fd6f>] migrate_pages+0x35f/0x7f0
[<ffffffff937c0371>] ? generic_file_aio_read+0x601/0x790
[<ffffffff9382ca20>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff93830b6c>] migrate_misplaced_page+0xcc/0x100
[<ffffffff937f344d>] do_numa_page+0x19d/0x250
[<ffffffff937f5dbb>] handle_mm_fault+0xadb/0xfb0
[<ffffffff9384c583>] ? do_sync_read+0x93/0xe0
[<ffffffff93d8d653>] __do_page_fault+0x213/0x500
[<ffffffff93d8d975>] do_page_fault+0x35/0x90
[<ffffffff93d89778>] page_fault+0x28/0x30
Code: 48 63 35 1e 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 0f 35 94 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 7a c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #24588
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffb4d175ca
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffb4c7e8f8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffb4c7ea94
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffb4e07858
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffffb4e014d8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffffb4e02d86
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffffb4e030eb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffffb4e01380
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffffb4e00490
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffffb4e01be0
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x860
11
? page_get_anon_vma
vmlinux 0xffffffffb4e02580
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x80
12
? invalid_mkclean_vma
vmlinux 0xffffffffb4e00460
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffffb4e2fdcf
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35f
14
? __switch_to
vmlinux 0xffffffffb4c2b621
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x151
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffffb4e2ca80
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffffb4e30bcc
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffffb4df34ad
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x19d
18
handle_mm_fault
vmlinux 0xffffffffb4df5e1b
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xadb
19
? schedule_hrtimeout_range_clock
vmlinux 0xffffffffb5384fa6
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xa6
20
__do_page_fault
vmlinux 0xffffffffb538d653
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x213
21
do_page_fault
vmlinux 0xffffffffb538d975
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35
22
? error_swapgs
vmlinux 0xffffffffb5389ac9
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xaa
23
page_fault
vmlinux 0xffffffffb5389778
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#27 stuck for 22s! [dbaWorker:25105]
Modules linked in: tcp_diag inet_diag sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul mptspi crct10dif_common libata scsi_transport_spi crc32c_intel e1000 mptscsih serio_raw mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 27 PID: 25105 Comm: dbaWorker Kdump: loaded Tainted: G             L ------------   3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff8f0f7bfa62a0 ti: ffff8f0f72e20000 task.ti: ffff8f0f72e20000
RIP: 0010:[<ffffffffb4d175ca>]  [<ffffffffb4d175ca>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8f0f72e23a98  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000000000001b840 RCX: ffff8f0befa20a70
RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff8f0f72e23ad0 R08: ffff8f0c7a66b400 R09: ffffffffb4f85179
R10: ffff8f0fff8df160 R11: fffff17d9c57f800 R12: ffff8f0f72e23a48
R13: ffff8f0f72e23b68 R14: ffff8f0fff6560a0 R15: ffff8f0f72e20000
FS:  00007f5f538eb700(0000) GS:ffff8f0fff8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5f5a9dfa40 CR3: 00000007bbfde000 CR4: 00000000000407e0
Call Trace:
[<ffffffffb4c7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb4c7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffb4e07858>] ptep_clear_flush+0x68/0xa0
[<ffffffffb4e014d8>] try_to_unmap_one+0x158/0x860
[<ffffffffb4e02d86>] rmap_walk+0x266/0x360
[<ffffffffb4e030eb>] try_to_unmap+0x8b/0xe0
[<ffffffffb4e01380>] ? page_remove_rmap+0x160/0x160
[<ffffffffb4e00490>] ? invalid_migration_vma+0x30/0x30
[<ffffffffb4e01be0>] ? try_to_unmap_one+0x860/0x860
[<ffffffffb4e02580>] ? page_get_anon_vma+0x80/0x80
[<ffffffffb4e00460>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffffb4e2fdcf>] migrate_pages+0x35f/0x7f0
[<ffffffffb4c2b621>] ? __switch_to+0x151/0x580
[<ffffffffb4e2ca80>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffffb4e30bcc>] migrate_misplaced_page+0xcc/0x100
[<ffffffffb4df34ad>] do_numa_page+0x19d/0x250
[<ffffffffb4df5e1b>] handle_mm_fault+0xadb/0xfb0
[<ffffffffb5384fa6>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[<ffffffffb538d653>] __do_page_fault+0x213/0x500
[<ffffffffb538d975>] do_page_fault+0x35/0x90
[<ffffffffb5389ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffb5389778>] page_fault+0x28/0x30
Code: 48 63 35 de 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 10 95 b5 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ad 7b c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #23581
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9f31761a
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9f27e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff9f27ea94
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff9f407888
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff9f401508
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x158
6
rmap_walk
vmlinux 0xffffffff9f402db6
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x266
7
try_to_unmap
vmlinux 0xffffffff9f40311b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x8b
8
? page_remove_rmap
vmlinux 0xffffffff9f4013b0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x160
9
? invalid_migration_vma
vmlinux 0xffffffff9f4004c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x30
10
? try_to_unmap_one
vmlinux 0xffffffff9f401c10
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x860
11
? page_get_anon_vma
vmlinux 0xffffffff9f4025b0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x80
12
? invalid_mkclean_vma
vmlinux 0xffffffff9f400490
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x20
13
migrate_pages
vmlinux 0xffffffff9f42fdff
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x35f
14
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff9f42cab0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xe0
15
migrate_misplaced_page
vmlinux 0xffffffff9f430bfc
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xcc
16
do_numa_page
vmlinux 0xffffffff9f3f34cd
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x19d
17
handle_mm_fault
vmlinux 0xffffffff9f3f5e3b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xadb
18
__do_page_fault
vmlinux 0xffffffff9f98d653
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x213
19
trace_do_page_fault
vmlinux 0xffffffff9f98da26
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x56
20
do_async_page_fault
vmlinux 0xffffffff9f98cfa2
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x22
21
async_page_fault
vmlinux 0xffffffff9f9897a8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [dcache:14060]
Modules linked in: xt_nat veth binfmt_misc xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) iptable_filter sunrpc ext4 mbcache jbd2 sb_edac iosf_mbi ppdev kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg virtio_balloon i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_console virtio_scsi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci ixgbevf virtio_ring
serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 14060 Comm: dcache Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff9a4c1db6a0e0 ti: ffff9a49a85a8000 task.ti: ffff9a49a85a8000
RIP: 0010:[<ffffffff9f31761a>]  [<ffffffff9f31761a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9a49a85aba68  EFLAGS: 00000202
RAX: 000000000000000a RBX: 00000000000000fc RCX: ffff9a482fca0558
RDX: 000000000000000a RSI: 0000000000000018 RDI: 0000000000000000
RBP: ffff9a49a85abaa0 R08: ffff9a457a2d6800 R09: ffffffff9f5851b9
R10: ffff9a482fa5f160 R11: ffffcbe790b5c200 R12: 0000000000000296
R13: ffff9a49a85aba18 R14: 0000000000000246 R15: ffffffff9f4281a9
FS:  00007f11ce7fc700(0000) GS:ffff9a482fa40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f11b002b800 CR3: 000000042a58c000 CR4: 0000000000160fe0
Call Trace:
[<ffffffff9f27e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9f27ea94>] flush_tlb_page+0x54/0xa0
[<ffffffff9f407888>] ptep_clear_flush+0x68/0xa0
[<ffffffff9f401508>] try_to_unmap_one+0x158/0x860
[<ffffffff9f402db6>] rmap_walk+0x266/0x360
[<ffffffff9f40311b>] try_to_unmap+0x8b/0xe0
[<ffffffff9f4013b0>] ? page_remove_rmap+0x160/0x160
[<ffffffff9f4004c0>] ? invalid_migration_vma+0x30/0x30
[<ffffffff9f401c10>] ? try_to_unmap_one+0x860/0x860
[<ffffffff9f4025b0>] ? page_get_anon_vma+0x80/0x80
[<ffffffff9f400490>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff9f42fdff>] migrate_pages+0x35f/0x7f0
[<ffffffff9f42cab0>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff9f430bfc>] migrate_misplaced_page+0xcc/0x100
[<ffffffff9f3f34cd>] do_numa_page+0x19d/0x250
[<ffffffff9f3f5e3b>] handle_mm_fault+0xadb/0xfb0
[<ffffffff9f98d653>] __do_page_fault+0x213/0x500
[<ffffffff9f98da26>] trace_do_page_fault+0x56/0x150
[<ffffffff9f98cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9f9897a8>] async_page_fault+0x28/0x30
Code: 48 63 35 ce 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 10 f5 9f f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 7b c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #15650
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff83916a7a
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8387e478
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8387e614
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff83a05a48
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x68
5
try_to_unmap_one
vmlinux 0xffffffff839ff6a8
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x158
6
? rmap_walk
vmlinux 0xffffffff83a00fc9
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x2b9
7
rmap_walk
vmlinux 0xffffffff83a00f76
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x266
8
try_to_unmap
vmlinux 0xffffffff83a012db
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x8b
9
? page_remove_rmap
vmlinux 0xffffffff839ff550
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x160
10
? invalid_migration_vma
vmlinux 0xffffffff839fe660
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x30
11
? try_to_unmap_one
vmlinux 0xffffffff839ffdb0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x860
12
? page_get_anon_vma
vmlinux 0xffffffff83a00c00
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xa0
13
? invalid_mkclean_vma
vmlinux 0xffffffff839fe630
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x20
14
migrate_pages
vmlinux 0xffffffff83a2db1f
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x45f
15
? migrate_vma_collect.constprop.52
vmlinux 0xffffffff83a2a690
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xe0
16
migrate_misplaced_page
vmlinux 0xffffffff83a2e7dc
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xcc
17
do_numa_page
vmlinux 0xffffffff839f0f4d
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x19d
18
handle_pte_fault
vmlinux 0xffffffff839f12b9
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x2b9
19
handle_mm_fault
vmlinux 0xffffffff839f3f3d
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x39d
20
__do_page_fault
vmlinux 0xffffffff83f88653
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x213
21
do_page_fault
vmlinux 0xffffffff83f88975
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x35
22
page_fault
vmlinux 0xffffffff83f84778
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [calico-node:12463]
Modules linked in: cfg80211 rfkill xt_set iptable_mangle iptable_raw ip_set_hash_ip ip_set_hash_net veth nf_conntrack_netlink xt_statistic xt_nat xt_recent ipt_REJECT nf_reject_ipv4 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 ipip tunnel4 ip_tunnel xt_multiport xt_addrtype xt_mark ip_set nfnetlink ip6_tables
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect crc_t10dif sysimgblt crct10dif_generic fb_sys_fops ttm ahci drm ata_piix libahci crct10dif_pclmul crct10dif_common crc32c_intel libata nfit serio_raw libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 14 PID: 12463 Comm: calico-node Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
task: ffff9c290530a0e0 ti: ffff9c2950af8000 task.ti: ffff9c2950af8000
RIP: 0010:[<ffffffff83916a7a>]  [<ffffffff83916a7a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9c2950afba18  EFLAGS: 00000202
RAX: 0000000000000009 RBX: 00000000000000fc RCX: ffff9c29bfc60868
RDX: 0000000000000009 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9c2950afba50 R08: ffff9c2234ea3c00 R09: ffffffff83b81839
R10: ffff9c29bfd9f160 R11: fffffb11fb8b9000 R12: 0000000000000296
R13: ffff9c2950afb9c8 R14: ffff9c2950afb988 R15: 0000000000000296
FS:  000000c42087a490(0000) GS:ffff9c29bfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4201d1ab8 CR3: 00000007ff64a000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8387e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8387e614>] flush_tlb_page+0x54/0xa0
[<ffffffff83a05a48>] ptep_clear_flush+0x68/0xa0
[<ffffffff839ff6a8>] try_to_unmap_one+0x158/0x860
[<ffffffff83a00fc9>] ? rmap_walk+0x2b9/0x360
[<ffffffff83a00f76>] rmap_walk+0x266/0x360
[<ffffffff83a012db>] try_to_unmap+0x8b/0xe0
[<ffffffff839ff550>] ? page_remove_rmap+0x160/0x160
[<ffffffff839fe660>] ? invalid_migration_vma+0x30/0x30
[<ffffffff839ffdb0>] ? try_to_unmap_one+0x860/0x860
[<ffffffff83a00c00>] ? page_get_anon_vma+0xa0/0xa0
[<ffffffff839fe630>] ? invalid_mkclean_vma+0x20/0x20
[<ffffffff83a2db1f>] migrate_pages+0x45f/0x7b0
[<ffffffff83a2a690>] ? migrate_vma_collect.constprop.52+0xe0/0xe0
[<ffffffff83a2e7dc>] migrate_misplaced_page+0xcc/0x100
[<ffffffff839f0f4d>] do_numa_page+0x19d/0x250
[<ffffffff839f12b9>] handle_pte_fault+0x2b9/0xe20
[<ffffffff839f3f3d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff83f88653>] __do_page_fault+0x213/0x500
[<ffffffff83f88975>] do_page_fault+0x35/0x90
[<ffffffff83f84778>] page_fault+0x28/0x30
Code: 48 63 35 ee 5e c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 e8 54 84 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 bd 5e c4 00 eb b7 0f b6 4d cc 4c 89