Complete report #1403198
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f99a2
Build id: 3.10.0-693.el7.x86_64
|
0x202
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811b08fa
Build id: 3.10.0-693.el7.x86_64
|
0x13a
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff811ac71d
Build id: 3.10.0-693.el7.x86_64
|
0x3cd
|
|
8
|
|
vmlinux |
0xffffffff81200bf5
Build id: 3.10.0-693.el7.x86_64
|
0xf5
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [CacheThread_Blo:98784]
Modules linked in: qax_tq_base(OE) isofs fuse vfat fat ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg parport_pc virtio_balloon parport joydev pcspkr i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_net virtio_console virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix serio_raw libata virtio_pci virtio_ring i2c_core virtio floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: qax_tq_base]
CPU: 1 PID: 98784 Comm: CacheThread_Blo Tainted: G OEL ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff880b36725ee0 ti: ffff8807fdeb8000 task.ti: ffff8807fdeb8000
RIP: 0010:[<ffffffff810f99a2>] [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff8807fdebbcb0 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000800008106e4a0 RCX: ffff880ffce9a6c8
RDX: 0000000000000002 RSI: 00000000000000a0 RDI: 0000000000000000
RBP: ffff8807fdebbce8 R08: ffff8801694c3400 R09: ffffffff813227d9
R10: ffff880ffce59c80 R11: ffffea0021259400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8807fdebbc60 R15: 0000000000000001
FS: 00007f7eb1af6700(0000) GS:ffff880ffce40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00001b09fb013000 CR3: 0000000c3d3e2000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00000001fb013fff ffff880fe3b5bb88 ffff880fe3b5b840 0000000000000000
ffffffffffffffff ffff880fe3b5bb88 ffff880fe3b5b840 ffff8807fdebbd38
ffffffff8106e668 ffff880fe3b5b840 0000000000000000 ffffffffffffffff
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b08fa>] zap_page_range+0x13a/0x180
[<ffffffff811ac71d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff81200bf5>] ? vfs_read+0xf5/0x170
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89
Complete report #1379881
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810fb892
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x202
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106f508
Build id: 3.10.0-693.17.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106f5d8
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811af0f3
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811b0415
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811b299a
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x13a
|
|
7
|
|
vmlinux |
0xffffffff811e0b02
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x1e2
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff811ae7bd
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x3cd
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff816b89fd
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [mysqld:16938]
Modules linked in: tcp_lp ip6table_filter ip6_tables fuse ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter vmw_vsock_vmci_transport vsock vfat fat crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon nfit libnvdimm sg parport_pc i2c_piix4 tpm_crb parport joydev shpchp vmw_vmci pcspkr nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi ata_piix libata e1000 mptscsih crc32c_intel mptbase i2c_core serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 16938 Comm: mysqld Tainted: G W L ------------ 3.10.0-693.17.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware20,1/440BX Desktop Reference Platform, BIOS VMW201.00V.24504846.B64.2501180339 01/18/2025
task: ffff8800626e8fd0 ti: ffff880013280000 task.ti: ffff880013280000
RIP: 0010:[<ffffffff810fb892>] [<ffffffff810fb892>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff880013283c80 EFLAGS: 00000202
RAX: 0000000000000003 RBX: 000800008106f340 RCX: ffff88063fcdc5b8
RDX: 0000000000000003 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff880013283cb8 R08: ffff880176bb7400 R09: ffffffff81324cb9
R10: ffff88063fc5b9c0 R11: ffffea0015641c00 R12: 0000000000000286
R13: 0000000000000286 R14: ffff880013283c30 R15: 0000000000000001
FS: 00007f504d35c700(0000) GS:ffff88063fc40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f458d4b0068 CR3: 0000000613eae000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8106f508>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106f5d8>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811af0f3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b0415>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b299a>] zap_page_range+0x13a/0x180
[<ffffffff811e0b02>] ? kmem_cache_free+0x1e2/0x200
[<ffffffff811ae7bd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816b89fd>] system_call_fastpath+0x16/0x1b
Code: 48 63 35 76 b6 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 8e b1 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 45 b6 a2 00 eb b7 0f b6 4d cc 4c 89
Complete report #1158121
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff8b31173a
Build id: 3.10.0-957.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8b27a598
Build id: 3.10.0-957.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8b27a608
Build id: 3.10.0-957.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff8b3e4077
Build id: 3.10.0-957.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8b3e5975
Build id: 3.10.0-957.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff8b3e87c2
Build id: 3.10.0-957.el7.x86_64
|
0x142
|
|
7
|
|
vmlinux |
0xffffffff8b4404d3
Build id: 3.10.0-957.el7.x86_64
|
0x93
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff8b3e368d
Build id: 3.10.0-957.el7.x86_64
|
0x49d
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff8b974ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [AVClauncher:10995]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev vmw_balloon crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common
crc32c_intel serio_raw vmwgfx e1000 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi drm_panel_orientation_quirks scsi_transport_spi mptscsih mptbase nfit libnvdimm
CPU: 3 PID: 10995 Comm: AVClauncher Kdump: loaded Tainted: G L ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9cf656ea0000 ti: ffff9cf623b44000 task.ti: ffff9cf623b44000
RIP: 0010:[<ffffffff8b31173a>] [<ffffffff8b31173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9cf623b47c78 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff9cf675620600
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9cf623b47cb0 R08: ffff9cf5bc85bc00 R09: ffffffff8b5755c9
R10: ffff9cf6756df160 R11: ffffe253085ad200 R12: 0000000000000296
R13: ffff9cf623b47c28 R14: 0000000000000001 R15: ffffe25307225580
FS: 00007f406ad57700(0000) GS:ffff9cf6756c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f403ee0b0d8 CR3: 00000000af032000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff8b27a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8b27a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8b3e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff8b3e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff8b3e87c2>] zap_page_range+0x142/0x190
[<ffffffff8b4404d3>] ? do_sync_read+0x93/0xe0
[<ffffffff8b3e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff8b974ddb>] system_call_fastpath+0x22/0x27
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 f4 8b f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1084503
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f99b2
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x202
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106e668
Build id: 3.10.0-693.11.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106e738
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811acfc3
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ae2e5
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811b086a
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x13a
|
|
7
|
|
vmlinux |
0xffffffff811b2048
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x858
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff811ac68d
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x3cd
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff816b5089
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#22 stuck for 23s! [scheduler.rb:73:15649]
Modules linked in: tcp_diag inet_diag vmw_vsock_vmci_transport vsock sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper nfit ppdev vmw_balloon cryptd pcspkr sg vmw_vmci joydev i2c_piix4 shpchp parport_pc libnvdimm parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi crct10dif_pclmul crct10dif_common libata crc32c_intel mptscsih e1000 serio_raw mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 22 PID: 15649 Comm: scheduler.rb:73 Tainted: G L ------------ 3.10.0-693.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff880164e8dee0 ti: ffff8800025ac000 task.ti: ffff8800025ac000
RIP: 0010:[<ffffffff810f99b2>] [<ffffffff810f99b2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff8800025afcb0 EFLAGS: 00000202
RAX: 000000000000000b RBX: ffffffff8106e4a0 RCX: ffff88081fcdb2d8
RDX: 000000000000000b RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff8800025afce8 R08: ffff8808b4dedc00 R09: ffffffff81322729
R10: ffff88101f399c80 R11: ffffea00007f5c00 R12: 0000000000000286
R13: ffff8800025afc60 R14: ffff88017fc03500 R15: 0000000000000001
FS: 00007fb488fff700(0000) GS:ffff88101f380000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb501d5de40 CR3: 000000012985a000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000101d5ffff ffff880121a92288 ffff880121a91f40 0000000000000000
ffffffffffffffff ffff880121a92288 ffff880121a91f40 ffff8800025afd38
ffffffff8106e668 ffff880121a91f40 0000000000000000 ffffffffffffffff
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811acfc3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae2e5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b086a>] zap_page_range+0x13a/0x180
[<ffffffff811b2048>] ? handle_mm_fault+0x858/0xfa0
[<ffffffff811ac68d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
Code: 48 63 35 16 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 e5 44 a2 00 eb b7 0f b6 4d cc 4c 89
Complete report #991079
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f98e2
Build id: 3.10.0-514.el7.x86_64
|
0x202
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106c4c8
Build id: 3.10.0-514.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106c599
Build id: 3.10.0-514.el7.x86_64
|
0x69
|
|
4
|
tlb_flush_mmu.part.61
|
vmlinux |
0xffffffff811ac093
Build id: 3.10.0-514.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ad385
Build id: 3.10.0-514.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811af75a
Build id: 3.10.0-514.el7.x86_64
|
0x13a
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 54s! [containerd-shim:5491]
Modules linked in: xt_nat veth fuse nf_conntrack_netlink nfnetlink xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi aesni_intel lrw snd_ac97_codec gf128mul glue_helper ablk_helper ac97_bus cryptd snd_seq ppdev snd_seq_device vmw_balloon snd_pcm snd_timer snd pcspkr soundcore nfit sg vmw_vmci i2c_piix4 shpchp libnvdimm parport_pc parport br_netfilter nfsd bridge auth_rpcgss nfs_acl stp lockd llc grace overlay(T) sunrpc ip_tables xfs libcrc32c
sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper mptspi syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ata_piix crct10dif_common ttm scsi_transport_spi crc32c_intel serio_raw drm e1000 i2c_core libata mptscsih mptbase fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 5491 Comm: containerd-shim Tainted: G ------------ T 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8800b41b6dd0 ti: ffff88006c55c000 task.ti: ffff88006c55c000
RIP: 0010:[<ffffffff810f98e2>] [<ffffffff810f98e2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff88006c55fcc8 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00000000000000fc RCX: ffff880139699fa0
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff88006c55fd00 R08: ffff880139075400 R09: ffffffff81317a89
R10: ffff880139619b40 R11: ffffea000223de00 R12: 0000000000000292
R13: ffff88006c55fc78 R14: 0000000000000200 R15: ffffffff81f064c0
FS: 000000c000210890(0000) GS:ffff880139600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0000e5010 CR3: 0000000097933000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00000001000de000 ffff880035dba8c8 ffff880035dba580 0000000000000000
ffffffffffffffff ffff880035dba8c8 ffff880035dba580 ffff88006c55fd50
ffffffff8106c4c8 ffff880035dba580 0000000000000000 ffffffffffffffff
Call Trace:
[<ffffffff8106c4c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c599>] flush_tlb_mm_range+0x69/0x140
[<ffffffff811ac093>] tlb_flush_mmu.part.61+0x33/0xc0
[<ffffffff811ad385>] tlb_finish_mmu+0x55/0x60
[<ffffffff811af75a>] zap_page_range+0x13a/0x180
Complete report #822072
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffac51173a
Build id: 3.10.0-957.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffac47a598
Build id: 3.10.0-957.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffac47a608
Build id: 3.10.0-957.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffffac5e4077
Build id: 3.10.0-957.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffffac5e5975
Build id: 3.10.0-957.el7.x86_64
|
0x55
|
NMI watchdog: BUG: soft lockup - CPU#22 stuck for 23s! [java:20511]
Modules linked in: udp_diag tcp_diag inet_diag binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log ppdev dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device joydev parport_pc virtio_rng snd_pcm parport snd_timer pcspkr snd virtio_balloon i2c_piix4 soundcore ip_tables ext4 mbcache jbd2 ata_generic pata_acpi virtio_console virtio_net virtio_blk bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix floppy serio_raw libata virtio_pci virtio_ring drm_panel_orientation_quirks
virtio
CPU: 22 PID: 20511 Comm: java Kdump: loaded Tainted: G L ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
task: ffff9568b8edd140 ti: ffff956648240000 task.ti: ffff956648240000
RIP: 0010:[<ffffffffac51173a>] [<ffffffffac51173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff956648243cb8 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff956abee60708
RDX: 0000000000000001 RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff956648243cf0 R08: ffff954c533a8800 R09: ffffffffac7755c9
R10: ffff956abf39f160 R11: ffffefbceceec600 R12: 0000000000000296
R13: ffff956648243c68 R14: 0000000000000001 R15: ffffefbcd5af5680
FS: 00007feae15d5700(0000) GS:ffff956abf380000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fed28000000 CR3: 0000000bfee1e000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffac47a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffac47a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffac5e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffac5e5975>] tlb_finish_mmu+0x55/0x60
Complete report #779334
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f99a2
Build id: 3.10.0-693.el7.x86_64
|
0x202
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811b08fa
Build id: 3.10.0-693.el7.x86_64
|
0x13a
|
|
7
|
|
vmlinux |
0xffffffff810b4370
Build id: 3.10.0-693.el7.x86_64
|
0x50
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff811ac71d
Build id: 3.10.0-693.el7.x86_64
|
0x3cd
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [sfavsrv:4715]
Modules linked in: nf_conntrack_ftp udp_diag veth tcp_diag inet_diag xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun binfmt_misc overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock vfat fat sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd
vmw_balloon pcspkr joydev sg vmw_vmci i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix crct10dif_pclmul mptscsih crct10dif_common libata crc32c_intel serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 4715 Comm: sfavsrv Tainted: G L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1410210136 10/21/2014
task: ffff88009d463f40 ti: ffff8803f9f64000 task.ti: ffff8803f9f64000
RIP: 0010:[<ffffffff810f99a2>] [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff8803f9f67cb0 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000800008106e4a0 RCX: ffff880237c9a0e0
RDX: 0000000000000002 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff8803f9f67ce8 R08: ffff88017fc39400 R09: ffffffff813227d9
R10: ffff880237c19c80 R11: ffffea00003a3a00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8803f9f67c60 R15: 0000000000000001
FS: 00007f9960451a80(0000) GS:ffff880237c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc5cec00000 CR3: 0000000230f0c000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00000001b403efff ffff880235777a08 ffff8802357776c0 0000000000000000
ffffffffffffffff ffff880235777a08 ffff8802357776c0 ffff8803f9f67d38
ffffffff8106e668 ffff8802357776c0 0000000000000000 ffffffffffffffff
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b08fa>] zap_page_range+0x13a/0x180
[<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
[<ffffffff811ac71d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89
Complete report #769596
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff8e91173a
Build id: 3.10.0-957.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8e87a598
Build id: 3.10.0-957.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8e87a608
Build id: 3.10.0-957.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff8e9e4077
Build id: 3.10.0-957.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8e9e5975
Build id: 3.10.0-957.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff8e9e87c2
Build id: 3.10.0-957.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff8e9e368d
Build id: 3.10.0-957.el7.x86_64
|
0x49d
|
|
8
|
system_call_fastpath
|
vmlinux |
0xffffffff8ef74ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 42s! [nessusd:23740]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev snd_ens1371 snd_rawmidi snd_ac97_codec vmw_balloon ac97_bus
snd_seq snd_seq_device snd_pcm snd_timer snd pcspkr joydev sg soundcore parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm e1000 ata_piix drm mptspi scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm
CPU: 0 PID: 23740 Comm: nessusd Kdump: loaded Tainted: G L ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8ddaaff0d140 ti: ffff8dda72350000 task.ti: ffff8dda72350000
RIP: 0010:[<ffffffff8e91173a>] [<ffffffff8e91173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8dda72353c78 EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00000000000000fc RCX: ffff8ddb396df610
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8dda72353cb0 R08: ffff8ddb3918d000 R09: ffffffff8eb755c9
R10: ffff8ddb3961f160 R11: fffff1edc28a7800 R12: 0000000000000296
R13: ffff8dda72353c28 R14: 0000000000000001 R15: fffff1edc2a8b2c0
FS: 00007f6c59bfd700(0000) GS:ffff8ddb39600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c643ab000 CR3: 0000000116042000 CR4: 00000000003607f0
Call Trace:
[<ffffffff8e87a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8e87a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8e9e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff8e9e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff8e9e87c2>] zap_page_range+0x142/0x190
[<ffffffff8e9e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff8ef74ddb>] system_call_fastpath+0x22/0x27
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 54 8f f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #595437
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f99a2
Build id: 3.10.0-693.2.2.el7.x86_64
|
0x202
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106e668
Build id: 3.10.0-693.2.2.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106e738
Build id: 3.10.0-693.2.2.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811ad053
Build id: 3.10.0-693.2.2.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ae375
Build id: 3.10.0-693.2.2.el7.x86_64
|
0x55
|
|
6
|
unmap_region
|
vmlinux |
0xffffffff811b5444
Build id: 3.10.0-693.2.2.el7.x86_64
|
0xf4
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [MonitoringResou:22430]
Modules linked in: gsch(OE) redirfs(OE) nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dsa_filter(POE) vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd ppdev sg vmw_vmci joydev pcspkr parport_pc shpchp parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common mptspi scsi_transport_spi mptscsih crc32c_intel serio_raw ata_piix libata mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log
dm_mod
CPU: 2 PID: 22430 Comm: MonitoringResou Tainted: P OEL ------------ 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff8800bb532f70 ti: ffff880037b48000 task.ti: ffff880037b48000
RIP: 0010:[<ffffffff810f99a2>] [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff880037b4bcf0 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 000800008106e4a0 RCX: ffff88023fc9af48
RDX: 0000000000000001 RSI: 0000000000000004 RDI: 0000000000000000
RBP: ffff880037b4bd28 R08: ffff88017ce0dc00 R09: ffffffff813229e9
R10: ffff88023fd19c80 R11: ffffea0001341400 R12: 0000000000000286
R13: 0000000000000286 R14: ffff880037b4bca0 R15: 00007f4927e33000
FS: 00007f48db076700(0000) GS:ffff88023fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4925bf81a0 CR3: 000000022925c000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00000001bb532f70 ffff8800b8a88348 ffff8800b8a88000 00007f4927e33000
00007f4927e34000 ffff8800b8a88348 ffff88013b559438 ffff880037b4bd78
ffffffff8106e668 ffff8800b8a88000 00007f4927e33000 00007f4927e34000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b5444>] unmap_region+0xf4/0x140
Complete report #582613
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff8c907e3a
Build id: 3.10.0-862.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8c877018
Build id: 3.10.0-862.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8c877088
Build id: 3.10.0-862.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.75
|
vmlinux |
0xffffffff8c9c0497
Build id: 3.10.0-862.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8c9c1c75
Build id: 3.10.0-862.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff8c9c4a02
Build id: 3.10.0-862.el7.x86_64
|
0x142
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [HTTPHandler:109335]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables binfmt_misc tcp_diag udp_diag inet_diag ip_set nfnetlink bridge stp llc ppdev parport_pc sg parport virtio_balloon joydev i2c_piix4 pcspkr xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_scsi virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci floppy virtio_ring i2c_core
serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 5 PID: 109335 Comm: HTTPHandler Kdump: loaded Tainted: G L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff9dad53696eb0 ti: ffff9da3475f0000 task.ti: ffff9da3475f0000
RIP: 0010:[<ffffffff8c907e3a>] [<ffffffff8c907e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9da3475f3c78 EFLAGS: 00000202
RAX: 0000000000000030 RBX: 00000000000000fc RCX: ffff9dc2fda1cc70
RDX: 0000000000000030 RSI: 00000000000000a0 RDI: 0000000000000000
RBP: ffff9da3475f3cb0 R08: ffff9da493054000 R09: ffffffff8cb4a3d9
R10: ffff9dc2fcf5bb00 R11: fffff3c104c12e00 R12: 0000000000000296
R13: 0000000000000018 R14: ffff9da3475f3be8 R15: 0000000000000296
FS: 00007f5050ff8700(0000) GS:ffff9dc2fcf40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000006a29b005 CR3: 000000011cd92000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8c877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c877088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8c9c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff8c9c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff8c9c4a02>] zap_page_range+0x142/0x190
Complete report #374836
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f99a6
Build id: 3.10.0-693.el7.x86_64
|
0x206
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811b08fa
Build id: 3.10.0-693.el7.x86_64
|
0x13a
|
|
7
|
|
vmlinux |
0xffffffff8120026d
Build id: 3.10.0-693.el7.x86_64
|
0x8d
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff811ac71d
Build id: 3.10.0-693.el7.x86_64
|
0x3cd
|
|
9
|
|
vmlinux |
0xffffffff81200dea
Build id: 3.10.0-693.el7.x86_64
|
0x17a
|
|
10
|
system_call_fastpath
|
vmlinux |
0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [mysqld:2957]
Modules linked in: sunrpc binfmt_misc veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device ppdev sg crc32_pclmul ghash_clmulni_intel aesni_intel snd_pcm snd_timer snd i2c_piix4 soundcore lrw gf128mul glue_helper ablk_helper cryptd pcspkr parport_pc parport video ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel serio_raw drm ahci libahci ata_piix libata e1000 i2c_core dm_mirror dm_region_hash
dm_log dm_mod
CPU: 0 PID: 2957 Comm: mysqld Tainted: G W ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8801f5a95ee0 ti: ffff88020c7d0000 task.ti: ffff88020c7d0000
RIP: 0010:[<ffffffff810f99a6>] [<ffffffff810f99a6>] smp_call_function_many+0x206/0x260
RSP: 0018:ffff88020c7d3cb0 EFLAGS: 00000202
RAX: 0000000000000003 RBX: 000800008106e4a0 RCX: ffff88021fd9a0e0
RDX: 0000000000000003 RSI: 0000000000000004 RDI: 0000000000000000
RBP: ffff88020c7d3ce8 R08: ffff88017fc3a400 R09: ffffffff813227d9
R10: ffff88021fc19c80 R11: ffffea0002b1d900 R12: 0000000000000282
R13: 0000000000000282 R14: ffff88020c7d3c60 R15: 0000000000000001
FS: 00007f9caa86e700(0000) GS:ffff88021fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9caa866bd0 CR3: 0000000213413000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
00000001aa868fff ffff88004873e748 ffff88004873e400 0000000000000000
ffffffffffffffff ffff88004873e748 ffff88004873e400 ffff88020c7d3d38
ffffffff8106e668 ffff88004873e400 0000000000000000 ffffffffffffffff
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b08fa>] zap_page_range+0x13a/0x180
[<ffffffff8120026d>] ? do_sync_write+0x8d/0xd0
[<ffffffff811ac71d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff81200dea>] ? vfs_write+0x17a/0x1e0
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #238911
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffab707e3a
Build id: 3.10.0-862.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffab677018
Build id: 3.10.0-862.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffab677088
Build id: 3.10.0-862.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.75
|
vmlinux |
0xffffffffab7c0497
Build id: 3.10.0-862.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffffab7c1c75
Build id: 3.10.0-862.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffffab7c4a02
Build id: 3.10.0-862.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffffab7bfafd
Build id: 3.10.0-862.el7.x86_64
|
0x3cd
|
|
8
|
|
vmlinux |
0xffffffffab6d80a6
Build id: 3.10.0-862.el7.x86_64
|
0x86
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffffabd1f7d5
Build id: 3.10.0-862.el7.x86_64
|
0x1c
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kubelet:91865]
Modules linked in: xt_set xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ip_set nf_conntrack_netlink nfnetlink xt_nat xt_statistic ip_vs_sh ip_vs_wrr ip_vs_rr veth ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache xt_comment xt_mark xt_addrtype br_netfilter ip_vs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock overlay(T) snd_seq_midi snd_seq_midi_event iosf_mbi snd_ens1371 crc32_pclmul snd_rawmidi snd_ac97_codec ghash_clmulni_intel aesni_intel lrw gf128mul ac97_bus snd_seq
snd_seq_device ppdev snd_pcm glue_helper snd_timer sg ablk_helper pcspkr snd cryptd joydev soundcore nfit libnvdimm vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix mptspi libata serio_raw scsi_transport_spi mptscsih e1000 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 91865 Comm: kubelet Kdump: loaded Tainted: G L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9e3ff5730fd0 ti: ffff9e3fd82a8000 task.ti: ffff9e3fd82a8000
RIP: 0010:[<ffffffffab707e3a>] [<ffffffffab707e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9e3fd82abc78 EFLAGS: 00000202
RAX: 0000000000000005 RBX: 00000000000000fc RCX: ffff9e401cd5cf28
RDX: 0000000000000005 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9e3fd82abcb0 R08: ffff9e3f3bf2c000 R09: ffffffffab94a3d9
R10: ffffdddd3df01fd0 R11: ffffeea487b52000 R12: 0000000000000296
R13: ffff9e3fd82abc28 R14: 0000000000000001 R15: ffffffffab7f8139
FS: 00007f593dffb700(0000) GS:ffff9e401cd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0d129249a8 CR3: 000000009e1a4000 CR4: 00000000003607e0
Call Trace:
[<ffffffffab677018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffab677088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffab7c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffab7c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffab7c4a02>] zap_page_range+0x142/0x190
[<ffffffffab7bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffab6d80a6>] ? update_curr+0x86/0x1e0
[<ffffffffabd1f7d5>] system_call_fastpath+0x1c/0x21
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 33 ac f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #175747
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffb5911a0a
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb587a628
Build id: 3.10.0-957.12.2.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb587a698
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffffb59e4607
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb59e5f05
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x55
|
|
6
|
unmap_region
|
vmlinux |
0xffffffffb59ef5a4
Build id: 3.10.0-957.12.2.el7.x86_64
|
0xf4
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [java:20012]
Modules linked in: nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod coretemp snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 ppdev snd_rawmidi snd_ac97_codec ac97_bus pcspkr joydev snd_seq snd_seq_device snd_pcm snd_timer snd sg vmw_balloon soundcore parport_pc parport vmw_vmci i2c_piix4 ip_tables
xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper syscopyarea sysfillrect e1000 sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm
CPU: 4 PID: 20012 Comm: java Kdump: loaded Tainted: G L ------------ T 3.10.0-957.12.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8b3ba0b04100 ti: ffff8b3bd0ac4000 task.ti: ffff8b3bd0ac4000
RIP: 0010:[<ffffffffb5911a0a>] [<ffffffffb5911a0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8b3bd0ac7cb8 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff8b3c1fa60628
RDX: 0000000000000001 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8b3bd0ac7cf0 R08: ffff8b3a7b3b4000 R09: ffffffffb5b75d69
R10: ffff8b3c1fb1f160 R11: ffffe5700a67e200 R12: 0000000000000296
R13: ffff8b3bd0ac7c68 R14: 0000000000000001 R15: ffffe570012ef680
FS: 00007fc9597f7700(0000) GS:ffff8b3c1fb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc9d401e000 CR3: 00000002faac2000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb587a628>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb587a698>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb59e4607>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffb59e5f05>] tlb_finish_mmu+0x55/0x60
[<ffffffffb59ef5a4>] unmap_region+0xf4/0x140
Complete report #173484
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff82d1173a
Build id: 3.10.0-957.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff82c7a598
Build id: 3.10.0-957.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff82c7a608
Build id: 3.10.0-957.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff82de4077
Build id: 3.10.0-957.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff82de5975
Build id: 3.10.0-957.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff82de87c2
Build id: 3.10.0-957.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff82de368d
Build id: 3.10.0-957.el7.x86_64
|
0x49d
|
|
8
|
system_call_fastpath
|
vmlinux |
0xffffffff83374ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [java:6898]
Modules linked in: kbase(OE) udp_diag tcp_diag inet_diag binfmt_misc fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd parport_pc parport virtio_balloon joydev i2c_piix4 pcspkr ip_tables ext4 mbcache jbd2 ata_generic pata_acpi
virtio_blk virtio_console virtio_net cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix floppy libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 6898 Comm: java Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
task: ffff9bfef5fe2080 ti: ffff9c03d9e1c000 task.ti: ffff9c03d9e1c000
RIP: 0010:[<ffffffff82d1173a>] [<ffffffff82d1173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9c03d9e1fc78 EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00000000000000fc RCX: ffff9c0e3f2e0528
RDX: 0000000000000003 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9c03d9e1fcb0 R08: ffff9bffa94f5000 R09: ffffffff82f755c9
R10: ffff9c0e3f45f160 R11: fffff81effe3de00 R12: 0000000000000296
R13: ffff9c03d9e1fc28 R14: 0000000000000001 R15: fffff81eeb1b8980
FS: 00007ff28106a700(0000) GS:ffff9c0e3f440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff5036e8530 CR3: 0000000fdba6e000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff82c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff82c7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff82de4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff82de5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff82de87c2>] zap_page_range+0x142/0x190
[<ffffffff82de368d>] SyS_madvise+0x49d/0xac0
[<ffffffff83374ddb>] system_call_fastpath+0x22/0x27
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 83 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #39230
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff8ad07e3a
Build id: 3.10.0-862.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8ac77018
Build id: 3.10.0-862.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8ac77088
Build id: 3.10.0-862.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.75
|
vmlinux |
0xffffffff8adc0497
Build id: 3.10.0-862.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8adc1c75
Build id: 3.10.0-862.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff8adc4a02
Build id: 3.10.0-862.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff8adbfafd
Build id: 3.10.0-862.el7.x86_64
|
0x3cd
|
|
8
|
system_call_fastpath
|
vmlinux |
0xffffffff8b31f7d5
Build id: 3.10.0-862.el7.x86_64
|
0x1c
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [checkThread:63079]
Modules linked in: tcp_lp binfmt_misc uinput fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd sg virtio_balloon joydev parport_pc parport i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect virtio_net virtio_blk virtio_scsi virtio_console sysimgblt fb_sys_fops ttm ata_piix drm virtio_pci crct10dif_pclmul crct10dif_common floppy virtio_ring i2c_core serio_raw
crc32c_intel libata virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 63079 Comm: checkThread Kdump: loaded Tainted: G L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.9.1-5.el7.centos 04/01/2014
task: ffff8daafd6a3f40 ti: ffff8dac21acc000 task.ti: ffff8dac21acc000
RIP: 0010:[<ffffffff8ad07e3a>] [<ffffffff8ad07e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8dac21acfc78 EFLAGS: 00200202
RAX: 0000000000000004 RBX: 00000000000000fc RCX: ffff8dac7d71bf60
RDX: 0000000000000004 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8dac21acfcb0 R08: ffff8d9dffd8d000 R09: ffffffff8af4a3d9
R10: ffff8dac7d61bb00 R11: ffffe71edac0e200 R12: 0000000000200296
R13: ffff8dac21acfc28 R14: 0000000000000001 R15: ffffe71ee19b3000
FS: 00007feee19d4700(0000) GS:ffff8dac7d600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007feee19c1a30 CR3: 0000000c5ea48000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8ac77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ac77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8adc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff8adc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff8adc4a02>] zap_page_range+0x142/0x190
[<ffffffff8adbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff8b31f7d5>] system_call_fastpath+0x1c/0x21
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 93 8b f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #34903
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810fd5fa
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff81070b98
Build id: 3.10.0-693.21.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff81070c68
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811b18c3
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811b2be5
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811b516a
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x13a
|
|
7
|
|
vmlinux |
0xffffffff811e3392
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x1e2
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff811b0f8d
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x3cd
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x1c
|
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [odoo:32266]
Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi aesni_intel snd_ac97_codec lrw gf128mul glue_helper ac97_bus ablk_helper snd_seq cryptd snd_seq_device snd_pcm ppdev vmw_balloon snd_timer snd soundcore
joydev vmw_vmci nfit pcspkr sg i2c_piix4 shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel mptspi scsi_transport_spi mptscsih e1000 serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 32266 Comm: odoo Tainted: G L ------------ 3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff88083fe0af70 ti: ffff880814f28000 task.ti: ffff880814f28000
RIP: 0010:[<ffffffff810fd5fa>] [<ffffffff810fd5fa>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff880814f2bc80 EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00080000810709d0 RCX: ffff88081d6dcdd0
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff880814f2bcb8 R08: ffff88017453dc00 R09: ffffffff81328989
R10: ffff88081d85ba00 R11: ffffea001f398e00 R12: 0000000000000286
R13: 0000000000000286 R14: ffff880814f2bc30 R15: 0000000000000001
FS: 00007f2e24ff9700(0000) GS:ffff88081d840000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff1bc3b9000 CR3: 000000081bcdc000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff81070b98>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81070c68>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811b18c3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b2be5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b516a>] zap_page_range+0x13a/0x180
[<ffffffff811e3392>] ? kmem_cache_free+0x1e2/0x200
[<ffffffff811b0f8d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 48 63 35 8e 1b a3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 10 b2 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d 1b a3 00 eb b7 0f b6 4d cc 4c 89
Complete report #22736
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f99a6
Build id: 3.10.0-693.el7.x86_64
|
0x206
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811b08fa
Build id: 3.10.0-693.el7.x86_64
|
0x13a
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff811ac71d
Build id: 3.10.0-693.el7.x86_64
|
0x3cd
|
|
8
|
system_call_fastpath
|
vmlinux |
0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [java:2656]
Modules linked in: fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus edac_core snd_seq coretemp snd_seq_device iosf_mbi crc32_pclmul snd_pcm ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_timer snd ppdev pcspkr joydev soundcore sg vmw_balloon i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic
pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ata_piix libata e1000 mptspi scsi_transport_spi mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2656 Comm: java Tainted: G L ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8803f96dcf10 ti: ffff8800ae7b8000 task.ti: ffff8800ae7b8000
RIP: 0010:[<ffffffff810f99a6>] [<ffffffff810f99a6>] smp_call_function_many+0x206/0x260
RSP: 0018:ffff8800ae7bbcb0 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800008106e4a0 RCX: ffff88042d61af70
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8800ae7bbce8 R08: ffff88017a3a3c00 R09: ffffffff813227d9
R10: ffff88042d6d9c80 R11: ffffea0010fe2c00 R12: 0000000000000282
R13: 0000000000000018 R14: ffff8800ae7bbc20 R15: 0000000000000282
FS: 00007f7ba0634700(0000) GS:ffff88042d6c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7c1805f680 CR3: 00000003cc688000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00000001a062efff ffff8803e8f10fc8 ffff8803e8f10c80 0000000000000000
ffffffffffffffff ffff8803e8f10fc8 ffff8803e8f10c80 ffff8800ae7bbd38
ffffffff8106e668 ffff8803e8f10c80 0000000000000000 ffffffffffffffff
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b08fa>] zap_page_range+0x13a/0x180
[<ffffffff811ac71d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #15597
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff95d11a8a
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff95c7a6a8
Build id: 3.10.0-957.21.3.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff95c7a718
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff95de4687
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff95de5f85
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff95de8dd2
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff95de3c9d
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x49d
|
|
8
|
system_call_fastpath
|
vmlinux |
0xffffffff96375ddb
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [freeswitch:3037]
Modules linked in: tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon pcspkr sg joydev vmw_vmci i2c_piix4 parport_pc parport
ip_tables xfs libcrc32c vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper ata_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm pata_acpi drm nfit mptspi scsi_transport_spi e1000 mptscsih ata_piix libnvdimm libata crct10dif_pclmul crct10dif_common mptbase crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 3037 Comm: freeswitch Kdump: loaded Tainted: G W L ------------ 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff94ccaa322080 ti: ffff94cba8968000 task.ti: ffff94cba8968000
RIP: 0010:[<ffffffff95d11a8a>] [<ffffffff95d11a8a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff94cba896bc78 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff94cd3fc20600
RDX: 0000000000000000 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff94cba896bcb0 R08: ffff94ca7a34f400 R09: ffffffff95f75e39
R10: ffff94cd3fcdf160 R11: ffffefd10e7ca600 R12: 0000000000000296
R13: ffff94cba896bc28 R14: 0000000000000001 R15: ffffefd1082a5e80
FS: 00007fcfc3788700(0000) GS:ffff94cd3fcc0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe54ea28900 CR3: 00000000bb7da000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff95c7a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff95c7a718>] flush_tlb_mm_range+0x68/0x140
[<ffffffff95de4687>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff95de5f85>] tlb_finish_mmu+0x55/0x60
[<ffffffff95de8dd2>] zap_page_range+0x142/0x190
[<ffffffff95de3c9d>] SyS_madvise+0x49d/0xac0
[<ffffffff96375ddb>] system_call_fastpath+0x22/0x27
Code: 48 63 35 3e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 bc 94 96 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 0d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #14536
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff810f98e2
Build id: 3.10.0-514.el7.x86_64
|
0x202
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106c4c8
Build id: 3.10.0-514.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106c599
Build id: 3.10.0-514.el7.x86_64
|
0x69
|
|
4
|
tlb_flush_mmu.part.61
|
vmlinux |
0xffffffff811ac093
Build id: 3.10.0-514.el7.x86_64
|
0x33
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811ad385
Build id: 3.10.0-514.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff811af75a
Build id: 3.10.0-514.el7.x86_64
|
0x13a
|
|
7
|
|
vmlinux |
0xffffffff811b0de7
Build id: 3.10.0-514.el7.x86_64
|
0x6d7
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff811ab7ec
Build id: 3.10.0-514.el7.x86_64
|
0x38c
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff816965c9
Build id: 3.10.0-514.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [java:8242]
Modules linked in: iptable_filter nfnetlink_queue nfnetlink_log nfnetlink bluetooth rfkill vmw_vsock_vmci_transport vsock intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg vmw_balloon pcspkr parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect mptspi sysimgblt scsi_transport_spi fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix drm crc32c_intel serio_raw mptscsih libata mptbase vmxnet3 i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 15 PID: 8242 Comm: java Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff88083330edd0 ti: ffff880712218000 task.ti: ffff880712218000
RIP: 0010:[<ffffffff810f98e2>] [<ffffffff810f98e2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff88071221bcc8 EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00000000000000fc RCX: ffff880237d9b100
RDX: 0000000000000003 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff88071221bd00 R08: ffff8806c5109400 R09: ffffffff81317a89
R10: ffff88083fd99b40 R11: ffffea0010db9e00 R12: 0000000000000292
R13: ffff88071221bc78 R14: ffff880187c70400 R15: 0000000000000001
FS: 00007f52615d1700(0000) GS:ffff88083fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5104000898 CR3: 00000001a9ee0000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00000001615cc000 ffff880234b5f3c8 ffff880234b5f080 0000000000000000
ffffffffffffffff ffff880234b5f3c8 ffff880234b5f080 ffff88071221bd50
ffffffff8106c4c8 ffff880234b5f080 0000000000000000 ffffffffffffffff
Call Trace:
[<ffffffff8106c4c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c599>] flush_tlb_mm_range+0x69/0x140
[<ffffffff811ac093>] tlb_flush_mmu.part.61+0x33/0xc0
[<ffffffff811ad385>] tlb_finish_mmu+0x55/0x60
[<ffffffff811af75a>] zap_page_range+0x13a/0x180
[<ffffffff811b0de7>] ? handle_mm_fault+0x6d7/0xfe0
[<ffffffff811ab7ec>] SyS_madvise+0x38c/0x8d0
[<ffffffff816965c9>] system_call_fastpath+0x16/0x1b
Code: 48 63 35 76 9e 9e 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 40 57 ad 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 45 9e 9e 00 eb b7 0f b6 4d cc 4c 89
Complete report #10181
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff84d1185a
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff84c7a5c8
Build id: 3.10.0-957.10.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff84c7a638
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff84de4507
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff84de5e05
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff84de8c52
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff84de3b1d
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x49d
|
|
8
|
system_call_fastpath
|
vmlinux |
0xffffffff85375ddb
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sidb2:60365]
Modules linked in: cvbf(POE) dm_snapshot dm_bufio binfmt_misc vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg nfit vmw_vmci i2c_piix4 libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect serio_raw sysimgblt fb_sys_fops ttm drm ata_piix vmw_pvscsi libata vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: cvblk]
CPU: 1 PID: 60365 Comm: sidb2 Kdump: loaded Tainted: P OEL ------------ 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff89d3ae2b8000 ti: ffff89d394a30000 task.ti: ffff89d394a30000
RIP: 0010:[<ffffffff84d1185a>] [<ffffffff84d1185a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff89d394a33c78 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff89d3ad620490
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff89d394a33cb0 R08: ffff89d0f9cb3400 R09: ffffffff84f75c99
R10: ffff89d3ad65f160 R11: ffffe5e81096e800 R12: 0000000000000296
R13: ffff89d394a33c28 R14: 0000000000000001 R15: ffffe5e802eb43c0
FS: 00007f70cf053700(0000) GS:ffff89d3ad640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9cbff68000 CR3: 000000010a240000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff84c7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff84c7a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff84de4507>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff84de5e05>] tlb_finish_mmu+0x55/0x60
[<ffffffff84de8c52>] zap_page_range+0x142/0x190
[<ffffffff84de3b1d>] SyS_madvise+0x49d/0xac0
[<ffffffff85375ddb>] system_call_fastpath+0x22/0x27
Code: 48 63 35 6e 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 b9 94 85 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 82 c4 00 eb b7 0f b6 4d cc 4c 89