Complete report #1375738
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffa651e2b8
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0x208
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa6481fa8
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa648201b
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa65fa12e
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0xae
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa65fbcaf
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa65fbde3
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffa65ff32a
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffa660931c
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0x39c
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffa65f971d
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0x49d
|
|
10
|
system_call_fastpath
|
vmlinux |
0xffffffffa6bc539a
Build id: 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [process reaper:46982]
Modules linked in: sunrpc vfat fat ppdev nfit libnvdimm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr parport_pc parport i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix xen_netfront xen_blkfront libata crct10dif_pclmul crct10dif_common crc32c_intel floppy serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod tun fuse
CPU: 7 PID: 46982 Comm: process reaper Kdump: loaded Tainted: G L ------------ 3.10.0-1160.119.1.el7.tuxcare.els19.x86_64 #1
Hardware name: Xen HVM domU, BIOS 4.13 09/09/2025
task: ffff9d22045a1080 ti: ffff9d2323914000 task.ti: ffff9d2323914000
RIP: 0010:[<ffffffffa651e2b8>] [<ffffffffa651e2b8>] smp_call_function_many+0x208/0x270
RSP: 0018:ffff9d2323917c70 EFLAGS: 00000202
RAX: 000000000000000d RBX: ffff9d2323917c10 RCX: ffff9d29ee5605b8
RDX: 000000000000000d RSI: 0000000000000040 RDI: 0000000000000000
RBP: ffff9d2323917ca8 R08: ffff9d236c79a800 R09: 0000000000000001
R10: ffff9d236c79a800 R11: 00007f07dad19000 R12: ffff9d2323917c00
R13: ffffffffa6877ff6 R14: ffff9d2323917be0 R15: ffffffffa655b78e
FS: 00007f07dad4e700(0000) GS:ffff9d29ee3c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc0b5c204b1 CR3: 000000018c920000 CR4: 00000000007606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa6481fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa648201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa65fa12e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa65fbcaf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa65fbde3>] tlb_finish_mmu+0x23/0x40
[<ffffffffa65ff32a>] zap_page_range+0xea/0x140
[<ffffffffa660931c>] ? do_mmap+0x39c/0x590
[<ffffffffa65f971d>] SyS_madvise+0x49d/0xac0
[<ffffffffa6bc539a>] system_call_fastpath+0x25/0x2a
Code: 29 00 48 63 35 0e 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 80 19 15 a7 f6 41 20 01 74 cd 0f 1f 44 00 00 <f3> 90 f6 41 20 01 75 f8 48 63 35 dd 17 c4 00 eb b7 0f b6 4d cc
Complete report #1363664
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffb311e07a
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb3081f48
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb3081fbb
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffb31f9f0e
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0xae
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffb31fba4f
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb31fbb83
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffb31ff0ca
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffb31194d0
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x90
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffb31f949d
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x49d
|
|
10
|
|
vmlinux |
0xffffffffb31ea203
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0xf3
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffffb37c539a
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [java:55853]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq sb_edac snd_seq_device snd_pcm iosf_mbi snd_timer crc32_pclmul ghash_clmulni_intel aesni_intel lrw snd gf128mul glue_helper ppdev ablk_helper cryptd soundcore vmw_balloon sg pcspkr joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix nfit drm libnvdimm libata crc32c_intel e1000 mptspi serio_raw scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 55853 Comm: java Kdump: loaded Tainted: G L ------------ T 3.10.0-1160.81.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff90c691722100 ti: ffff90c8e7028000 task.ti: ffff90c8e7028000
RIP: 0010:[<ffffffffb311e07a>] [<ffffffffb311e07a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff90c8e702bc70 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00080000b3081ab0 RCX: ffff90c8e9660650
RDX: 0000000000000001 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff90c8e702bca8 R08: ffff90c53255c000 R09: ffffffffb339c139
R10: ffff90c8e96df160 R11: fffffa350b455400 R12: 0000000000000282
R13: 0000000000000018 R14: ffff90c8e702bbe0 R15: 0000000000000282
FS: 00007f146bd17700(0000) GS:ffff90c8e96c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b77484c38 CR3: 00000004f55de000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb3081f48>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb3081fbb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb31f9f0e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffb31fba4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb31fbb83>] tlb_finish_mmu+0x23/0x40
[<ffffffffb31ff0ca>] zap_page_range+0xea/0x140
[<ffffffffb31194d0>] ? futex_wake+0x90/0x190
[<ffffffffb31f949d>] SyS_madvise+0x49d/0xac0
[<ffffffffb31ea203>] ? vm_mmap_pgoff+0xf3/0x120
[<ffffffffb37c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ce 19 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 00 19 d5 b3 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 19 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1302727
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffa5516c0a
Build id: 3.10.0-1160.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa547e868
Build id: 3.10.0-1160.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa547e8db
Build id: 3.10.0-1160.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa55edd06
Build id: 3.10.0-1160.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa55ef7cf
Build id: 3.10.0-1160.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa55ef8f3
Build id: 3.10.0-1160.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffa55f2dba
Build id: 3.10.0-1160.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffa54daaa8
Build id: 3.10.0-1160.el7.x86_64
|
0x58
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffa55ed2ad
Build id: 3.10.0-1160.el7.x86_64
|
0x49d
|
|
10
|
|
vmlinux |
0xffffffffa55de1bb
Build id: 3.10.0-1160.el7.x86_64
|
0xeb
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffffa5b93f92
Build id: 3.10.0-1160.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [Metrics-HealthC:23553]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) nfit libnvdimm snd_hda_codec_generic snd_hda_intel snd_hda_codec iosf_mbi crc32_pclmul ghash_clmulni_intel snd_hda_core snd_hwdep aesni_intel snd_seq lrw gf128mul snd_seq_device snd_pcm ppdev glue_helper ablk_helper cryptd snd_timer i2c_piix4 snd soundcore joydev parport_pc virtio_balloon parport pcspkr ip_tables xfs libcrc32c qxl drm_kms_helper ata_generic pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_blk drm ata_piix 8139too libata crct10dif_pclmul crct10dif_common floppy crc32c_intel virtio_pci
serio_raw virtio_ring 8139cp virtio drm_panel_orientation_quirks mii dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 23553 Comm: Metrics-HealthC Kdump: loaded Tainted: G L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8dd8f6cdd280 ti: ffff8dd661b74000 task.ti: ffff8dd661b74000
RIP: 0010:[<ffffffffa5516c0a>] [<ffffffffa5516c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8dd661b77c70 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00080000a547e6a0 RCX: ffff8dda7fc60398
RDX: 0000000000000001 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff8dd661b77ca8 R08: ffff8dd69cbb9c00 R09: ffffffffa5785f99
R10: ffff8dda7fd1f160 R11: fffff87190a1d600 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8dd661b77c20 R15: 0000000000000001
FS: 00007f8cfbdfd640(0000) GS:ffff8dda7fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055dd0e4e0330 CR3: 0000000449bc2000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa547e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa547e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa55edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa55ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa55ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffa55f2dba>] zap_page_range+0xea/0x130
[<ffffffffa54daaa8>] ? wake_up_q+0x58/0x80
[<ffffffffa55ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffa55de1bb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffa5b93f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 15 a6 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1249396
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffb4116c0a
Build id: 3.10.0-1160.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb407e868
Build id: 3.10.0-1160.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb407e8db
Build id: 3.10.0-1160.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffb41edd06
Build id: 3.10.0-1160.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffb41ef7cf
Build id: 3.10.0-1160.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb41ef8f3
Build id: 3.10.0-1160.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffb41f2dba
Build id: 3.10.0-1160.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffb40daaa8
Build id: 3.10.0-1160.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffffb4112040
Build id: 3.10.0-1160.el7.x86_64
|
0x90
|
|
10
|
SyS_madvise
|
vmlinux |
0xffffffffb41ed2ad
Build id: 3.10.0-1160.el7.x86_64
|
0x49d
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffffb4793f92
Build id: 3.10.0-1160.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [JS Helper:97005]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel ac97_bus snd_seq vmw_balloon lrw gf128mul glue_helper ablk_helper snd_seq_device cryptd snd_pcm snd_timer joydev snd
sg pcspkr soundcore vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul drm crct10dif_common crc32c_intel ata_piix nfit mptspi scsi_transport_spi libata mptscsih libnvdimm e1000 mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 6 PID: 97005 Comm: JS Helper Kdump: loaded Tainted: G L ------------ 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffffa01f25b54200 ti: ffffa01e879ec000 task.ti: ffffa01e879ec000
RIP: 0010:[<ffffffffb4116c0a>] [<ffffffffb4116c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffffa01e879efc70 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00080000b407e6a0 RCX: ffffa01f7d6206c8
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffffa01e879efca8 R08: ffffa01ebe34c400 R09: ffffffffb4385f99
R10: ffffa01f7d79f160 R11: ffffea6142d2e400 R12: 0000000000000282
R13: 0000000000000282 R14: ffffa01e879efc20 R15: 0000000000000001
FS: 00007fda39406700(0000) GS:ffffa01f7d780000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fda294d2ff0 CR3: 000000003d08e000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb407e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb407e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb41edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb41ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb41ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffb41f2dba>] zap_page_range+0xea/0x130
[<ffffffffb40daaa8>] ? wake_up_q+0x58/0x80
[<ffffffffb4112040>] ? futex_wake+0x90/0x180
[<ffffffffb41ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffb4793f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 d5 b4 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #913573
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff90516c0a
Build id: 3.10.0-1160.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff9047e868
Build id: 3.10.0-1160.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9047e8db
Build id: 3.10.0-1160.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff905edd06
Build id: 3.10.0-1160.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff905ef7cf
Build id: 3.10.0-1160.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffff905ef8f3
Build id: 3.10.0-1160.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffff905f2dba
Build id: 3.10.0-1160.el7.x86_64
|
0xea
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff905ed2ad
Build id: 3.10.0-1160.el7.x86_64
|
0x49d
|
|
9
|
|
vmlinux |
0xffffffff905de1bb
Build id: 3.10.0-1160.el7.x86_64
|
0xeb
|
|
10
|
system_call_fastpath
|
vmlinux |
0xffffffff90b93f92
Build id: 3.10.0-1160.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [java:39553]
Modules linked in: vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon pcspkr sg joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix mptspi libata scsi_transport_spi mptscsih serio_raw nfit e1000 mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 39553 Comm: java Kdump: loaded Tainted: G W L ------------ 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff97d75a6eb180 ti: ffff97d42e1c0000 task.ti: ffff97d42e1c0000
RIP: 0010:[<ffffffff90516c0a>] [<ffffffff90516c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff97d42e1c3c70 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800009047e6a0 RCX: ffff97d8a3620600
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff97d42e1c3ca8 R08: ffff97d37ef5b800 R09: ffffffff90785f99
R10: ffff97d8a365f160 R11: ffffee74157b5600 R12: 0000000000000282
R13: 0000000000000282 R14: ffff97d42e1c3c20 R15: 0000000000000001
FS: 00007f07744d0700(0000) GS:ffff97d8a3640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558b231a36f8 CR3: 0000000035f8c000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff9047e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9047e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff905edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff905ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff905ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffff905f2dba>] zap_page_range+0xea/0x130
[<ffffffff905ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffff905de1bb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff90b93f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 15 91 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #807602
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffa31170d8
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x208
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa307e258
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa307e2cb
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa31ee676
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa31f013f
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa31f0263
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffa31f372a
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffa31fd36c
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x39c
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffa31edc1d
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x49d
|
|
10
|
|
vmlinux |
0xffffffffa31dedab
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0xeb
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffffa3795f92
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [ssh-agent: stdo:23537]
Modules linked in: xt_comment xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink tun vmw_vsock_vmci_transport vsock sunrpc vfat fat sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common mptspi libata crc32c_intel scsi_transport_spi mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ipmi_msghandler]
CPU: 4 PID: 23537 Comm: ssh-agent: stdo Tainted: G W L ------------ 3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1704120155 04/12/2017
task: ffff9d141f2fb180 ti: ffff9d0ea5fbc000 task.ti: ffff9d0ea5fbc000
RIP: 0010:[<ffffffffa31170d8>] [<ffffffffa31170d8>] smp_call_function_many+0x208/0x270
RSP: 0018:ffff9d0ea5fbfc70 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00080000a307ddd0 RCX: ffff9d157fca0678
RDX: 0000000000000002 RSI: 0000000000000007 RDI: 0000000000000000
RBP: ffff9d0ea5fbfca8 R08: ffff9d0ca5624c00 R09: ffffffffa3386bf9
R10: ffff9d157fd1f160 R11: ffffed871bb1a400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9d0ea5fbfc20 R15: 0000000000000001
FS: 00007ff4d81c3700(0000) GS:ffff9d157fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff4d81c0908 CR3: 000000093e804000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa307e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa307e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa31ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa31f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa31f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffa31f372a>] zap_page_range+0xea/0x130
[<ffffffffa31fd36c>] ? do_mmap+0x39c/0x590
[<ffffffffa31edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffa31dedab>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffa3795f92>] system_call_fastpath+0x25/0x2a
Code: 28 00 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 d5 a3 f6 41 20 01 74 cd 0f 1f 44 00 00 <f3> 90 f6 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc
Complete report #589266
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffa111718a
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa107e258
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa107e2cb
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa11ee6a6
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa11f016f
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa11f0293
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffa11f375a
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffa11fd34c
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x39c
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffa11edc4d
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0x49d
|
|
10
|
|
vmlinux |
0xffffffffa11deddb
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xeb
|
|
11
|
tracesys
|
vmlinux |
0xffffffffa179a226
Build id: 3.10.0-1160.76.1.el7.x86_64
|
0xa6
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [RemoteInvocatio:30691]
Modules linked in: binfmt_misc unix_diag af_packet_diag netlink_diag udp_diag tcp_diag inet_diag uinput nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink cmac arc4 md4 nls_utf8 cifs ccm dns_resolver overlay(T) dm_mirror dm_region_hash dm_log dm_mod sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul sg glue_helper joydev pcspkr ablk_helper cryptd i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover virtio_console
virtio_scsi failover qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy fuse
CPU: 0 PID: 30691 Comm: RemoteInvocatio Tainted: G L ------------ T 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffff978a267f4200 ti: ffff978ad644c000 task.ti: ffff978ad644c000
RIP: 0010:[<ffffffffa111718a>] [<ffffffffa111718a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff978ad644fc70 EFLAGS: 00000202
RAX: 0000000000000004 RBX: 00080000a107ddd0 RCX: ffff978dbfd1fc10
RDX: 0000000000000004 RSI: 0000000000000006 RDI: 0000000000000000
RBP: ffff978ad644fca8 R08: ffff978affd4f800 R09: ffffffffa13869f9
R10: ffff978dbfc1f160 R11: fffff42684b13500 R12: 0000000000000282
R13: 0000000000000282 R14: ffff978ad644fc20 R15: 0000000000000001
FS: 00007fbf078f8700(0000) GS:ffff978dbfc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbf364a0768 CR3: 000000009fede000 CR4: 0000000000160ff0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa107e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa107e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa11ee6a6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa11f016f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa11f0293>] tlb_finish_mmu+0x23/0x30
[<ffffffffa11f375a>] zap_page_range+0xea/0x130
[<ffffffffa11fd34c>] ? do_mmap+0x39c/0x590
[<ffffffffa11edc4d>] SyS_madvise+0x49d/0xac0
[<ffffffffa11deddb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffa179a226>] tracesys+0xa6/0xcc
Code: 48 63 35 be 88 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 00 19 d5 a1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 8d 88 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #243730
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffb4b16a7a
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb4a7e478
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb4a7e4eb
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffb4bec096
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffb4bedb5f
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb4bedc83
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffb4bf098a
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffb4bfad78
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x378
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffb4beb69d
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x49d
|
|
10
|
|
vmlinux |
0xffffffffb4bddffb
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0xeb
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffffb518dede
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [java:5257]
Modules linked in: cts rpcsec_gss_krb5 veth nfsd nfsv3 nfs_acl nfs lockd grace fscache openafs(POE) xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) nls_utf8 isofs vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr i2c_piix4 parport_pc vmw_vmci parport binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit crc32c_intel serio_raw libnvdimm vmxnet3 ata_generic drm_panel_orientation_quirks pata_acpi sd_mod
crc_t10dif crct10dif_generic sr_mod crct10dif_pclmul crct10dif_common cdrom mptspi scsi_transport_spi mptscsih mptbase ata_piix libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 5257 Comm: java Kdump: loaded Tainted: P OEL ------------ T 3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8c7ec2a2b150 ti: ffff8c76edfc8000 task.ti: ffff8c76edfc8000
RIP: 0010:[<ffffffffb4b16a7a>] [<ffffffffb4b16a7a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8c76edfcbc70 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00080000b4a7e2b0 RCX: ffff8c7effc60678
RDX: 0000000000000001 RSI: 0000000000000006 RDI: 0000000000000000
RBP: ffff8c76edfcbca8 R08: ffff8c742d600800 R09: ffffffffb4d81839
R10: ffff8c7effd1f160 R11: fffffb7a8251ac00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8c76edfcbc20 R15: 0000000000000001
FS: 00007f8f1361f700(0000) GS:ffff8c7effd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8f13609eb8 CR3: 000000003653e000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb4a7e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb4a7e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb4bec096>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb4bedb5f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb4bedc83>] tlb_finish_mmu+0x23/0x30
[<ffffffffb4bf098a>] zap_page_range+0xea/0x130
[<ffffffffb4bfad78>] ? do_mmap+0x378/0x530
[<ffffffffb4beb69d>] SyS_madvise+0x49d/0xac0
[<ffffffffb4bddffb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffb518dede>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ee 5e c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 e8 74 b5 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 bd 5e c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #75681
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff8bf16e4a
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8be7e868
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8be7e8db
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff8bfede86
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff8bfef94f
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8bfefa73
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffff8bff2f3a
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xea
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff8bfed42d
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x49d
|
|
9
|
|
vmlinux |
0xffffffff8bfde32b
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xeb
|
|
10
|
system_call_fastpath
|
vmlinux |
0xffffffff8c594f92
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [Worker-97:27210]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc nfsv3 nfs_acl nfs lockd ebtable_filter grace ebtables fscache ip6table_filter devlink ip6_tables iptable_filter mvfs(OE) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul vmw_balloon glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul drm crct10dif_common crc32c_intel ata_piix
nfit libata serio_raw mptspi libnvdimm scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks floppy fuse
CPU: 4 PID: 27210 Comm: Worker-97 Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9c4ccf1fd280 ti: ffff9c4b0c83c000 task.ti: ffff9c4b0c83c000
RIP: 0010:[<ffffffff8bf16e4a>] [<ffffffff8bf16e4a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9c4b0c83fc70 EFLAGS: 00000202
RAX: 0000000000000005 RBX: ffffffff8be7e6a0 RCX: ffff9c4dee760678
RDX: 0000000000000005 RSI: 0000000000000040 RDI: 0000000000000000
RBP: ffff9c4b0c83fca8 R08: ffff9c4b3a343800 R09: ffffffff8c186479
R10: ffff9c4dee71f160 R11: ffffe2b146631a00 R12: 0000000000000286
R13: ffff9c4b0c83fc20 R14: ffff9c4b38740000 R15: 0000000000000001
FS: 00007fa65fafe700(0000) GS:ffff9c4dee700000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000015cdd840e000 CR3: 0000000405532000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8be7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8be7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8bfede86>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8bfef94f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8bfefa73>] tlb_finish_mmu+0x23/0x30
[<ffffffff8bff2f3a>] zap_page_range+0xea/0x130
[<ffffffff8bfed42d>] SyS_madvise+0x49d/0xac0
[<ffffffff8bfde32b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff8c594f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 de 86 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 b5 8c f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ad 86 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #73650
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff91516e4a
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff9147e868
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9147e8db
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff915ede86
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff915ef94f
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffff915efa73
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffff915f2f3a
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffff915fcb7c
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x39c
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffff915ed42d
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x49d
|
|
10
|
|
vmlinux |
0xffffffff915de32b
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xeb
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffff91b94f92
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [systemd-journal:15084]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [java:15334]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev i2c_piix4 sg joydev pcspkr virtio_balloon parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi bochs_drm drm_kms_helper virtio_net net_failover failover syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm ahci ata_piix libahci crc32c_intel
libata serio_raw virtio_pci virtio_ring drm_panel_orientation_quirks virtio floppy fuse
CPU: 2 PID: 15334 Comm: java Tainted: G L ------------ 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.0-0-g63451fca13-prebuilt.qemu-project.org 04/01/2014
task: ffff8e565ff9b180 ti: ffff8e5538f9c000 task.ti: ffff8e5538f9c000
RIP: 0010:[<ffffffff91516e4a>] [<ffffffff91516e4a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8e5538f9fc70 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 000800009147e6a0 RCX: ffff8e5937ca0628
RDX: 0000000000000001 RSI: 0000000000000004 RDI: 0000000000000000
RBP: ffff8e5538f9fca8 R08: ffff8e567937a800 R09: ffffffff91786479
R10: ffff8e5937d1f160 R11: ffffe44449aa9c00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8e5538f9fc20 R15: 0000000000000001
FS: 00007f9d6f206700(0000) GS:ffff8e5937d00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9d6f1f1a18 CR3: 00000003f2a4a000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9147e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9147e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff915ede86>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff915ef94f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff915efa73>] tlb_finish_mmu+0x23/0x30
[<ffffffff915f2f3a>] zap_page_range+0xea/0x130
[<ffffffff915fcb7c>] ? do_mmap+0x39c/0x590
[<ffffffff915ed42d>] SyS_madvise+0x49d/0xac0
[<ffffffff915de32b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff91b94f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 de 86 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 15 92 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ad 86 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #56223
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff9bd16a6a
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff9bc7e478
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9bc7e4eb
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff9bdec026
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff9bdedaef
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffff9bdedc13
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffff9bdf091a
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xea
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff9bdeb62d
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x49d
|
|
9
|
|
vmlinux |
0xffffffff9bdddfdb
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xeb
|
|
10
|
tracesys
|
vmlinux |
0xffffffff9c38e172
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xa6
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [elasticsearch[c:59626]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter fuse vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac coretemp iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper snd_ens1371 vmw_balloon snd_rawmidi snd_ac97_codec nfit ablk_helper cryptd ac97_bus snd_seq libnvdimm snd_seq_device snd_pcm pcspkr snd_timer snd joydev soundcore vmw_vmci sg i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi libata scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common mptbase crc32c_intel e1000 drm_panel_orientation_quirks serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 59626 Comm: elasticsearch[c Tainted: G L ------------ 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff88a2cad30000 ti: ffff88a2ed32c000 task.ti: ffff88a2ed32c000
RIP: 0010:[<ffffffff9bd16a6a>] [<ffffffff9bd16a6a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff88a2ed32fc70 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800009bc7e2b0 RCX: ffff88a4f5620600
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff88a2ed32fca8 R08: ffff88a43c84bc00 R09: ffffffff9bf817e9
R10: ffff88a4f565f160 R11: ffffbbce02e23200 R12: 0000000000000282
R13: 0000000000000282 R14: ffff88a2ed32fc20 R15: 0000000000000001
FS: 00007f0694c6e700(0000) GS:ffff88a4f5640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055af9bb36acc CR3: 00000002217e8000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9bc7e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9bc7e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9bdec026>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9bdedaef>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9bdedc13>] tlb_finish_mmu+0x23/0x30
[<ffffffff9bdf091a>] zap_page_range+0xea/0x130
[<ffffffff9bdeb62d>] SyS_madvise+0x49d/0xac0
[<ffffffff9bdddfdb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff9c38e172>] tracesys+0xa6/0xcc
Code: 48 63 35 fe 5e c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 e8 94 9c f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 cd 5e c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #9570
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffb2716a6a
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb267e478
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb267e4eb
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffb27ec026
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffb27edaef
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb27edc13
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffb27f091a
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xea
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffffb27eb62d
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x49d
|
|
9
|
|
vmlinux |
0xffffffffb27ddfdb
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xeb
|
|
10
|
system_call_fastpath
|
vmlinux |
0xffffffffb2d8dede
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [RMI TCP Connect:17128]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter binfmt_misc sunrpc nfit libnvdimm ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper sg ablk_helper cryptd joydev pcspkr virtio_balloon parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom cirrus ata_generic drm_kms_helper pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_blk ata_piix crct10dif_pclmul drm 8139too libata crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw 8139cp floppy drm_panel_orientation_quirks
mii virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 17128 Comm: RMI TCP Connect Kdump: loaded Tainted: G L ------------ 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8b31ef1541c0 ti: ffff8b30cfdc0000 task.ti: ffff8b30cfdc0000
RIP: 0010:[<ffffffffb2716a6a>] [<ffffffffb2716a6a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8b30cfdc3c70 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00080000b267e2b0 RCX: ffff8b323a120320
RDX: 0000000000000002 RSI: 000000000000000a RDI: 0000000000000000
RBP: ffff8b30cfdc3ca8 R08: ffff8b3237ab9c00 R09: ffffffffb29817e9
R10: ffff8b323a09f160 R11: ffffd6dcc5d46a00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8b30cfdc3c20 R15: 0000000000000001
FS: 00007f2f0f4cc700(0000) GS:ffff8b323a080000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000013460001 CR3: 00000000a7120000 CR4: 00000000007606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb267e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb267e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb27ec026>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb27edaef>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb27edc13>] tlb_finish_mmu+0x23/0x30
[<ffffffffb27f091a>] zap_page_range+0xea/0x130
[<ffffffffb27eb62d>] SyS_madvise+0x49d/0xac0
[<ffffffffb27ddfdb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffb2d8dede>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 fe 5e c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 e8 34 b3 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 cd 5e c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1203125
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffbc7170da
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffbc67e258
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffbc67e2cb
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffbc7ee676
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffbc7f013f
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffbc7f0263
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffbc7f372a
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffbc712670
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x90
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffbc7edc1d
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x49d
|
|
10
|
system_call_fastpath
|
vmlinux |
0xffffffffbcd95f92
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [MallocTrimThrea:122844]
Modules linked in: veeamsnap(OE) tcp_lp tun bridge stp llc ebtable_filter ebtables xt_comment devlink ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter iptable_raw iptable_security xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq iosf_mbi crc32_pclmul ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw snd_timer gf128mul snd glue_helper ablk_helper cryptd soundcore ppdev vmw_balloon sg pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c
sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit ata_piix libnvdimm drm crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi mptscsih e1000 libata serio_raw mptbase drm_panel_orientation_quirks fuse
CPU: 1 PID: 122844 Comm: MallocTrimThrea Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9e707dd46300 ti: ffff9e71a565c000 task.ti: ffff9e71a565c000
RIP: 0010:[<ffffffffbc7170da>] [<ffffffffbc7170da>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9e71a565fc70 EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00080000bc67ddd0 RCX: ffff9e71b76e0600
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9e71a565fca8 R08: ffff9e717b88b000 R09: ffffffffbc986bd9
R10: ffff9e71b765f160 R11: ffffd93806d8f400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9e71a565fc20 R15: 0000000000000001
FS: 00007f0f88be4700(0000) GS:ffff9e71b7640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000071adef CR3: 00000001b55f4000 CR4: 00000000003607e0
Call Trace:
[<ffffffffbc67e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbc67e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbc7ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbc7f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbc7f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffbc7f372a>] zap_page_range+0xea/0x130
[<ffffffffbc712670>] ? futex_wake+0x90/0x180
[<ffffffffbc7edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffbcd95f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 35 bd f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1178591
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffa1d1e25e
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x20e
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa1c81fa8
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa1c8201b
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa1dfa0ce
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xae
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa1dfbc4f
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa1dfbd83
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffa1dff2ca
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffa1ce1118
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffffa1d196b0
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x90
|
|
10
|
SyS_madvise
|
vmlinux |
0xffffffffa1df96bd
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x49d
|
|
11
|
tracesys
|
vmlinux |
0xffffffffa23c562e
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xa6
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [s1-agent:1343]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter cts nfsv3 nfs_acl cmac arc4 md4 nls_utf8 rpcsec_gss_krb5 cifs nfsv4 nfs ccm dns_resolver lockd grace fscache vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ahci
fb_sys_fops ttm libahci drm sd_mod ata_piix nfit crc_t10dif crct10dif_generic libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 1343 Comm: s1-agent Kdump: loaded Tainted: G W L ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9e2928ad5280 ti: ffff9e264ceb8000 task.ti: ffff9e264ceb8000
RIP: 0010:[<ffffffffa1d1e25e>] [<ffffffffa1d1e25e>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff9e264cebbc70 EFLAGS: 00000202
RAX: 0000000000000005 RBX: 00080000a1c81b10 RCX: ffff9e297fd60650
RDX: 0000000000000005 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff9e264cebbca8 R08: ffff9e26b63c1000 R09: ffffffffa1f9c7f9
R10: ffff9e297fcdf160 R11: ffffe254d0ba2400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9e264cebbc20 R15: 0000000000000001
FS: 00007fa7a61b6700(0000) GS:ffff9e297fcc0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e3c5341620 CR3: 00000002a7278000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa1c81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa1c8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa1dfa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa1dfbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa1dfbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffa1dff2ca>] zap_page_range+0xea/0x140
[<ffffffffa1ce1118>] ? wake_up_q+0x58/0x90
[<ffffffffa1d196b0>] ? futex_wake+0x90/0x190
[<ffffffffa1df96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffa23c562e>] tracesys+0xa6/0xcc
Code: 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 95 a2 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #1005564
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffacd1e25a
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffacc81fa8
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffacc8201b
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffacdfa0ce
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xae
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffacdfbc4f
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffacdfbd83
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffacdff2ca
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffacd196b0
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x90
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffacdf96bd
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x49d
|
|
10
|
|
vmlinux |
0xffffffffacce5d58
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x78
|
|
11
|
|
vmlinux |
0xffffffffaccec6ab
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x8b
|
|
12
|
system_call_fastpath
|
vmlinux |
0xffffffffad3c539a
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#23 stuck for 23s! [bwa:15744]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ext4 mbcache jbd2 nfit libnvdimm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi
virtio_net net_failover virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix libahci drm crct10dif_pclmul virtio_pci crct10dif_common crc32c_intel libata virtio_ring serio_raw drm_panel_orientation_quirks virtio floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 23 PID: 15744 Comm: bwa Tainted: G L ------------ 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffffa0f5b400b180 ti: ffffa0f58ffa0000 task.ti: ffffa0f58ffa0000
RIP: 0010:[<ffffffffacd1e25a>] [<ffffffffacd1e25a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffffa0f58ffa3c70 EFLAGS: 00000202
RAX: 000000000000007f RBX: 00080000acc81b10 RCX: ffffa0f5bb5e0970
RDX: 000000000000007f RSI: 0000000000000080 RDI: 0000000000000040
RBP: ffffa0f58ffa3ca8 R08: ffffa01b22900808 R09: ffffffffacf9c7f9
R10: ffffa0f5b9bdf160 R11: ffffda6f3652ee00 R12: 0000000000000282
R13: 0000000000000018 R14: ffffa0f58ffa3be0 R15: 0000000000000282
FS: 00007f9874fe1700(0000) GS:ffffa0f5b9bc0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f83a42d6000 CR3: 000000c7e1406000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffacc81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffacc8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffacdfa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffacdfbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffacdfbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffacdff2ca>] zap_page_range+0xea/0x140
[<ffffffffacd196b0>] ? futex_wake+0x90/0x190
[<ffffffffacdf96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffacce5d58>] ? __enqueue_entity+0x78/0x80
[<ffffffffaccec6ab>] ? put_prev_entity+0x8b/0x400
[<ffffffffad3c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ae 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 95 ad f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1003448
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffb191e25e
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x20e
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb1881fa8
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb188201b
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffb19fa0ce
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xae
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffb19fbc4f
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb19fbd83
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffb19ff2ca
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffb18e1118
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffffb19196b0
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x90
|
|
10
|
SyS_madvise
|
vmlinux |
0xffffffffb19f96bd
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x49d
|
|
11
|
|
vmlinux |
0xffffffffb18e8699
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xa9
|
|
12
|
|
vmlinux |
0xffffffffb18e918c
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x3c
|
|
13
|
system_call_fastpath
|
vmlinux |
0xffffffffb1fc539a
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [containerd-shim:4762]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype br_netfilter bridge stp llc iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon sg pcspkr joydev vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm nfit sd_mod libahci ata_piix crc_t10dif crct10dif_generic libnvdimm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod fuse
CPU: 3 PID: 4762 Comm: containerd-shim Kdump: loaded Tainted: G L ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9d0fe4c09080 ti: ffff9d13eef80000 task.ti: ffff9d13eef80000
RIP: 0010:[<ffffffffb191e25e>] [<ffffffffb191e25e>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff9d13eef83c70 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00080000b1881b10 RCX: ffff9d147fca0650
RDX: 0000000000000002 RSI: 0000000000000006 RDI: 0000000000000000
RBP: ffff9d13eef83ca8 R08: ffff9d10b2bf6000 R09: ffffffffb1b9c7f9
R10: ffff9d147fcdf160 R11: ffffe2b6d21de200 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9d13eef83c20 R15: 0000000000000001
FS: 000000c000200090(0000) GS:ffff9d147fcc0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000156000 CR3: 00000004aa8c0000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb1881fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb188201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb19fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffb19fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb19fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffb19ff2ca>] zap_page_range+0xea/0x140
[<ffffffffb18e1118>] ? wake_up_q+0x58/0x90
[<ffffffffb19196b0>] ? futex_wake+0x90/0x190
[<ffffffffb19f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffb18e8699>] ? pick_next_entity+0xa9/0x190
[<ffffffffb18e918c>] ? set_next_entity+0x3c/0x100
[<ffffffffb1fc539a>] system_call_fastpath+0x25/0x2a
Code: 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 55 b2 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #997488
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffa931e23a
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa9281fa8
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa928201b
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa93fa07e
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0xae
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa93fbbff
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa93fbd33
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffa93ff27a
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffa940228a
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0xb3a
|
|
9
|
|
vmlinux |
0xffffffffa940926c
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x39c
|
|
10
|
SyS_madvise
|
vmlinux |
0xffffffffa93f966d
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x49d
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffffa99c539a
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 25s! [InactivityMonit:7913]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 auth_rpcgss binfmt_misc sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops sd_mod ttm crc_t10dif crct10dif_generic ahci drm ata_piix libahci
nfit libata crct10dif_pclmul crct10dif_common libnvdimm crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks fuse
CPU: 7 PID: 7913 Comm: InactivityMonit Kdump: loaded Tainted: G L ------------ 3.10.0-1160.108.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff97f95a6d0000 ti: ffff97f9c6f7c000 task.ti: ffff97f9c6f7c000
RIP: 0010:[<ffffffffa931e23a>] [<ffffffffa931e23a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff97f9c6f7fc70 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00080000a9281b10 RCX: ffff97f90da20750
RDX: 0000000000000000 RSI: 000000000000001e RDI: 0000000000000000
RBP: ffff97f9c6f7fca8 R08: ffff97ede0592000 R09: ffffffffa959c7b9
R10: ffff97f90dbdf160 R11: fffff947e3fbf800 R12: 0000000000000282
R13: 0000000000000282 R14: ffff97f9c6f7fc20 R15: 0000000000000001
FS: 00007fc04fafa700(0000) GS:ffff97f90dbc0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc1313062b0 CR3: 000000190c528000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa9281fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa928201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa93fa07e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa93fbbff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa93fbd33>] tlb_finish_mmu+0x23/0x40
[<ffffffffa93ff27a>] zap_page_range+0xea/0x140
[<ffffffffa940228a>] ? handle_mm_fault+0xb3a/0x1190
[<ffffffffa940926c>] ? do_mmap+0x39c/0x590
[<ffffffffa93f966d>] SyS_madvise+0x49d/0xac0
[<ffffffffa99c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 8e 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 80 19 f5 a9 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d 18 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #976538
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff9311e25a
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff93081fa8
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9308201b
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff931fa0ce
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xae
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff931fbc4f
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffff931fbd83
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffff931ff2ca
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffff930e1118
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff931196b0
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x90
|
|
10
|
SyS_madvise
|
vmlinux |
0xffffffff931f96bd
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x49d
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffff937c539a
Build id: 3.10.0-1160.118.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [mariadbd:25492]
Modules linked in: veeamsnap(OE) binfmt_misc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vfat fat sg pcspkr joydev virtio_balloon i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net net_failover virtio_console virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 25492 Comm: mariadbd Kdump: loaded Tainted: G OEL ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 3.20230228-4~bpo11+3 02/15/2024
task: ffff977ca7bec200 ti: ffff977ad6dd4000 task.ti: ffff977ad6dd4000
RIP: 0010:[<ffffffff9311e25a>] [<ffffffff9311e25a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff977ad6dd7c70 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 0008000093081b10 RCX: ffff977ccfc60428
RDX: 0000000000000001 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff977ad6dd7ca8 R08: ffff9777ef221400 R09: ffffffff9339c7f9
R10: ffff977ccfc9f160 R11: ffffde0c80f40600 R12: 0000000000000282
R13: 0000000000000282 R14: ffff977ad6dd7c20 R15: 0000000000000001
FS: 00007f97d93f4700(0000) GS:ffff977ccfc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056367dffcbb8 CR3: 00000005c1a74000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff93081fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9308201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff931fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff931fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff931fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffff931ff2ca>] zap_page_range+0xea/0x140
[<ffffffff930e1118>] ? wake_up_q+0x58/0x90
[<ffffffff931196b0>] ? futex_wake+0x90/0x190
[<ffffffff931f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffff937c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ae 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 d5 93 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #924807
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff8e11722a
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8e07e258
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8e07e2cb
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff8e1ee666
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff8e1f012f
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8e1f0253
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffff8e1f371a
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffff8e0daef8
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff8e1127c0
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x90
|
|
10
|
SyS_madvise
|
vmlinux |
0xffffffff8e1edc0d
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x49d
|
|
11
|
|
vmlinux |
0xffffffff8e0e2429
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xa9
|
|
12
|
ia32_do_call
|
vmlinux |
0xffffffff8e79e68e
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x4c
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 68s! [frpc:83226]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm snd_ens1371 irqbypass snd_rawmidi crc32_pclmul snd_ac97_codec ac97_bus ghash_clmulni_intel snd_seq aesni_intel lrw snd_seq_device gf128mul snd_pcm snd_timer ppdev glue_helper ablk_helper cryptd snd vmw_balloon soundcore joydev pcspkr sg parport_pc
parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ttm crct10dif_pclmul crct10dif_common scsi_transport_spi ata_piix crc32c_intel mptscsih drm nfit libata mptbase serio_raw libnvdimm e1000 drm_panel_orientation_quirks fuse
CPU: 6 PID: 83226 Comm: frpc Kdump: loaded Tainted: G W L ------------ 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8a820ad14200 ti: ffff8a81c15c0000 task.ti: ffff8a81c15c0000
RIP: 0010:[<ffffffff8e11722a>] [<ffffffff8e11722a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8a81c15c3c70 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000800008e07ddd0 RCX: ffff8a822d6a06c8
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8a81c15c3ca8 R08: ffff8a7f75ccc400 R09: ffffffff8e3869d9
R10: ffff8a822d79f160 R11: ffffd8a589457c00 R12: 0000000000000282
R13: 0000000000000000 R14: ffff8a81c15c3be0 R15: 0000000000000282
FS: 0000000000000000(0000) GS:ffff8a822d780000(0063) knlGS:0000000009442294
CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 000016bd7ca74000 CR3: 0000000073f1a000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8e07e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8e07e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8e1ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8e1f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8e1f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffff8e1f371a>] zap_page_range+0xea/0x130
[<ffffffff8e0daef8>] ? wake_up_q+0x58/0x80
[<ffffffff8e1127c0>] ? futex_wake+0x90/0x180
[<ffffffff8e1edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffff8e0e2429>] ? pick_next_entity+0xa9/0x190
[<ffffffff8e79e68e>] ia32_do_call+0x4c/0x4c
Code: 48 63 35 7e 87 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 18 d5 8e f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 4d 87 c4 00 eb b7 0f b6 4d cc 4c 89
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:33:plane-0] flip_done timed out
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:33:plane-0] flip_done timed out
Complete report #581873
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffffa8b16c0a
Build id: 3.10.0-1160.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa8a7e868
Build id: 3.10.0-1160.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa8a7e8db
Build id: 3.10.0-1160.el7.x86_64
|
0x6b
|
|
4
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa8bedd06
Build id: 3.10.0-1160.el7.x86_64
|
0xa6
|
|
5
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa8bef7cf
Build id: 3.10.0-1160.el7.x86_64
|
0x3f
|
|
6
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa8bef8f3
Build id: 3.10.0-1160.el7.x86_64
|
0x23
|
|
7
|
zap_page_range
|
vmlinux |
0xffffffffa8bf2dba
Build id: 3.10.0-1160.el7.x86_64
|
0xea
|
|
8
|
|
vmlinux |
0xffffffffa8b12040
Build id: 3.10.0-1160.el7.x86_64
|
0x90
|
|
9
|
SyS_madvise
|
vmlinux |
0xffffffffa8bed2ad
Build id: 3.10.0-1160.el7.x86_64
|
0x49d
|
|
10
|
system_call_fastpath
|
vmlinux |
0xffffffffa9193f92
Build id: 3.10.0-1160.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [server.rb:286:98299]
Modules linked in: tcp_lp veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi ppdev snd_ac97_codec ac97_bus aesni_intel
snd_seq lrw gf128mul snd_seq_device glue_helper ablk_helper vmw_balloon snd_pcm cryptd snd_timer joydev pcspkr snd soundcore i2c_piix4 sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ata_generic pata_acpi crct10dif_pclmul mptspi fb_sys_fops crct10dif_common ttm scsi_transport_spi drm ata_piix e1000 mptscsih crc32c_intel serio_raw mptbase nfit libnvdimm libata drm_panel_orientation_quirks fuse
CPU: 0 PID: 98299 Comm: server.rb:286 Kdump: loaded Tainted: G L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8895ba961080 ti: ffff8896c697c000 task.ti: ffff8896c697c000
RIP: 0010:[<ffffffffa8b16c0a>] [<ffffffffa8b16c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8896c697fc70 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000c0000a8a7e6a0 RCX: ffff8897af29fc10
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8896c697fca8 R08: ffff8896ffdd7000 R09: ffffffffa8d85f99
R10: ffff8897af21f160 R11: ffffe442c1312200 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8896c697fc20 R15: ffffffffa8a2b621
FS: 00007f1298177700(0000) GS:ffff8897af200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0e39bac000 CR3: 000000010f7a2000 CR4: 00000000003607f0
Call Trace:
[<ffffffffa8a7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa8a7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa8bedd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa8bef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa8bef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffa8bf2dba>] zap_page_range+0xea/0x130
[<ffffffffa8b12040>] ? futex_wake+0x90/0x180
[<ffffffffa8bed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffa9193f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 75 a9 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #586584
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8c87e868
Build id: 3.10.0-1160.el7.x86_64
|
0xb8
|
|
2
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8c87e8db
Build id: 3.10.0-1160.el7.x86_64
|
0x6b
|
|
3
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff8c9edd06
Build id: 3.10.0-1160.el7.x86_64
|
0xa6
|
|
4
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff8c9ef7cf
Build id: 3.10.0-1160.el7.x86_64
|
0x3f
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8c9ef8f3
Build id: 3.10.0-1160.el7.x86_64
|
0x23
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff8c9f2dba
Build id: 3.10.0-1160.el7.x86_64
|
0xea
|
|
7
|
|
vmlinux |
0xffffffff8c912040
Build id: 3.10.0-1160.el7.x86_64
|
0x90
|
|
8
|
SyS_madvise
|
vmlinux |
0xffffffff8c9ed2ad
Build id: 3.10.0-1160.el7.x86_64
|
0x49d
|
|
9
|
system_call_fastpath
|
vmlinux |
0xffffffff8cf93f92
Build id: 3.10.0-1160.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 100s! [vmtoolsd:683]
FS: 00007faa599a0700(0000) GS:ffff99a779640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fce5e86f150 CR3: 000000004671e000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8c87e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c87e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8c9edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8c9ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8c9ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffff8c9f2dba>] zap_page_range+0xea/0x130
[<ffffffff8c912040>] ? futex_wake+0x90/0x180
[<ffffffff8c9ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffff8cf93f92>] system_call_fastpath+0x25/0x2a