NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [JS Helper:97005]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel ac97_bus snd_seq vmw_balloon lrw gf128mul glue_helper ablk_helper snd_seq_device cryptd snd_pcm snd_timer joydev snd
sg pcspkr soundcore vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul drm crct10dif_common crc32c_intel ata_piix nfit mptspi scsi_transport_spi libata mptscsih libnvdimm e1000 mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 6 PID: 97005 Comm: JS Helper Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffffa01f25b54200 ti: ffffa01e879ec000 task.ti: ffffa01e879ec000
RIP: 0010:[<ffffffffb4116c0a>]  [<ffffffffb4116c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffffa01e879efc70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00080000b407e6a0 RCX: ffffa01f7d6206c8
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffffa01e879efca8 R08: ffffa01ebe34c400 R09: ffffffffb4385f99
R10: ffffa01f7d79f160 R11: ffffea6142d2e400 R12: 0000000000000282
R13: 0000000000000282 R14: ffffa01e879efc20 R15: 0000000000000001
FS:  00007fda39406700(0000) GS:ffffa01f7d780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fda294d2ff0 CR3: 000000003d08e000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb407e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb407e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb41edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb41ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb41ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffb41f2dba>] zap_page_range+0xea/0x130
[<ffffffffb40daaa8>] ? wake_up_q+0x58/0x80
[<ffffffffb4112040>] ? futex_wake+0x90/0x180
[<ffffffffb41ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffb4793f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 d5 b4 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [MallocTrimThrea:122844]
Modules linked in: veeamsnap(OE) tcp_lp tun bridge stp llc ebtable_filter ebtables xt_comment devlink ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter iptable_raw iptable_security xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq iosf_mbi crc32_pclmul ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw snd_timer gf128mul snd glue_helper ablk_helper cryptd soundcore ppdev vmw_balloon sg pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c
sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit ata_piix libnvdimm drm crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi mptscsih e1000 libata serio_raw mptbase drm_panel_orientation_quirks fuse
CPU: 1 PID: 122844 Comm: MallocTrimThrea Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9e707dd46300 ti: ffff9e71a565c000 task.ti: ffff9e71a565c000
RIP: 0010:[<ffffffffbc7170da>]  [<ffffffffbc7170da>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9e71a565fc70  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00080000bc67ddd0 RCX: ffff9e71b76e0600
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9e71a565fca8 R08: ffff9e717b88b000 R09: ffffffffbc986bd9
R10: ffff9e71b765f160 R11: ffffd93806d8f400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9e71a565fc20 R15: 0000000000000001
FS:  00007f0f88be4700(0000) GS:ffff9e71b7640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000071adef CR3: 00000001b55f4000 CR4: 00000000003607e0
Call Trace:
[<ffffffffbc67e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbc67e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbc7ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbc7f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbc7f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffbc7f372a>] zap_page_range+0xea/0x130
[<ffffffffbc712670>] ? futex_wake+0x90/0x180
[<ffffffffbc7edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffbcd95f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 35 bd f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [s1-agent:1343]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter cts nfsv3 nfs_acl cmac arc4 md4 nls_utf8 rpcsec_gss_krb5 cifs nfsv4 nfs ccm dns_resolver lockd grace fscache vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ahci
fb_sys_fops ttm libahci drm sd_mod ata_piix nfit crc_t10dif crct10dif_generic libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 1343 Comm: s1-agent Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9e2928ad5280 ti: ffff9e264ceb8000 task.ti: ffff9e264ceb8000
RIP: 0010:[<ffffffffa1d1e25e>]  [<ffffffffa1d1e25e>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff9e264cebbc70  EFLAGS: 00000202
RAX: 0000000000000005 RBX: 00080000a1c81b10 RCX: ffff9e297fd60650
RDX: 0000000000000005 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff9e264cebbca8 R08: ffff9e26b63c1000 R09: ffffffffa1f9c7f9
R10: ffff9e297fcdf160 R11: ffffe254d0ba2400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9e264cebbc20 R15: 0000000000000001
FS:  00007fa7a61b6700(0000) GS:ffff9e297fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e3c5341620 CR3: 00000002a7278000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa1c81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa1c8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa1dfa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa1dfbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa1dfbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffa1dff2ca>] zap_page_range+0xea/0x140
[<ffffffffa1ce1118>] ? wake_up_q+0x58/0x90
[<ffffffffa1d196b0>] ? futex_wake+0x90/0x190
[<ffffffffa1df96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffa23c562e>] tracesys+0xa6/0xcc
Code: 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 95 a2 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [containerd-shim:18386]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc cmac arc4 md4 nls_utf8 cifs ccm dns_resolver overlay(T) vmw_vsock_vmci_transport vsock vfat fat snd_hda_codec_generic snd_hda_intel snd_hda_codec sb_edac snd_hda_core iosf_mbi crc32_pclmul snd_hwdep ghash_clmulni_intel snd_seq aesni_intel snd_seq_device lrw gf128mul glue_helper snd_pcm ablk_helper ppdev cryptd snd_timer snd soundcore joydev vmw_balloon pcspkr sg i2c_piix4 vmw_vmci tpm_crb parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix sd_mod drm
crc_t10dif crct10dif_generic libata crct10dif_pclmul crct10dif_common nfit crc32c_intel serio_raw libnvdimm pcnet32 vmw_pvscsi mii drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 18386 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.81.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.21100432.B64.2301110304 01/11/2023
task: ffff8b56d3b86300 ti: ffff8b56d3b60000 task.ti: ffff8b56d3b60000
RIP: 0010:[<ffffffffa091e07a>]  [<ffffffffa091e07a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8b56d3b63c70  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00080000a0881ab0 RCX: ffff8b57756e0600
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8b56d3b63ca8 R08: ffff8b56ba863800 R09: ffffffffa0b9c139
R10: ffff8b577565f160 R11: ffffbc7445a90e00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8b56d3b63c20 R15: ffffffffa1a0db80
FS:  000000c00029c490(0000) GS:ffff8b5775640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0000bb010 CR3: 00000001c9e32000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa0881f48>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa0881fbb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa09f9f0e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa09fba4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa09fbb83>] tlb_finish_mmu+0x23/0x40
[<ffffffffa09ff0ca>] zap_page_range+0xea/0x140
[<ffffffffa08e0fc8>] ? wake_up_q+0x58/0x90
[<ffffffffa09f949d>] SyS_madvise+0x49d/0xac0
[<ffffffffa091c5e0>] ? SyS_futex+0x80/0x190
[<ffffffffa0fc539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ce 19 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 00 19 55 a1 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 19 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 21s! [containerd:18006]
Modules linked in: xt_owner xt_REDIRECT nf_nat_redirect xt_multiport ipt_rpfilter iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel rpcsec_gss_krb5 veth auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipportip ip_set_hash_ipport dummy nf_tables ip_set ip_vs_sh ip_vs_wrr ip_vs_rr ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_filter ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6_tables xt_mark xt_comment iptable_mangle xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype iptable_filter scini(POE) ip_vs nf_conntrack binfmt_misc vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon
sg parport_pc joydev parport pcspkr vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi scsi_transport_spi mptscsih drm libata crc32c_intel serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc overlay(T)
CPU: 5 PID: 18006 Comm: containerd Tainted: P        W  OEL ------------ T 3.10.0-1160.108.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffffa05ce4a90000 ti: ffffa05cc6328000 task.ti: ffffa05cc6328000
RIP: 0010:[<ffffffffb7d1e23a>]  [<ffffffffb7d1e23a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffffa05cc632bc70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00080000b7c81b10 RCX: ffffa05dfbc20750
RDX: 0000000000000000 RSI: 000000000000000c RDI: 0000000000000000
RBP: ffffa05cc632bca8 R08: ffffa05e7d4ef000 R09: ffffffffb7f9c7b9
R10: ffffa05efbc9f160 R11: fffff0f300105400 R12: 0000000000000282
R13: 0000000000000282 R14: ffffa05cc632bc20 R15: 0000000000000001
FS:  00007fb8c1e43700(0000) GS:ffffa05efbc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0037f8000 CR3: 00000000b6876000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb7c81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb7c8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb7dfa07e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffb7dfbbff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb7dfbd33>] tlb_finish_mmu+0x23/0x40
[<ffffffffb7dff27a>] zap_page_range+0xea/0x140
[<ffffffffb7ce1118>] ? wake_up_q+0x58/0x90
[<ffffffffb7df966d>] SyS_madvise+0x49d/0xac0
[<ffffffffb83c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 8e 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 80 19 95 b8 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d 18 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#23 stuck for 23s! [bwa:15744]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ext4 mbcache jbd2 nfit libnvdimm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi
virtio_net net_failover virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix libahci drm crct10dif_pclmul virtio_pci crct10dif_common crc32c_intel libata virtio_ring serio_raw drm_panel_orientation_quirks virtio floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 23 PID: 15744 Comm: bwa Tainted: G             L ------------   3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffffa0f5b400b180 ti: ffffa0f58ffa0000 task.ti: ffffa0f58ffa0000
RIP: 0010:[<ffffffffacd1e25a>]  [<ffffffffacd1e25a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffffa0f58ffa3c70  EFLAGS: 00000202
RAX: 000000000000007f RBX: 00080000acc81b10 RCX: ffffa0f5bb5e0970
RDX: 000000000000007f RSI: 0000000000000080 RDI: 0000000000000040
RBP: ffffa0f58ffa3ca8 R08: ffffa01b22900808 R09: ffffffffacf9c7f9
R10: ffffa0f5b9bdf160 R11: ffffda6f3652ee00 R12: 0000000000000282
R13: 0000000000000018 R14: ffffa0f58ffa3be0 R15: 0000000000000282
FS:  00007f9874fe1700(0000) GS:ffffa0f5b9bc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f83a42d6000 CR3: 000000c7e1406000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffacc81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffacc8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffacdfa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffacdfbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffacdfbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffacdff2ca>] zap_page_range+0xea/0x140
[<ffffffffacd196b0>] ? futex_wake+0x90/0x190
[<ffffffffacdf96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffacce5d58>] ? __enqueue_entity+0x78/0x80
[<ffffffffaccec6ab>] ? put_prev_entity+0x8b/0x400
[<ffffffffad3c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ae 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 95 ad f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [containerd-shim:4762]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype br_netfilter bridge stp llc iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon sg pcspkr joydev vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm nfit sd_mod libahci ata_piix crc_t10dif crct10dif_generic libnvdimm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod fuse
CPU: 3 PID: 4762 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9d0fe4c09080 ti: ffff9d13eef80000 task.ti: ffff9d13eef80000
RIP: 0010:[<ffffffffb191e25e>]  [<ffffffffb191e25e>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff9d13eef83c70  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00080000b1881b10 RCX: ffff9d147fca0650
RDX: 0000000000000002 RSI: 0000000000000006 RDI: 0000000000000000
RBP: ffff9d13eef83ca8 R08: ffff9d10b2bf6000 R09: ffffffffb1b9c7f9
R10: ffff9d147fcdf160 R11: ffffe2b6d21de200 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9d13eef83c20 R15: 0000000000000001
FS:  000000c000200090(0000) GS:ffff9d147fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000156000 CR3: 00000004aa8c0000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb1881fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb188201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb19fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffb19fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb19fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffb19ff2ca>] zap_page_range+0xea/0x140
[<ffffffffb18e1118>] ? wake_up_q+0x58/0x90
[<ffffffffb19196b0>] ? futex_wake+0x90/0x190
[<ffffffffb19f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffb18e8699>] ? pick_next_entity+0xa9/0x190
[<ffffffffb18e918c>] ? set_next_entity+0x3c/0x100
[<ffffffffb1fc539a>] system_call_fastpath+0x25/0x2a
Code: 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 55 b2 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [mariadbd:25492]
Modules linked in: veeamsnap(OE) binfmt_misc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vfat fat sg pcspkr joydev virtio_balloon i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net net_failover virtio_console virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 25492 Comm: mariadbd Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 3.20230228-4~bpo11+3 02/15/2024
task: ffff977ca7bec200 ti: ffff977ad6dd4000 task.ti: ffff977ad6dd4000
RIP: 0010:[<ffffffff9311e25a>]  [<ffffffff9311e25a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff977ad6dd7c70  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 0008000093081b10 RCX: ffff977ccfc60428
RDX: 0000000000000001 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff977ad6dd7ca8 R08: ffff9777ef221400 R09: ffffffff9339c7f9
R10: ffff977ccfc9f160 R11: ffffde0c80f40600 R12: 0000000000000282
R13: 0000000000000282 R14: ffff977ad6dd7c20 R15: 0000000000000001
FS:  00007f97d93f4700(0000) GS:ffff977ccfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056367dffcbb8 CR3: 00000005c1a74000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff93081fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9308201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff931fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff931fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff931fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffff931ff2ca>] zap_page_range+0xea/0x140
[<ffffffff930e1118>] ? wake_up_q+0x58/0x90
[<ffffffff931196b0>] ? futex_wake+0x90/0x190
[<ffffffff931f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffff937c539a>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 ae 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 19 d5 93 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 7d 18 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [s1-perf:21699]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype br_netfilter bridge stp llc overlay(T) xt_owner ebtable_filter ebtables devlink ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect sysimgblt fb_sys_fops crc_t10dif ttm crct10dif_generic ata_piix drm nfit libata crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm serio_raw
vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 7 PID: 21699 Comm: s1-perf Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.92.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff99e0d7b49080 ti: ffff99e0f8a18000 task.ti: ffff99e0f8a18000
RIP: 0010:[<ffffffff9051e1ce>]  [<ffffffff9051e1ce>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff99e0f8a1bc70  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0008000090481b10 RCX: ffff99f0ffc206f0
RDX: 0000000000000000 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff99e0f8a1bca8 R08: ffff99e21955c000 R09: ffffffff9079c669
R10: ffff99f0ffddf160 R11: fffffc74801c8400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff99e0f8a1bc20 R15: 0000000000000001
FS:  00007ff55f7fe700(0000) GS:ffff99f0ffdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbb10448d70 CR3: 000000023f0a4000 CR4: 00000000001607e0
Call Trace:
[<ffffffff90481fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9048201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff905fa00e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff905fbb8f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff905fbcc3>] tlb_finish_mmu+0x23/0x40
[<ffffffff905ff20a>] zap_page_range+0xea/0x140
[<ffffffff904e1118>] ? wake_up_q+0x58/0x90
[<ffffffff905f95fd>] SyS_madvise+0x49d/0xac0
[<ffffffff90bc562e>] tracesys+0xa6/0xcc
Code: 18 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 00 19 15 91 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 4d 18 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 

NMI watchdog: BUG: soft lockup - CPU#6 stuck for 71s! [frpc:83226]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm snd_ens1371 irqbypass snd_rawmidi crc32_pclmul snd_ac97_codec ac97_bus ghash_clmulni_intel snd_seq aesni_intel lrw snd_seq_device gf128mul snd_pcm snd_timer ppdev glue_helper ablk_helper cryptd snd vmw_balloon soundcore joydev pcspkr sg parport_pc
parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ttm crct10dif_pclmul crct10dif_common scsi_transport_spi ata_piix crc32c_intel mptscsih drm nfit libata mptbase serio_raw libnvdimm e1000 drm_panel_orientation_quirks fuse
CPU: 6 PID: 83226 Comm: frpc Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8a820ad14200 ti: ffff8a81c15c0000 task.ti: ffff8a81c15c0000
RIP: 0010:[<ffffffff8e11722a>]  [<ffffffff8e11722a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8a81c15c3c70  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000800008e07ddd0 RCX: ffff8a822d6a06c8
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8a81c15c3ca8 R08: ffff8a7f75ccc400 R09: ffffffff8e3869d9
R10: ffff8a822d79f160 R11: ffffd8a589457c00 R12: 0000000000000282
R13: 0000000000000000 R14: ffff8a81c15c3be0 R15: 0000000000000282
FS:  0000000000000000(0000) GS:ffff8a822d780000(0063) knlGS:0000000009442294
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 000016bd7ca74000 CR3: 0000000073f1a000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8e07e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8e07e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8e1ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8e1f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8e1f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffff8e1f371a>] zap_page_range+0xea/0x130
[<ffffffff8e0daef8>] ? wake_up_q+0x58/0x80

NMI watchdog: BUG: soft lockup - CPU#6 stuck for 68s! [frpc:83226]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm snd_ens1371 irqbypass snd_rawmidi crc32_pclmul snd_ac97_codec ac97_bus ghash_clmulni_intel snd_seq aesni_intel lrw snd_seq_device gf128mul snd_pcm snd_timer ppdev glue_helper ablk_helper cryptd snd vmw_balloon soundcore joydev pcspkr sg parport_pc
parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ttm crct10dif_pclmul crct10dif_common scsi_transport_spi ata_piix crc32c_intel mptscsih drm nfit libata mptbase serio_raw libnvdimm e1000 drm_panel_orientation_quirks fuse
CPU: 6 PID: 83226 Comm: frpc Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8a820ad14200 ti: ffff8a81c15c0000 task.ti: ffff8a81c15c0000
RIP: 0010:[<ffffffff8e11722a>]  [<ffffffff8e11722a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff8a81c15c3c70  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000800008e07ddd0 RCX: ffff8a822d6a06c8
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8a81c15c3ca8 R08: ffff8a7f75ccc400 R09: ffffffff8e3869d9
R10: ffff8a822d79f160 R11: ffffd8a589457c00 R12: 0000000000000282
R13: 0000000000000000 R14: ffff8a81c15c3be0 R15: 0000000000000282
FS:  0000000000000000(0000) GS:ffff8a822d780000(0063) knlGS:0000000009442294
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 000016bd7ca74000 CR3: 0000000073f1a000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8e07e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8e07e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8e1ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8e1f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8e1f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffff8e1f371a>] zap_page_range+0xea/0x130
[<ffffffff8e0daef8>] ? wake_up_q+0x58/0x80
[<ffffffff8e1127c0>] ? futex_wake+0x90/0x180
[<ffffffff8e1edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffff8e0e2429>] ? pick_next_entity+0xa9/0x190
[<ffffffff8e79e68e>] ia32_do_call+0x4c/0x4c
Code: 48 63 35 7e 87 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 18 d5 8e f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 4d 87 c4 00 eb b7 0f b6 4d cc 4c 89 
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:33:plane-0] flip_done timed out
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:33:plane-0] flip_done timed out

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 100s! [vmtoolsd:683]
FS:  00007faa599a0700(0000) GS:ffff99a779640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fce5e86f150 CR3: 000000004671e000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8c87e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c87e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8c9edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8c9ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8c9ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffff8c9f2dba>] zap_page_range+0xea/0x130
[<ffffffff8c912040>] ? futex_wake+0x90/0x180
[<ffffffff8c9ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffff8cf93f92>] system_call_fastpath+0x25/0x2a

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [server.rb:286:98299]
Modules linked in: tcp_lp veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi ppdev snd_ac97_codec ac97_bus aesni_intel
snd_seq lrw gf128mul snd_seq_device glue_helper ablk_helper vmw_balloon snd_pcm cryptd snd_timer joydev pcspkr snd soundcore i2c_piix4 sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ata_generic pata_acpi crct10dif_pclmul mptspi fb_sys_fops crct10dif_common ttm scsi_transport_spi drm ata_piix e1000 mptscsih crc32c_intel serio_raw mptbase nfit libnvdimm libata drm_panel_orientation_quirks fuse
CPU: 0 PID: 98299 Comm: server.rb:286 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8895ba961080 ti: ffff8896c697c000 task.ti: ffff8896c697c000
RIP: 0010:[<ffffffffa8b16c0a>]  [<ffffffffa8b16c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8896c697fc70  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000c0000a8a7e6a0 RCX: ffff8897af29fc10
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8896c697fca8 R08: ffff8896ffdd7000 R09: ffffffffa8d85f99
R10: ffff8897af21f160 R11: ffffe442c1312200 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8896c697fc20 R15: ffffffffa8a2b621
FS:  00007f1298177700(0000) GS:ffff8897af200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0e39bac000 CR3: 000000010f7a2000 CR4: 00000000003607f0
Call Trace:
[<ffffffffa8a7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa8a7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa8bedd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa8bef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa8bef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffa8bf2dba>] zap_page_range+0xea/0x130
[<ffffffffa8b12040>] ? futex_wake+0x90/0x180
[<ffffffffa8bed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffa9193f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 75 a9 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [zabbix_agent2:54908]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag nf_conntrack_netlink xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ipmi_devintf ipmi_msghandler nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod squashfs loop iosf_mbi crc32_pclmul
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon joydev sg pcspkr vmw_vmci i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi ata_piix mptscsih crct10dif_pclmul crct10dif_common nfit drm mptbase libata crc32c_intel libnvdimm serio_raw e1000 drm_panel_orientation_quirks fuse
CPU: 2 PID: 54908 Comm: zabbix_agent2 Not tainted 3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff92aaecf50000 ti: ffff92a7759bc000 task.ti: ffff92a7759bc000
RIP: 0010:[<ffffffff827170de>]  [<ffffffff827170de>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff92a7759bfc70  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 000800008267ddd0 RCX: ffff92aaed6e0628
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff92a7759bfca8 R08: ffff92a835ccb400 R09: ffffffff82986bf9
R10: ffff92aaed69f160 R11: ffffe6df1023a400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff92a7759bfc20 R15: 0000000000000001
FS:  00007f67467fc700(0000) GS:ffff92aaed680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001089987 CR3: 00000003d23f0000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8267e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8267e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff827ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff827f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff827f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff827f372a>] zap_page_range+0xea/0x130
[<ffffffff826dada8>] ? wake_up_q+0x58/0x80
[<ffffffff827edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff826dadf0>] ? wake_up_state+0x20/0x20
[<ffffffff82d95f92>] system_call_fastpath+0x25/0x2a
Code: 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 35 83 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [n9e-agent:129795]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi sb_edac snd_ac97_codec coretemp ac97_bus snd_seq snd_seq_device iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw snd_pcm gf128mul glue_helper ablk_helper ppdev snd_timer cryptd
vmw_balloon snd soundcore pcspkr joydev vmw_vmci sg i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata mptspi scsi_transport_spi e1000 mptscsih serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 129795 Comm: n9e-agent Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9af4f4fcb180 ti: ffff9b0184f74000 task.ti: ffff9b0184f74000
RIP: 0010:[<ffffffffbcb16c0a>]  [<ffffffffbcb16c0a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9b0184f77c70  EFLAGS: 00000202
RAX: 000000000000000f RBX: 00080000bca7e6a0 RCX: ffff9b037d9e0678
RDX: 000000000000000f RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9b0184f77ca8 R08: ffff9af4e4345000 R09: ffffffffbcd85f99
R10: ffff9b037d71f160 R11: ffffda7dc5f7d400 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9b0184f77c20 R15: 0000000000000001
FS:  00007f09cb7fe700(0000) GS:ffff9b037d700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6081c90158 CR3: 00000001d5eee000 CR4: 00000000001607e0
Call Trace:
[<ffffffffbca7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbca7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbcbedd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbcbef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbcbef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffbcbf2dba>] zap_page_range+0xea/0x130
[<ffffffffbcadaaa8>] ? wake_up_q+0x58/0x80
[<ffffffffbcbed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffbd193f92>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 1e 89 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 75 bd f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ed 88 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 43s! [prometheus:78719]
Modules linked in: tcp_lp fuse xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipportip ip_set_hash_ipport ip_set ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs dummy iptable_raw xt_CT ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc nfit libnvdimm nls_utf8 isofs ppdev coretemp vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel
aesni_intel lrw gf128mul sg pcspkr glue_helper ablk_helper cryptd joydev parport_pc parport vmw_vmci pcc_cpufreq i2c_piix4 br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi drm scsi_transport_spi ata_piix mptscsih crct10dif_pclmul crct10dif_common crc32c_intel libata mptbase serio_raw vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 78719 Comm: prometheus Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9495d1f48000 ti: ffff948bfcf24000 task.ti: ffff948bfcf24000
RIP: 0010:[<ffffffffb6b150ca>]  [<ffffffffb6b150ca>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff948bfcf27c70  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00080000b6a7d1d0 RCX: ffff94983d6dfc10
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff948bfcf27ca8 R08: ffff9489bfdd7000 R09: ffffffffb6d7f629
R10: ffffd55535e23fa0 R11: ffffe954b875ec00 R12: 0000000000000282
R13: 0000000000000282 R14: ffff948bfcf27c20 R15: 0000000000000001
FS:  000000c00f694410(0000) GS:ffff94983d600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0223a7000 CR3: 0000000e76a3a000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb6a7d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb6a7d40b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb6bea166>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb6bebc2f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb6bebd53>] tlb_finish_mmu+0x23/0x30
[<ffffffffb6beea5a>] zap_page_range+0xea/0x130
[<ffffffffb6ada068>] ? wake_up_q+0x58/0x80
[<ffffffffb6be976d>] SyS_madvise+0x49d/0xac0
[<ffffffffb718bede>] system_call_fastpath+0x25/0x2a
Code: 48 63 35 3e 68 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 d7 74 b7 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 0d 68 c4 00 eb b7 0f b6 4d cc 4c 89