NMI watchdog: BUG: soft lockup - CPU#3 stuck for 27s! [pilot-agent:28629]
Modules linked in: xt_owner xt_REDIRECT nf_nat_redirect xt_multiport ipt_rpfilter iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel rpcsec_gss_krb5 veth auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipportip ip_set_hash_ipport dummy nf_tables ip_set ip_vs_sh ip_vs_wrr ip_vs_rr ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_filter ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6_tables xt_mark xt_comment iptable_mangle xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype iptable_filter scini(POE) ip_vs nf_conntrack binfmt_misc vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon
sg parport_pc joydev parport pcspkr vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi scsi_transport_spi mptscsih drm libata crc32c_intel serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc overlay(T)
CPU: 3 PID: 28629 Comm: pilot-agent Tainted: P        W  OEL ------------ T 3.10.0-1160.108.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffffa05df5de2100 ti: ffffa05d26998000 task.ti: ffffa05d26998000
RIP: 0010:[<ffffffffb83ba720>]  [<ffffffffb83ba720>] _raw_spin_lock+0x10/0x30
RSP: 0018:ffffa05dfbd83e20  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 0000000000000005 RCX: ffffd2c3bf9b1500
RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffffa05dc2e330c8
RBP: ffffa05dfbd83e60 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000005 R12: ffffa05dfbd83d98
R13: ffffffffb83c63f2 R14: ffffa05dfbd83e60 R15: 00003263bfc31500
FS:  000000c00007c890(0000) GS:ffffa05dfbd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000b61000 CR3: 000000011e918000 CR4: 00000000000007e0
Call Trace:

watchdog: BUG: soft lockup - CPU#11 stuck for 29s! [swapper/11:0]
 cpu_startup_entry+0x19/0x20
Modules linked in:
CPU: 11 PID: 0 Comm: swapper/11 Tainted: G             L     6.1.10-100.fc36.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
RIP: 0010:__do_softirq+0x92/0x31b
Code: df e8 72 df 14 ff c7 44 24 10 0a 00 00 00 48 c7 c7 eb cd 7e 8a e8 1e e6 dd ff 65 66 c7 05 f4 11 03 76 00 00 fb 0f 1f 44 00 00 <bb> ff ff ff ff 41 0f bc dc 83 c3 01 0f 84 91 00 00 00 49 c7 c2 c0
RSP: 0000:ffff9c41802e0f90 EFLAGS: 00000292
RAX: 000000000000000b RBX: ffff90294094a880 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffffffff8a7ecdeb RDI: ffffffff8a767116
RBP: 0000000000000001 R08: 0000000000000000 R09: 000000002aaa891d
R10: 0000000000000000 R11: ffff9c41802e0ff8 R12: 0000000000000080
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff903200ac0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000356010001 CR4: 00000000000706e0
Call Trace:

BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
Modules linked in: binfmt_misc tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables ppp_deflate zlib_deflate bsd_comp ppp_async ppp_generic slhc dahdi(OF) crc_ccitt netatop(OF) xt_CHECKSUM iptable_mangle xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_nat_ipv4 nf_nat nf_log_ipv4 nf_log_common xt_LOG ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_recent xt_multiport xt_mark xt_conntrack nf_conntrack iptable_filter coretemp ext4 mbcache jbd2 snd_ens1371 snd_rawmidi snd_ac97_codec crc32_pclmul ac97_bus snd_seq ghash_clmulni_intel snd_seq_device snd_pcm aesni_intel ppdev lrw gf128mul snd_timer glue_helper vmw_balloon ablk_helper cryptd snd ahci pcspkr soundcore libahci sg vmw_vmci i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd sunrpc
ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel ttm mptspi e1000 serio_raw scsi_transport_spi ata_piix drm mptscsih libata mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 0 Comm: swapper/1 Tainted: GF       W  O--------------   3.10.0-229.14.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff88032cce16c0 ti: ffff88032ccf0000 task.ti: ffff88032ccf0000
RIP: 0010:[<ffffffff8160b722>]  [<ffffffff8160b722>] _raw_spin_lock+0x32/0x50
RSP: 0018:ffff880332623de0  EFLAGS: 00000202
RAX: 0000000000007ca0 RBX: ffff880332623e10 RCX: 000000000000ca8a
RDX: 000000000000ca8c RSI: 000000000000ca8c RDI: ffff88003568ea80
RBP: ffff880332623de0 R08: 0001cfaa64c913c0 R09: ffff88032c80c000
R10: 0000000000000002 R11: 0000000000000005 R12: ffff880332623d58
R13: ffffffff81614fdd R14: ffff880332623de0 R15: ffff88003568ea00
FS:  0000000000000000(0000) GS:ffff880332620000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000164f98c CR3: 0000000068286000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880332623e20 ffffffff8151d592 ffff88003542a320 ffff88032c80c000
ffff88003542a348 0000000000000100 ffffffff8151d520 ffff88003542a000
ffff880332623e58 ffffffff8107df66 ffff88032c80c000 00000000000000b0
Call Trace:

WARNING: CPU: 11 PID: 72 at net/mac80211/rx.c:822 ieee80211_rx_list+0x50a/0xc80 [mac80211]
CPU: 11 PID: 72 Comm: ksoftirqd/11 Tainted: G        W          6.1.8-200.fc37.x86_64 #1
Modules linked in: nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink bnep qrtr btusb btrtl btbcm uvcvideo btintel btmtk videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 bluetooth videobuf2_common videodev mc sunrpc vfat fat snd_ctl_led intel_rapl_msr intel_rapl_common snd_soc_dmic snd_acp3x_rn snd_acp3x_pdm_dma snd_sof_amd_rembrandt rtw88_8822ce snd_sof_amd_renoir snd_sof_amd_acp edac_mce_amd rtw88_8822c snd_sof_pci snd_sof rtw88_pci kvm_amd rtw88_core snd_sof_utils snd_hda_codec_conexant snd_hda_codec_generic kvm snd_hda_codec_hdmi snd_soc_core snd_hda_intel mac80211 snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec irqbypass snd_compress ac97_bus rapl snd_hda_core snd_pcm_dmaengine snd_pci_ps libarc4 snd_hwdep snd_rpl_pci_acp6x pcspkr snd_seq snd_pci_acp6x
CPU: 11 PID: 72 Comm: ksoftirqd/11 Tainted: G        W          6.1.8-200.fc37.x86_64 #1
RDX: 0000000000000000 RSI: ffff9dd3242f08e0 RDI: ffff9dd3242f08e0
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
CPU: 11 PID: 72 Comm: ksoftirqd/11 Tainted: G        W          6.1.8-200.fc37.x86_64 #1
RDX: 0000000000000000 RSI: ffff9dd3242f08e0 RDI: ffff9dd3242f08e0
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
CPU: 11 PID: 72 Comm: ksoftirqd/11 Tainted: G        W          6.1.8-200.fc37.x86_64 #1
RDX: 0000000000000000 RSI: ffff9dd3242f08e0 RDI: ffff9dd3242f08e0
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
CPU: 11 PID: 72 Comm: ksoftirqd/11 Tainted: G        W          6.1.8-200.fc37.x86_64 #1
RDX: 0000000000000000 RSI: ffff9dd3242f08e0 RDI: ffff9dd3242f08e0
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
CPU: 11 PID: 72 Comm: ksoftirqd/11 Tainted: G        W          6.1.8-200.fc37.x86_64 #1
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
R10: 0000000000000000 R11: ffffffffb8120070 R12: 0000000000000000
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
CPU: 11 PID: 72 Comm: ksoftirqd/11 Tainted: G        W          6.1.8-200.fc37.x86_64 #1
R13: 0000000000000001 R14: ffff9dd308285c58 R15: ffffc0c9c03dfc88
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
R13: 0000000000000001 R14: ffff9dd308285c58 R15: ffffc0c9c03dfc88
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
RIP: 0010:ieee80211_rx_list+0x50a/0xc80 [mac80211]
FS:  0000000000000000(0000) GS:ffff9dd60e0c0000(0000) knlGS:0000000000000000
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
RIP: 0010:ieee80211_rx_list+0x50a/0xc80 [mac80211]
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
Code: 85 63 fd ff ff 48 8b 0c 24 48 8b 49 58 83 e1 02 0f 84 08 01 00 00 41 8d 48 04 c7 44 24 14 04 00 00 00 39 d1 0f 82 fc 00 00 00 <0f> 0b 48 89 df eb 66 80 3d 44 44 0c 00 00 0f 85 c1 fb ff ff 0f b6
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
RSP: 0018:ffffc0c9c03dfba0 EFLAGS: 00010202
CR2: 000056476a145e40 CR3: 00000002e1010000 CR4: 0000000000750ee0
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
RSP: 0018:ffffc0c9c03dfba0 EFLAGS: 00010202
 ? __kmem_cache_alloc_node+0x14f/0x2b0
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
 snd_seq_device huawei_wmi snd_pci_acp5x snd_pcm cfg80211 ledtrig_audio snd_rn_pci_acp3x sparse_keymap wmi_bmof snd_timer snd_acp_config joydev k10temp snd snd_soc_acpi cm32181 rfkill soundcore snd_pci_acp3x i2c_piix4 industrialio amd_pmc acpi_cpufreq zram amdgpu drm_ttm_helper ttm iommu_v2 nvme gpu_sched drm_buddy nvme_core drm_display_helper video hid_multitouch crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 serio_raw ccp sp5100_tco cec nvme_common wmi i2c_hid_acpi i2c_hid ip6_tables ip_tables fuse
Call Trace:

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [Wrapper-Control:2251]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vboxsf(OE) sunrpc snd_intel8x0 nfit snd_ac97_codec ac97_bus libnvdimm snd_seq joydev snd_seq_device ppdev snd_pcm iosf_mbi vboxguest(OE) sg pcspkr snd_timer parport_pc video parport snd soundcore i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix libahci e1000 libata crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod fuse
CPU: 0 PID: 2251 Comm: Wrapper-Control Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-1160.80.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8e37efe38000 ti: ffff8e37efe34000 task.ti: ffff8e37efe34000
RIP: 0010:[<ffffffff94fba71c>]  [<ffffffff94fba71c>] _raw_spin_lock+0xc/0x30
RSP: 0000:ffff8e381fc03d88  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8e381fc03d10 RCX: 00000000314fe5a8
RDX: 0000000000000001 RSI: 0000000000000031 RDI: ffffd360bfc050d8
RBP: ffff8e381fc03da0 R08: 000000007664ac48 R09: 0000000041338eea
R10: ffff8e380afbe570 R11: 0000000083928143 R12: ffff8e381fc03cf8
R13: ffffffff94fc63f2 R14: ffff8e381fc03da0 R15: ffffd360bfc050d8
FS:  00007f7cc6d7c700(0000) GS:ffff8e381fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7cb119ffe8 CR3: 000000040ce3e000 CR4: 00000000000206f0
Call Trace:

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [systemd-journal:18298]
Modules linked in: fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic iosf_mbi crc32_pclmul snd_hda_intel ghash_clmulni_intel aesni_intel snd_hda_codec lrw gf128mul glue_helper ablk_helper snd_hda_core snd_hwdep snd_seq ppdev snd_seq_device cryptd snd_pcm snd_timer snd pcspkr parport_pc virtio_balloon parport sg i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect
crc32c_intel sysimgblt ata_piix fb_sys_fops serio_raw ttm drm virtio_pci virtio_ring virtio libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 18298 Comm: systemd-journal Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff880227290fb0 ti: ffff88000672c000 task.ti: ffff88000672c000
RIP: 0010:[<ffffffff8168d8ea>]  [<ffffffff8168d8ea>] _raw_spin_lock+0xa/0x50
RSP: 0000:ffff880239c03d98  EFLAGS: 00010282
RAX: 0000000000020000 RBX: ffff88021f687400 RCX: ffffffff81aa2528
RDX: 0000000000005cc3 RSI: 00000000e44ee44c RDI: ffffc90000d50210
RBP: ffff880239c03dc0 R08: ffffffff81e38ee8 R09: ffffffff81e380c0
R10: 0000000000000000 R11: ffff880239c03da0 R12: ffff880239c03d08
R13: ffffffff8169721d R14: ffff880239c03dc0 R15: ffff88021f687400
FS:  00007f0cea1e2880(0000) GS:ffff880239c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000a4004000 CR3: 000000000394b000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff815bb198 0000000000000001 ffff88021f687400 ffffffff81aa2340
ffffffff81aa24a0 ffff880239c03e00 ffffffff815bb2ec 0000000000000000
ffffffff81aa2340 ffffffff81aa24a0 0000000000000100 ffffffff815bb350
Call Trace:

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
[<ffffffffa5d1b750>] _raw_spin_lock+0x20/0x30
[<ffffffffa569df25>] irq_exit+0x105/0x110

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:2H:22966]
Modules linked in: xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache overlay(T) sunrpc vfat fat dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) nvidia(POE) snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_codec_generic snd_hda_intel snd_hda_codec edac_mce_amd kvm_amd kvm snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm irqbypass snd_timer snd crc32_pclmul soundcore
drm_kms_helper ghash_clmulni_intel syscopyarea sysfillrect sysimgblt fb_sys_fops aesni_intel sp5100_tco lrw gf128mul drm glue_helper ablk_helper cryptd pcspkr drm_panel_orientation_quirks ipmi_devintf sg ipmi_msghandler tpm_infineon fam15h_power k10temp i2c_piix4 acpi_cpufreq ip_tables ext4 mbcache jbd2 raid1 sd_mod crc_t10dif crct10dif_generic mxm_wmi ahci libahci libata crct10dif_pclmul crct10dif_common firewire_ohci crc32c_intel r8169 firewire_core serio_raw crc_itu_t wmi fuse [last unloaded: hwmon_vid]
CPU: 0 PID: 22966 Comm: kworker/0:2H Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./990FXA-UD3, BIOS F2 07/15/2013
Workqueue: xprtiod xs_tcp_data_receive_workfn [sunrpc]
task: ffff9cfe86bc0000 ti: ffff9d01cadb4000 task.ti: ffff9d01cadb4000
RIP: 0010:[<ffffffff941896e0>]  [<ffffffff941896e0>] _raw_spin_lock+0x10/0x30
RSP: 0018:ffff9d057ec03c68  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9cfeab3cc210 RCX: ffff9cfebe74b28e
RDX: 0000000000000001 RSI: ffff9cfea77c7a00 RDI: ffff9cfea9bd2170
RBP: ffff9d057ec03d08 R08: ffff9d057ec03d40 R09: 0000000000000000
R10: ffff9cfeae3ea4b8 R11: ffffffffc00ff5f8 R12: ffff9d057ec03bd8
R13: ffffffff94194fba R14: ffff9d057ec03d08 R15: ffff9cfea77c7a00
FS:  000000c000480090(0000) GS:ffff9d057ec00000(0000) knlGS:00000000089b36f0
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000c000167010 CR3: 00000007c4410000 CR4: 00000000000407f0
Call Trace:

NMI watchdog: BUG: soft lockup - CPU#14 stuck for 23s! [node:20206]
Modules linked in: xt_REDIRECT nf_nat_redirect ip_vs_rr xt_ipvs ip_vs xt_nat veth vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_nat_ipv4 nf_nat br_netfilter bridge stp llc overlay(T) ipmi_devintf gpio_ich iTCO_wdt iTCO_vendor_support dcdbas intel_powerclamp coretemp kvm irqbypass ipmi_ssif pcspkr sg ipmi_si ipmi_msghandler lpc_ich i7core_edac edac_core shpchp wmi acpi_power_meter nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crc32c_intel drm serio_raw
mptsas libata scsi_transport_sas mptscsih i2c_core mptbase bnx2 fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 14 PID: 20206 Comm: node Tainted: G        W I  L ------------ T 3.10.0-514.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R410/0WWR83, BIOS 1.11.0 07/20/2012
task: ffff880407a10fb0 ti: ffff880503a30000 task.ti: ffff880503a30000
RIP: 0010:[<ffffffff8168d910>]  [<ffffffff8168d910>] _raw_spin_lock+0x30/0x50
RSP: 0000:ffff88083f7c3df8  EFLAGS: 00000202
RAX: 000000000000247d RBX: ffff88083f7c3e18 RCX: 0000000000000068
RDX: 000000000000006a RSI: 000000000000006a RDI: ffff8801946b74b0
RBP: ffff88083f7c3df8 R08: 0002b3d9991533c0 R09: ffff8804c1c74000
R10: 0000000000000000 R11: ffff88083f7c3da0 R12: ffff88083f7c3d68
R13: ffffffff8169721d R14: ffff88083f7c3df8 R15: ffff8801946b7440
FS:  00007fea58411740(0000) GS:ffff88083f7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000009284c0 CR3: 00000005a1a6f000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88083f7c3e20 ffffffff815d25ad ffff8804c1c74000 ffff8801946b7850
0000000000000100 ffff88083f7c3e58 ffffffff81095976 ffff8804c1c74000
0000000000000038 ffffffff815d2590 0000000000000000 ffff8801946b7850
Call Trace:

NMI watchdog: BUG: soft lockup - CPU#14 stuck for 23s! [ldlm_poold:8086]
Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 ebtable_filter ebtables ip6table_filter ip6_tables xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark veth iptable_filter xt_conntrack nf_nat bridge stp llc nf_conntrack_netlink nfnetlink nf_conntrack nfsv3 nfs fscache osc(OE) mgc(OE) lustre(OE) lmv(OE) mdc(OE) lov(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) sha512_ssse3 sha512_generic overlay(T) crypto_null libcfs(OE) tun xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key l2tp_ppp l2tp_netlink l2tp_core pppox ppp_generic slhc dm_mirror dm_region_hash dm_log dm_mod intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ipmi_ssif ablk_helper cryptd ipmi_devintf dcdbas iTCO_wdt sg iTCO_vendor_support ipmi_si acpi_power_meter sb_edac edac_core ipmi_msghandler pcspkr wmi shpchp mei_me mei lpc_ich nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops ttm bnx2x drm ahci libahci mdio libata libcrc32c i2c_core ptp megaraid_sas pps_core fjes
CPU: 14 PID: 8086 Comm: ldlm_poold Tainted: G           OEL ------------ T 3.10.0-514.el7_lustre.x86_64 #1
Hardware name: Dell Inc. PowerEdge R730/0H21J3, BIOS 1.2.10 03/09/2015
task: ffff88104eb84e70 ti: ffff881051e1c000 task.ti: ffff881051e1c000
RIP: 0010:[<ffffffff8168da6a>]  [<ffffffff8168da6a>] _raw_spin_lock+0x3a/0x50
RSP: 0018:ffff881051e1fc80  EFLAGS: 00000202
RAX: 00000000000015c8 RBX: 000000000027cc90 RCX: 0000000000009bfc
RDX: 0000000000009c0c RSI: 0000000000009c0c RDI: ffff881053347218
RBP: ffff881051e1fc80 R08: 0000000000000000 R09: 0000000000000001
R10: 00000001a54f4a55 R11: 0000000000016ab9 R12: 0000000000000000
R13: 0000000000000001 R14: 00000001a54f35c2 R15: 0000000000016ab9
FS:  0000000000000000(0000) GS:ffff88085edc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000c75ac80 CR3: 00000000019ba000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff881051e1fcb0 ffffffffa108819c ffff8808dffec800 0000000000000000
0000400000000000 0000008000000000 ffff881051e1fd30 ffffffffa10a7d57
ffff881051e1fd48 000000005776ce70 ffffffffa10a5d70 0000000000089e32
Call Trace:

NMI watchdog: BUG: soft lockup - CPU#12 stuck for 24s! [java:7640]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr sg i2c_piix4 parport_pc nfit shpchp vmw_vmci parport libnvdimm nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect crc32c_intel sysimgblt fb_sys_fops
ttm mptspi ata_piix scsi_transport_spi drm serio_raw mptscsih libata mptbase vmxnet3 i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 12 PID: 7640 Comm: java Tainted: G        W    L ------------   3.10.0-514.10.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/03/2018
task: ffff88040cfc9f60 ti: ffff8803fa74c000 task.ti: ffff8803fa74c000
RIP: 0010:[<ffffffff8168de52>]  [<ffffffff8168de52>] _raw_spin_lock+0x32/0x50
RSP: 0018:ffff880625903c48  EFLAGS: 00000202
RAX: 0000000000007500 RBX: ffff88061de08990 RCX: 0000000000001dbc
RDX: 0000000000001dbe RSI: 0000000000001dbe RDI: ffff8805197cf770
RBP: ffff880625903c48 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffff81a9cd00 R11: 0000000000000000 R12: ffff880625903bb8
R13: ffffffff8169775d R14: ffff880625903c48 R15: ffff8806240cd700
FS:  00007f96da8e8700(0000) GS:ffff880625900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa548369000 CR3: 00000004917e8000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880625903ce0 ffffffff815d706d ffff880100000001 ffff8806240cd700
ffffffff81aebbd0 ffff880625903c90 00008eeda053e306 ffffffff81a9cd00
0100007f00001f90 0000000000000000 ffff880044edd0dc ffffffff81aebbd0
Call Trace: