NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [10.210.38.12-ma:74271]
Modules linked in: panfs(POE) 8021q garp mrp stp llc nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ib_isert iscsi_target_mod iTCO_wdt iTCO_vendor_support ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi ib_ucm rpcrdma kvm_intel rdma_ucm kvm ib_uverbs ib_iser irqbypass ib_ipoib libiscsi crc32_pclmul scsi_transport_iscsi ghash_clmulni_intel aesni_intel lrw ib_umad gf128mul glue_helper ablk_helper cryptd rdma_cm ib_cm joydev pcspkr iw_cm sg i2c_i801 mei_me mei shpchp lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad mlx5_ib ib_core nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ast drm_kms_helper crct10dif_pclmul
crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops mlx5_core ttm drm igb ahci libahci libata nvme nvme_core devlink dca i2c_algo_bit ptp i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 74271 Comm: 10.210.38.12-ma Tainted: P           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: Penguin Computing Relion XE1112/MR91-FS0-ZB, BIOS R03 06/03/2019
task: ffff8887d9bb2f70 ti: ffff88c409a9c000 task.ti: ffff88c409a9c000
RIP: 0010:[<ffffffffc09b5649>]  [<ffffffffc09b5649>] nfs4_schedule_state_manager+0x29/0xf0 [nfsv4]
RSP: 0018:ffff88c409a9fd28  EFLAGS: 00000247
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000b
RDX: 0000000000000001 RSI: ffff886e79b03f80 RDI: ffff88bd320ad000
RBP: ffff88c409a9fd78 R08: 0000000000000000 R09: 0000000000000000
R10: ffff88bd3da59c40 R11: ffffea02f4ea2800 R12: ffff88bd3da59c40
R13: ffffea02f4ea2800 R14: 000000000000b1ac R15: ffff88c409a9fd50
FS:  0000000000000000(0000) GS:ffff88bd3da40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2385d61000 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffffc09b74ce 0000000000000010 0000000000000206 000000007a27b1ac
ffff885b7810f050 ffff88c409a9fd78 ffffffffc09b8a6d ffff880497f87200
000000007a27b1ac ffff880497f87200 ffff88c409a9fde8 ffffffffc09b5c28
Call Trace:
[<ffffffffc09b74ce>] ? nfs_mark_test_expired_delegation.isra.6+0xe/0x20 [nfsv4]
[<ffffffffc09b8a6d>] ? nfs_inode_find_delegation_state_and_recover+0x4d/0x80 [nfsv4]
[<ffffffffc09b5c28>] nfs_inode_find_state_and_recover+0xf8/0x190 [nfsv4]
[<ffffffffc09b89bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc09b4c22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc09b4ef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc09b4f0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 00 00 0f 1f 44 00 00 55 48 89 e5 53 48 89 fb 48 83 ec 48 65 48 8b 04 25 28 00 00 00 48 89 45 f0 31 c0 f0 0f ba af 28 01 00 00 00 <19> c0 85 c0 74 19 48 8b 45 f0 65 48 33 04 25 28 00 00 00 75 6d 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [10.210.38.25-ma:102603]
Modules linked in: nvidia_uvm(OE) nvidia(POE) 8021q garp mrp stp llc panfs(POE) nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm dm_mirror dm_region_hash dm_log dm_mod rpcrdma rdma_ucm ib_uverbs ib_umad ib_iser ib_ipoib rdma_cm iw_cm libiscsi scsi_transport_iscsi ib_cm skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel iTCO_wdt iTCO_vendor_support lrw gf128mul glue_helper ablk_helper cryptd mlx5_ib ib_core pcspkr joydev mei_me mei sg shpchp lpc_ich i2c_i801 ipmi_si nfit libnvdimm acpi_power_meter acpi_pad acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif
crct10dif_generic ast i2c_algo_bit drm_kms_helper mlx5_core ttm syscopyarea sysfillrect sysimgblt fb_sys_fops drm ahci crct10dif_pclmul crct10dif_common libahci crc32c_intel libata ptp pps_core devlink i2c_core wmi ipmi_devintf ipmi_msghandler [last unloaded: nvidia]
CPU: 1 PID: 102603 Comm: 10.210.38.25-ma Tainted: P           OE  ------------   3.10.0-693.el7.x86_64 #1
Hardware name: Supermicro SYS-1029GP-TR/X11DPG-SN, BIOS 2.0b 02/26/2018
task: ffff880cac4b4f10 ti: ffff8814b3bdc000 task.ti: ffff8814b3bdc000
RIP: 0010:[<ffffffffc191d620>]  [<ffffffffc191d620>] nfs_wait_on_sequence+0x90/0x90 [nfsv4]
RSP: 0018:ffff8814b3bdfd58  EFLAGS: 00000202
RAX: ffff8816c73ec000 RBX: 000000000000000b RCX: 000000000000000b
RDX: 0000000000000001 RSI: ffff881650a15d00 RDI: ffff8816c73ec000
RBP: ffff8814b3bdfd78 R08: 0000000000000000 R09: 0000000000000000
R10: ffff880c0f659c40 R11: ffffea005fc08500 R12: 0000000000000000
R13: 0000000000000000 R14: ffff880c0f659c40 R15: ffffea005fc08500
FS:  0000000000000000(0000) GS:ffff880c0f640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5375a53000 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffffc1920a6d ffff880a99fd95c0 ffff880062b884e0 ffff8816c73ec000
ffff8814b3bdfde8 ffffffffc191dc19 ffff8814b3bdfe14 ffff880062b885e0
ffff880062b88668 0000000000000010 ffff8816c73ec000 01ffffff8121d05e
Call Trace:
[<ffffffffc1920a6d>] ? nfs_inode_find_delegation_state_and_recover+0x4d/0x80 [nfsv4]
[<ffffffffc191dc19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffffc19209bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc191cc22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc191cef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc191cf0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 00 5b 41 5c 41 5d 41 5e 89 d0 5d c3 0f 1f 80 00 00 00 00 49 8b 75 20 49 8d 55 18 e8 2b 02 a2 bf eb b3 66 0f 1f 84 00 00 00 00 00 <0f> 1f 44 00 00 55 48 89 e5 53 48 89 fb 48 83 ec 48 65 48 8b 04 

NMI watchdog: BUG: soft lockup - CPU#25 stuck for 22s! [10.62.105.205-m:134410]
Modules linked in: tcp_lp fuse rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache sr_mod cdrom dm_mirror dm_region_hash dm_log skx_edac edac_core coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_service_time iTCO_wdt iTCO_vendor_support uas usb_storage joydev sg pcspkr mei_me mei i2c_i801 shpchp lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_cpufreq dm_multipath nfsd dm_mod auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod lpfc i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crc32c_intel mpt3sas ahci libahci crc_t10dif crct10dif_generic libata crct10dif_pclmul scsi_transport_fc scsi_tgt i40e(OE) raid_class
ptp i2c_core crct10dif_common scsi_transport_sas pps_core
CPU: 25 PID: 134410 Comm: 10.62.105.205-m Tainted: G           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: Sugon I620-G30/60P24-US, BIOS 0JGST219 05/30/2019
task: ffff8804701e9fa0 ti: ffff880835bb0000 task.ti: ffff880835bb0000
RIP: 0010:[<ffffffffc0893649>]  [<ffffffffc0893649>] nfs4_schedule_state_manager+0x29/0xf0 [nfsv4]
RSP: 0018:ffff880835bb3d00  EFLAGS: 00000247
RAX: 0000000000000000 RBX: ffff880835bb3d20 RCX: 000000000000000b
RDX: 0000000000000001 RSI: ffff88085e271c00 RDI: ffff88046c019000
RBP: ffff880835bb3d50 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 7fffffffffffffff R12: ffff880835bb3e2b
R13: ffffffff8132c312 R14: ffff880835bb3c80 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff88046e940000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff4663319e5 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
000000000000000b 0000000000000000 ffff88085e271c00 ffff880179131000
ffffffffffffff10 ffffffffc08954de 0000000000000010 0000000000000202
00000000db21002e ffff880501ace408 ffff880835bb3d78 ffffffffc0896a6d
Call Trace:
[<ffffffffc08954de>] ? nfs_mark_test_expired_delegation.isra.6+0x1e/0x20 [nfsv4]
[<ffffffffc0896a6d>] nfs_inode_find_delegation_state_and_recover+0x4d/0x80 [nfsv4]
[<ffffffffc0893c19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffffc08969bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc0892c22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc0892ef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc0892f0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 00 00 0f 1f 44 00 00 55 48 89 e5 53 48 89 fb 48 83 ec 48 65 48 8b 04 25 28 00 00 00 48 89 45 f0 31 c0 f0 0f ba af 28 01 00 00 00 <19> c0 85 c0 74 19 48 8b 45 f0 65 48 33 04 25 28 00 00 00 75 6d 

NMI watchdog: BUG: soft lockup - CPU#10 stuck for 67s! [10.200.20.11-ma:35854]
Modules linked in: binfmt_misc joydev rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi iptable_security iptable_raw ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_CHECKSUM iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sb_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif iTCO_wdt hpwdt iTCO_vendor_support sg pcspkr wmi ipmi_si ioatdma hpilo ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich dca shpchp pcc_cpufreq nfsd auth_rpcgss nfs_acl lockd
grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel tg3 serio_raw hpsa i2c_core ptp scsi_transport_sas pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 10 PID: 35854 Comm: 10.200.20.11-ma Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: H3C FlexServer R390, BIOS P70 02/10/2014
task: ffff8804283cbf40 ti: ffff8802b43c0000 task.ti: ffff8802b43c0000
RIP: 0010:[<ffffffffc0730653>]  [<ffffffffc0730653>] nfs4_schedule_state_manager+0x33/0xf0 [nfsv4]
RSP: 0018:ffff8802b43c3d00  EFLAGS: 00000286
RAX: 00000000e5d4b304 RBX: 000000000000b304 RCX: 000000000000000b
RDX: 0000000000000001 RSI: ffff88081f7d9100 RDI: ffff88040d3c7000
RBP: ffff8802b43c3d50 R08: 0000000000000000 R09: 0000000000000000
R10: ffff88083f499c40 R11: ffffea001e6eb480 R12: ffff8802b43c3cd8
R13: ffff88079bad3800 R14: ffffffffc0398996 R15: ffff8802b43c3c70
FS:  0000000000000000(0000) GS:ffff88083f480000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcdb402b000 CR3: 00000000019f2000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000005 000000000000000b ffff8802b43c3e18 ffff88081f7d9124
ffffffffffffff10 ffff8802b43c3dc8 0000000000000010 ffffea001e6eb480
00000000e5d4b304 ffff88019448cf90 ffff8802b43c3d78 ffffffffc0733a6d
Call Trace:
[<ffffffffc0733a6d>] nfs_inode_find_delegation_state_and_recover+0x4d/0x80 [nfsv4]
[<ffffffffc0730c19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffffc07339bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc072fc22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc072fef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc072ff0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: e5 53 48 89 fb 48 83 ec 48 65 48 8b 04 25 28 00 00 00 48 89 45 f0 31 c0 f0 0f ba af 28 01 00 00 00 19 c0 85 c0 74 19 48 8b 45 f0 <65> 48 33 04 25 28 00 00 00 75 6d 48 83 c4 48 5b 5d c3 0f 1f 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [192.168.0.200-m:1439]
Modules linked in: arc4 md4 nls_utf8 cifs rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) vfat fat snd_hda_codec_hdmi dm_mirror dm_region_hash dm_log dm_mod intel_powerclamp coretemp kvm_intel kvm irqbypass snd_hda_intel crc32_pclmul snd_hda_codec snd_hda_core joydev ghash_clmulni_intel snd_hwdep snd_seq aesni_intel snd_seq_device lrw gf128mul snd_pcm glue_helper ablk_helper cryptd snd_timer snd ppdev soundcore pcspkr sg shpchp tpm_crb parport_pc parport int3400_thermal
acpi_pad nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic nouveau mxm_wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm libata crct10dif_pclmul crct10dif_common crc32c_intel r8169 serio_raw mii i2c_hid i2c_core video wmi
CPU: 1 PID: 1439 Comm: 192.168.0.200-m Tainted: G           OE  ------------   3.10.0-693.el7.x86_64 #1
Hardware name: LENOVO 90KQCTO1WW/317A, BIOS M31KT22A 10/15/2020
task: ffff8807fea68fd0 ti: ffff8803b7f44000 task.ti: ffff8803b7f44000
RIP: 0010:[<ffffffffc09204ce>]  [<ffffffffc09204ce>] nfs_mark_test_expired_delegation.isra.6+0xe/0x20 [nfsv4]
RSP: 0018:ffff8803b7f47d50  EFLAGS: 00000206
RAX: ffff880800f67800 RBX: ffffffffffffff10 RCX: 000000000000000b
RDX: 0000000000000000 RSI: ffff88049b254300 RDI: ffff880800f62000
RBP: ffff8803b7f47d50 R08: 0000000000000000 R09: 0000000000000000
R10: ffff88083ea59c40 R11: ffffea001e979d80 R12: ffff88049b254300
R13: 0000000000000001 R14: 000000000000000b R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff88083ea40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: fffff9802b24a058 CR3: 0000000836aea000 CR4: 00000000003427e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8803b7f47d78 ffffffffc0921a8b ffff8803dbe281c8 ffff8803dbe280c8
ffff88081deeb800 ffff8803b7f47de8 ffffffffc091ec19 ffff8803b7f47e14
ffff8803dbe281c8 ffff8803dbe28250 ffff8803b7f47db8 ffff88081deeb800
Call Trace:
[<ffffffffc0921a8b>] nfs_inode_find_delegation_state_and_recover+0x6b/0x80 [nfsv4]
[<ffffffffc091ec19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffffc09219bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc091dc22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc091def0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc091df0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 5b 41 5c 41 5d 5d c3 31 c0 eb e2 e8 6d 72 76 c0 0f 1f 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 f0 80 66 48 fe <f0> 80 4e 48 40 48 8b 07 f0 80 88 29 01 00 00 20 5d c3 0f 1f 44 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [192.168.0.200-m:6560]
Modules linked in: arc4 md4 nls_utf8 cifs fuse rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) vfat fat snd_hda_codec_hdmi dm_mirror dm_region_hash dm_log dm_mod snd_hda_intel snd_hda_codec snd_hda_core intel_powerclamp coretemp snd_hwdep kvm_intel kvm snd_seq joydev snd_seq_device snd_pcm irqbypass crc32_pclmul ghash_clmulni_intel snd_timer aesni_intel snd lrw gf128mul glue_helper ablk_helper cryptd soundcore ppdev sg pcspkr parport_pc shpchp parport tpm_crb acpi_pad int3400_thermal
nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic nouveau mxm_wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci libata crct10dif_pclmul crct10dif_common crc32c_intel r8169 serio_raw mii i2c_hid i2c_core video wmi
CPU: 4 PID: 6560 Comm: 192.168.0.200-m Tainted: G           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: LENOVO 90KQCTO1WW/317A, BIOS M31KT22A 10/15/2020
task: ffff8807bf371fa0 ti: ffff8807e5a90000 task.ti: ffff8807e5a90000
RIP: 0010:[<ffffffffc09184de>]  [<ffffffffc09184de>] nfs_mark_test_expired_delegation.isra.6+0x1e/0x20 [nfsv4]
RSP: 0018:ffff8807e5a93d50  EFLAGS: 00000202
RAX: ffff8804bcfc4800 RBX: ffffffffffffff10 RCX: 000000000000000b
RDX: 0000000000000000 RSI: ffff8804be362580 RDI: ffff880493698800
RBP: ffff8807e5a93d50 R08: 0000000000000000 R09: 0000000000000000
R10: 000000002ef2be01 R11: ffffea0020bbca00 R12: ffff8804be362580
R13: 0000000000000001 R14: 000000000000000b R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff88083eb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffc08f4fd5 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8807e5a93d78 ffffffffc0919a8b ffff88036ae932e8 ffff88036ae931e8
ffff8804bcfc4800 ffff8807e5a93de8 ffffffffc0916c19 ffff8807e5a93e14
ffff88036ae932e8 ffff88036ae93370 ffffffff8132297d ffff8804bcfc4800
Call Trace:
[<ffffffffc0919a8b>] nfs_inode_find_delegation_state_and_recover+0x6b/0x80 [nfsv4]
[<ffffffffc0916c19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffff8132297d>] ? _atomic_dec_and_lock+0x4d/0x70
[<ffffffffc09199bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc0915c22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc0915ef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc0915f0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 0f 1f 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 f0 80 66 48 fe f0 80 4e 48 40 48 8b 07 f0 80 88 29 01 00 00 20 <5d> c3 0f 1f 44 00 00 55 48 89 e5 41 55 41 54 49 89 fc 53 48 8b 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [192.168.0.200-m:6560]
Modules linked in: arc4 md4 nls_utf8 cifs fuse rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) vfat fat snd_hda_codec_hdmi dm_mirror dm_region_hash dm_log dm_mod snd_hda_intel snd_hda_codec snd_hda_core intel_powerclamp coretemp snd_hwdep kvm_intel kvm snd_seq joydev snd_seq_device snd_pcm irqbypass crc32_pclmul ghash_clmulni_intel snd_timer aesni_intel snd lrw gf128mul glue_helper ablk_helper cryptd soundcore ppdev sg pcspkr parport_pc shpchp parport tpm_crb acpi_pad int3400_thermal
nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic nouveau mxm_wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci libata crct10dif_pclmul crct10dif_common crc32c_intel r8169 serio_raw mii i2c_hid i2c_core video wmi
CPU: 4 PID: 6560 Comm: 192.168.0.200-m Tainted: G           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: LENOVO 90KQCTO1WW/317A, BIOS M31KT22A 10/15/2020
task: ffff8807bf371fa0 ti: ffff8807e5a90000 task.ti: ffff8807e5a90000
RIP: 0010:[<ffffffff8132ba65>]  [<ffffffff8132ba65>] memcmp+0x35/0x50
RSP: 0018:ffff8807e5a93d50  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffffffff10 RCX: 0000000000000008
RDX: 000000000000000b RSI: ffff8807e5a93e18 RDI: ffff8804be3625a4
RBP: ffff8807e5a93d50 R08: 0000000000000000 R09: 0000000000000000
R10: 000000002ef2be01 R11: ffffea0020bbca00 R12: ffff8804be362580
R13: 0000000000000001 R14: 000000000000000b R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff88083eb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffc08f4fe4 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8807e5a93d78 ffffffffc0919a5b ffff88036ae932e8 ffff88036ae931e8
ffff8804bcfc4800 ffff8807e5a93de8 ffffffffc0916c19 ffff8807e5a93e14
ffff88036ae932e8 ffff88036ae93370 ffffffff8132297d ffff8804bcfc4800
Call Trace:
[<ffffffffc0919a5b>] nfs_inode_find_delegation_state_and_recover+0x3b/0x80 [nfsv4]
[<ffffffffc0916c19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffff8132297d>] ? _atomic_dec_and_lock+0x4d/0x70
[<ffffffffc09199bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc0915c22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc0915ef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc0915f0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: b6 07 0f b6 0e 29 c8 75 27 48 83 ea 01 31 c9 eb 1a 0f 1f 44 00 00 44 0f b6 4c 0f 01 44 0f b6 44 0e 01 48 83 c1 01 45 29 c1 75 0b <48> 39 d1 75 e6 5d c3 0f 1f 40 00 44 89 c8 5d c3 31 c0 5d c3 0f 

NMI watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [192.168.0.200-m:12705]
Modules linked in: arc4 md4 nls_utf8 cifs fuse rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) dm_mirror dm_region_hash dm_log dm_mod vfat fat snd_hda_codec_hdmi intel_powerclamp coretemp kvm_intel snd_hda_intel kvm joydev snd_hda_codec snd_hda_core snd_hwdep snd_seq irqbypass crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel lrw snd_timer gf128mul glue_helper ablk_helper snd cryptd ppdev soundcore pcspkr sg parport_pc shpchp parport tpm_crb acpi_pad int3400_thermal
nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic nouveau mxm_wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw r8169 mii i2c_hid i2c_core video wmi
CPU: 13 PID: 12705 Comm: 192.168.0.200-m Tainted: G           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: LENOVO 90KQCTO1WW/317A, BIOS M31KT22A 10/15/2020
task: ffff880839d30fd0 ti: ffff8805147cc000 task.ti: ffff8805147cc000
RIP: 0010:[<ffffffffc0941649>]  [<ffffffffc0941649>] nfs4_schedule_state_manager+0x29/0xf0 [nfsv4]
RSP: 0018:ffff8805147cfd00  EFLAGS: 00000247
RAX: 0000000000000000 RBX: 0000000000006789 RCX: 000000000000000b
RDX: 0000000000000001 RSI: ffff8805732ff480 RDI: ffff880834fc4c00
RBP: ffff8805147cfd50 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000021af6201 R11: ffffea001486bd00 R12: ffff8805147cfd30
R13: ffffea001486bd00 R14: ffffffffc045d996 R15: ffff8805147cfc70
FS:  0000000000000000(0000) GS:ffff88083ed40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7cb810d550 CR3: 00000000019f2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
000000000000000b 0000000000000000 ffff8805732ff480 ffff88051094b800
ffffffffffffff10 ffffffffc09434de 0000000000000010 0000000000000202
00000000c5546789 ffff8804fd8e4b78 ffff8805147cfd78 ffffffffc0944a6d
Call Trace:
[<ffffffffc09434de>] ? nfs_mark_test_expired_delegation.isra.6+0x1e/0x20 [nfsv4]
[<ffffffffc0944a6d>] nfs_inode_find_delegation_state_and_recover+0x4d/0x80 [nfsv4]
[<ffffffffc0941c19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffffc09449bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc0940c22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc0940ef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc0940f0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 00 00 0f 1f 44 00 00 55 48 89 e5 53 48 89 fb 48 83 ec 48 65 48 8b 04 25 28 00 00 00 48 89 45 f0 31 c0 f0 0f ba af 28 01 00 00 00 <19> c0 85 c0 74 19 48 8b 45 f0 65 48 33 04 25 28 00 00 00 75 6d 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [192.168.20.225-:23262]
Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_nat veth vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark nfsv3 xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache overlay(T) vmw_vsock_vmci_transport vsock ppdev vmw_balloon joydev pcspkr nfit sg libnvdimm vmw_vmci shpchp i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect crct10dif_pclmul crct10dif_common sysimgblt crc32_pclmul fb_sys_fops ttm crc32c_intel drm scsi_transport_iscsi ghash_clmulni_intel
8021q garp stp llc mrp ahci libahci aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ata_piix libata serio_raw i2c_core vmxnet3 vmw_pvscsi sunrpc dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 23262 Comm: 192.168.20.225- Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8800254a1fa0 ti: ffff880b7ae10000 task.ti: ffff880b7ae10000
RIP: 0010:[<ffffffffc05c7649>]  [<ffffffffc05c7649>] nfs4_schedule_state_manager+0x29/0xf0 [nfsv4]
RSP: 0018:ffff880b7ae13d28  EFLAGS: 00000247
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000b
RDX: 0000000000000001 RSI: ffff880bf1e29d80 RDI: ffff880bf4fdf800
RBP: ffff880b7ae13d78 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: dead000000000200 R15: ffff880b7ae13d50
FS:  0000000000000000(0000) GS:ffff880c3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000c001f07000 CR3: 00000000019f2000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffffc05c94d3 0000000000000010 ffff880b7ae13dc0 000000006b040d95
ffff8804fc89c348 ffff880b7ae13d78 ffffffffc05caa6d ffff880c02e5fa80
000000006b040d95 ffff880c02e5fa80 ffff880b7ae13de8 ffffffffc05c7c28
Call Trace:
[<ffffffffc05c94d3>] ? nfs_mark_test_expired_delegation.isra.6+0x13/0x20 [nfsv4]
[<ffffffffc05caa6d>] ? nfs_inode_find_delegation_state_and_recover+0x4d/0x80 [nfsv4]
[<ffffffffc05c7c28>] nfs_inode_find_state_and_recover+0xf8/0x190 [nfsv4]
[<ffffffffc05ca9bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc05c6c22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc05c6ef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc05c6f0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 00 00 66 66 66 66 90 55 48 89 e5 53 48 89 fb 48 83 ec 48 65 48 8b 04 25 28 00 00 00 48 89 45 f0 31 c0 f0 0f ba af 28 01 00 00 00 <19> c0 85 c0 74 19 48 8b 45 f0 65 48 33 04 25 28 00 00 00 75 6d 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [192.168.20.225-:27108]
Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_nat veth vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache overlay(T) vmw_vsock_vmci_transport vsock ppdev vmw_balloon sg joydev pcspkr i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx crc32_pclmul crc32c_intel drm_kms_helper sd_mod ghash_clmulni_intel crc_t10dif syscopyarea sysfillrect crct10dif_generic sysimgblt crct10dif_pclmul fb_sys_fops scsi_transport_iscsi aesni_intel
ttm crct10dif_common 8021q ahci garp stp drm ata_piix libahci llc mrp lrw gf128mul libata glue_helper ablk_helper serio_raw cryptd vmw_pvscsi vmxnet3 i2c_core sunrpc dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 27108 Comm: 192.168.20.225- Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff880c04b86eb0 ti: ffff880b47b00000 task.ti: ffff880b47b00000
RIP: 0010:[<ffffffffc05c664b>]  [<ffffffffc05c664b>] nfs4_schedule_state_manager+0x2b/0xf0 [nfsv4]
RSP: 0018:ffff880b47b03d00  EFLAGS: 00000297
RAX: 00000000ffffffff RBX: ffff880bfe649fa0 RCX: 000000000000000b
RDX: 0000000000000001 RSI: ffff880b318a4180 RDI: ffff880c013bb800
RBP: ffff880b47b03d50 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff880b47b03c88
R13: ffff880c3fd56cc0 R14: 0000000000000005 R15: ffff880c3fd56cc0
FS:  0000000000000000(0000) GS:ffff880c3fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000007c0843428 CR3: 00000000019f2000 CR4: 00000000000007f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880b318a4180 ffff880c013bb800 ffffffffffffff10 ffffffffc05c662a
0000000000000010 ffff880b47b03dc8 ffff880b47b03d48 0000000000000000
000000004db1e6df ffff88046cf281c8 ffff880b47b03d78 ffffffffc05c9a6d
Call Trace:
[<ffffffffc05c662a>] ? nfs4_schedule_state_manager+0xa/0xf0 [nfsv4]
[<ffffffffc05c9a6d>] nfs_inode_find_delegation_state_and_recover+0x4d/0x80 [nfsv4]
[<ffffffffc05c6c19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffffc05c99bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc05c5c22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc05c5ef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc05c5f0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 66 66 66 66 90 55 48 89 e5 53 48 89 fb 48 83 ec 48 65 48 8b 04 25 28 00 00 00 48 89 45 f0 31 c0 f0 0f ba af 28 01 00 00 00 19 c0 <85> c0 74 19 48 8b 45 f0 65 48 33 04 25 28 00 00 00 75 6d 48 83 

NMI watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [10.1.1.202-mana:9757]
Modules linked in: fuse ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ip_set nfnetlink ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_security iptable_raw 8021q garp mrp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache binfmt_misc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core dm_mirror dm_region_hash
dm_log sb_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel iTCO_wdt iTCO_vendor_support kvm irqbypass dm_mod crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq joydev snd_seq_device snd_pcm pcspkr snd_timer mei_me sg snd mei soundcore ipmi_si lpc_ich shpchp i2c_i801 ipmi_devintf ipmi_msghandler tpm_infineon nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic mlx4_en uas usb_storage ast drm_kms_helper syscopyarea qla2xxx sysfillrect sysimgblt isci fb_sys_fops mlx4_core ttm igb ahci libsas drm libahci scsi_transport_sas crct10dif_pclmul libata crct10dif_common ptp crc32c_intel pps_core serio_raw megaraid_sas
devlink dca scsi_transport_fc i2c_algo_bit i2c_core scsi_tgt
CPU: 10 PID: 9757 Comm: 10.1.1.202-mana Tainted: G        W      ------------   3.10.0-693.2.2.el7.x86_64 #1
Hardware name: ASUSTeK COMPUTER INC. Z9PE-D16 Series/Z9PE-D16 Series, BIOS 0805 06/22/2012
task: ffff8815a87bcf10 ti: ffff8800264b0000 task.ti: ffff8800264b0000
RIP: 0010:[<ffffffffc0ac24d3>]  [<ffffffffc0ac24d3>] nfs_mark_test_expired_delegation.isra.6+0x13/0x20 [nfsv4]
RSP: 0018:ffff8800264b3d50  EFLAGS: 00000202
RAX: ffff882032ae0800 RBX: ffffffffffffff10 RCX: 000000000000000b
RDX: 0000000000000000 RSI: ffff8813bad50600 RDI: ffff882032ae0000
RBP: ffff8800264b3d50 R08: 0000000000000000 R09: 0000000000000000
R10: ffff88207fc99c40 R11: ffffea0050238580 R12: ffff8813bad50600
R13: 0000000000000001 R14: 000000000000000b R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff88207fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055c82c847cb0 CR3: 00000000019f2000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800264b3d78 ffffffffc0ac3a8b ffff881520eab2e8 ffff881520eab1e8
ffff88200d200c00 ffff8800264b3de8 ffffffffc0ac0c19 ffff8800264b3e14
ffff881520eab2e8 ffff881520eab370 ffffffffffffff10 ffff88200d200c00
Call Trace:
[<ffffffffc0ac3a8b>] nfs_inode_find_delegation_state_and_recover+0x6b/0x80 [nfsv4]
[<ffffffffc0ac0c19>] nfs_inode_find_state_and_recover+0xe9/0x190 [nfsv4]
[<ffffffffc0ac39bb>] nfs_reap_expired_delegations+0x1bb/0x220 [nfsv4]
[<ffffffffc0abfc22>] nfs4_state_manager+0x5f2/0x8c0 [nfsv4]
[<ffffffffc0abfef0>] ? nfs4_state_manager+0x8c0/0x8c0 [nfsv4]
[<ffffffffc0abff0f>] nfs4_run_state_manager+0x1f/0x40 [nfsv4]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f58>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 5d c3 31 c0 eb e2 e8 6d 52 5c c0 0f 1f 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 f0 80 66 48 fe f0 80 4e 48 40 <48> 8b 07 f0 80 88 29 01 00 00 20 5d c3 66 66 66 66 90 55 48 89