Complete report #1374699
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
rpc_xprt_switch_has_addr
|
sunrpc |
0xffffffffc063576f
Build id: 3.10.0-957.el7.x86_64
|
0x6f
|
|
2
|
rpc_clnt_xprt_switch_has_addr
|
sunrpc |
0xffffffffc0611175
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
3
|
nfs_get_client
|
nfs |
0xffffffffc069fe02
Build id: 3.10.0-957.el7.x86_64
|
0x182
|
|
4
|
nfs4_set_client
|
nfsv4 |
0xffffffffc076039c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
5
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0761b4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
6
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0758bae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
7
|
mount_fs
|
vmlinux |
0xffffffff90245bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
8
|
|
vmlinux |
0xffffffff901daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
9
|
vfs_kern_mount
|
vmlinux |
0xffffffff90263777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
10
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0758ad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
11
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0758ed4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
12
|
|
nfs |
0xffffffffc06a06e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
13
|
nfs_fs_mount
|
nfs |
0xffffffffc06acd3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
14
|
|
nfs |
0xffffffffc06adca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
15
|
|
nfs |
0xffffffffc06abb40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
16
|
mount_fs
|
vmlinux |
0xffffffff90245bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
17
|
|
vmlinux |
0xffffffff901daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
18
|
vfs_kern_mount
|
vmlinux |
0xffffffff90263777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
19
|
do_mount
|
vmlinux |
0xffffffff90265d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
20
|
|
vmlinux |
0xffffffff9023e03a
Build id: 3.10.0-957.el7.x86_64
|
0x1ca
|
|
21
|
|
vmlinux |
0xffffffff9021c4dc
Build id: 3.10.0-957.el7.x86_64
|
0x3c
|
|
22
|
SyS_mount
|
vmlinux |
0xffffffff90266bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
23
|
system_call_fastpath
|
vmlinux |
0xffffffff90774ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [mount.nfs:20882]
Modules linked in: rpcsec_gss_krb5 binfmt_misc sch_fq_codel knem(OE) ip_tables bnx2 be2net dm_mirror dm_region_hash dm_log dm_mod megaraid_sas uas usb_storage tg3 bnxt_en ixgbe mdio pata_acpi sr_mod cdrom dcdbas mxm_wmi mlx4_en(OE) mlx5_ib(OE) mlx5_fpga_tools(OE) mlx5_core(OE) mlxfw(OE) ib_umad(OE) ib_ipoib(OE) ib_ucm(OE) rdma_ucm(OE) rdma_cm(OE) ib_cm(OE) iw_cm(OE) ib_uverbs(OE) xfs libcrc32c ext4 jbd2 mbcache nfsv4 dns_resolver nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc grace bridge stp llc mlx4_ib(OE) ib_core(OE) sd_mod crc_t10dif crct10dif_generic sg mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm sb_edac crct10dif_pclmul crct10dif_common drm_panel_orientation_quirks ahci crc32c_intel igb libahci libata ptp pps_core mlx4_core(OE) i2c_algo_bit mei_me
mlx_compat(OE) devlink mei wmi dca
CPU: 2 PID: 20882 Comm: mount.nfs Tainted: G OE ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: IBM IBM NeXtScale nx360 M4: -[5455I4B]-/00KF518, BIOS -[FHE114CUS-1.60]- 12/10/2015
task: ffff91db9b27a080 ti: ffff91db672bc000 task.ti: ffff91db672bc000
RIP: 0010:[<ffffffffc063576f>] [<ffffffffc063576f>] rpc_xprt_switch_has_addr+0x6f/0x190 [sunrpc]
RSP: 0018:ffff91db672bfab0 EFLAGS: 00000246
RAX: ffff91db9bda4cd0 RBX: 0000000000000246 RCX: 0000000000000005
RDX: ffff91db99e28360 RSI: ffff91db8ccae128 RDI: ffff91db9bda4cc0
RBP: ffff91db672bfae8 R08: ffff91db672bfad0 R09: ffff91db99e28300
R10: ffff91db8ccae128 R11: 0000000000000005 R12: ffff91db9b27a080
R13: 0000000000000246 R14: ffff91db6d3b7ad0 R15: ffff91db99e28300
FS: 00002b20278751c0(0000) GS:ffff91db9f900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002b847df0a030 CR3: 000000085be04000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc0611175>] rpc_clnt_xprt_switch_has_addr+0x15/0x20 [sunrpc]
[<ffffffffc069fe02>] nfs_get_client+0x182/0x470 [nfs]
[<ffffffffc076039c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0761b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0758bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff90245bde>] mount_fs+0x3e/0x1b0
[<ffffffff901daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff90263777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0758ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0758ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc06a06e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc06acd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc06adca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc06abb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff90245bde>] mount_fs+0x3e/0x1b0
[<ffffffff901daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff90263777>] vfs_kern_mount+0x67/0x110
[<ffffffff90265d9f>] do_mount+0x1ef/0xce0
[<ffffffff9023e03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff9021c4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff90266bd3>] SyS_mount+0x83/0xd0
[<ffffffff90774ddb>] system_call_fastpath+0x22/0x27
Code: 45 c8 48 8b 45 c8 49 39 c6 48 8d 98 e0 fb ff ff 74 7a 4c 8d 6e 08 eb 1f 0f 1f 40 00 48 8b 83 20 04 00 00 48 89 45 c8 48 8b 45 c8 <49> 39 c6 48 8d 98 e0 fb ff ff 74 55 41 0f b7 04 24 66 3b 43 18
Complete report #1188351
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_lock
|
vmlinux |
0xffffffffa6d7c3f0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x10
|
|
2
|
|
nfs |
0xffffffffc0982ebb
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x23b
|
|
3
|
nfs4_set_client
|
nfsv4 |
0xffffffffc09ec39c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
4
|
nfs4_create_server
|
nfsv4 |
0xffffffffc09edb4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
5
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc09e4bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
6
|
mount_fs
|
vmlinux |
0xffffffffa6846bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
7
|
|
vmlinux |
0xffffffffa67dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
8
|
vfs_kern_mount
|
vmlinux |
0xffffffffa6864777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
9
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc09e4ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
10
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc09e4ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
11
|
|
nfs |
0xffffffffc09836e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
12
|
nfs_fs_mount
|
nfs |
0xffffffffc098fd3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
13
|
|
nfs |
0xffffffffc0990ca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
14
|
|
nfs |
0xffffffffc098eb40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
15
|
mount_fs
|
vmlinux |
0xffffffffa6846bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
16
|
|
vmlinux |
0xffffffffa67dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
17
|
vfs_kern_mount
|
vmlinux |
0xffffffffa6864777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
18
|
do_mount
|
vmlinux |
0xffffffffa6866d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
19
|
|
vmlinux |
0xffffffffa6866ad0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xc0
|
|
20
|
SyS_mount
|
vmlinux |
0xffffffffa6867bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
21
|
tracesys
|
vmlinux |
0xffffffffa6d8705b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#61 stuck for 22s! [mount.nfs:77929]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nfsv3 nfs_acl rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cachefiles fscache dm_mirror dm_region_hash dm_log dm_mod vfat fat sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd iTCO_wdt iTCO_vendor_support pcspkr sg hpilo hpwdt lpc_ich i2c_i801 ipmi_si ioatdma ipmi_devintf dca ipmi_msghandler pcc_cpufreq wmi acpi_power_meter auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit ahci crct10dif_pclmul drm_kms_helper crct10dif_common crc32c_intel libahci syscopyarea sysfillrect sysimgblt libata fb_sys_fops tg3 ttm ptp drm pps_core drm_panel_orientation_quirks
CPU: 61 PID: 77929 Comm: mount.nfs Kdump: loaded Tainted: G ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff9e90b3e0d140 ti: ffff9e901e208000 task.ti: ffff9e901e208000
RIP: 0010:[<ffffffffa6d7c3f0>] [<ffffffffa6d7c3f0>] _raw_spin_lock+0x10/0x30
RSP: 0018:ffff9e901e20bb00 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffa66c3956 RCX: 0000000000000002
RDX: 0000000000000001 RSI: ffff9e80b776e5b8 RDI: ffff9e80b776e5b8
RBP: ffff9e901e20bb60 R08: ffff9e901e20bad0 R09: ffff9e80b776e500
R10: ffff9e90257f5928 R11: 0000000000000005 R12: 0000000000000293
R13: 0000000000000293 R14: ffffffffc09aa070 R15: ffff9e901e20bad0
FS: 00007fe4abb36880(0000) GS:ffff9e90bf140000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ff93897e0 CR3: 0000001fbe7a6000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0982ebb>] ? nfs_get_client+0x23b/0x470 [nfs]
[<ffffffffc09ec39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc09edb4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc09e4bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa6846bde>] mount_fs+0x3e/0x1b0
[<ffffffffa67dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa6864777>] vfs_kern_mount+0x67/0x110
[<ffffffffc09e4ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc09e4ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc09836e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc098fd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0990ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc098eb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa6846bde>] mount_fs+0x3e/0x1b0
[<ffffffffa67dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa6864777>] vfs_kern_mount+0x67/0x110
[<ffffffffa6866d9f>] do_mount+0x1ef/0xce0
[<ffffffffa6866ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffffa6867bd3>] SyS_mount+0x83/0xd0
[<ffffffffa6d8705b>] tracesys+0xa3/0xc9
Code: 44 00 00 85 d2 74 e4 0f 1f 40 00 eb ed 66 0f 1f 44 00 00 b8 01 00 00 00 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17 <85> c0 75 01 c3 55 89 c6 48 89 e5 e8 40 1b ff ff 5d c3 0f 1f 40
Complete report #856704
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_lock_irqsave
|
vmlinux |
0xffffffffa5f6a461
Build id: 3.10.0-957.el7.x86_64
|
0x11
|
|
2
|
prepare_to_wait
|
vmlinux |
0xffffffffa58c27c7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
3
|
|
vmlinux |
0xffffffffa58c2956
Build id: 3.10.0-957.el7.x86_64
|
0x56
|
|
4
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0798358
Build id: 3.10.0-957.el7.x86_64
|
0x88
|
|
5
|
|
vmlinux |
0xffffffffa58c2d00
Build id: 3.10.0-957.el7.x86_64
|
0x30
|
|
6
|
nfs_get_client
|
nfs |
0xffffffffc0799eab
Build id: 3.10.0-957.el7.x86_64
|
0x22b
|
|
7
|
nfs4_set_client
|
nfsv4 |
0xffffffffc080339c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
8
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0804b4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
9
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc07fbbae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
10
|
mount_fs
|
vmlinux |
0xffffffffa5a45bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
11
|
|
vmlinux |
0xffffffffa59daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
12
|
vfs_kern_mount
|
vmlinux |
0xffffffffa5a63777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
13
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc07fbad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
14
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc07fbed4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
15
|
|
nfs |
0xffffffffc079a6e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
16
|
nfs_fs_mount
|
nfs |
0xffffffffc07a6d3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
17
|
|
nfs |
0xffffffffc07a7ca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
18
|
|
nfs |
0xffffffffc07a5b40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
19
|
mount_fs
|
vmlinux |
0xffffffffa5a45bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
20
|
|
vmlinux |
0xffffffffa59daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
21
|
vfs_kern_mount
|
vmlinux |
0xffffffffa5a63777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
22
|
do_mount
|
vmlinux |
0xffffffffa5a65d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
23
|
|
vmlinux |
0xffffffffa5a65ad0
Build id: 3.10.0-957.el7.x86_64
|
0xc0
|
|
24
|
SyS_mount
|
vmlinux |
0xffffffffa5a66bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
25
|
system_call_fastpath
|
vmlinux |
0xffffffffa5f74ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#63 stuck for 23s! [mount.nfs:41741]
Modules linked in: nfsd nfs_acl xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel xt_set ip_set_bitmap_port ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport dummy rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_comment xt_mark xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter
nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack sunrpc vfat fat skx_edac coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr wdat_wdt joydev ses enclosure scsi_transport_sas sg mei_me mei lpc_ich i2c_i801 acpi_cpufreq acpi_power_meter ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic uas crct10dif_pclmul crct10dif_common crc32c_intel i40e igb usb_storage ahci libahci ptp i2c_algo_bit megaraid_sas pps_core dca libata nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 63 PID: 41741 Comm: mount.nfs Kdump: loaded Tainted: G ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: Huawei 2288H V5/BC11SPSCB0, BIOS 7.93 01/14/2021
task: ffff959122608000 ti: ffff9581519cc000 task.ti: ffff9581519cc000
RIP: 0010:[<ffffffffa5f6a461>] [<ffffffffa5f6a461>] _raw_spin_lock_irqsave+0x11/0x40
RSP: 0018:ffff9581519cfa68 EFLAGS: 00000246
RAX: 0000000000000246 RBX: ffff95919f2dab80 RCX: ffff9581519cfab8
RDX: 0000000000000082 RSI: ffff9581519cfab8 RDI: ffffffffc07c1070
RBP: ffff9581519cfa70 R08: ffff9581519cfad0 R09: ffff959038ebc000
R10: ffff959199d8c128 R11: 0000000000000005 R12: ffff959122608698
R13: 00000001659a10a8 R14: ffff95919f2c0000 R15: ffffffffa582a621
FS: 00007f7086c9c880(0000) GS:ffff95919f2c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa9bde41028 CR3: 000000005eb08000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa58c27c7>] prepare_to_wait+0x27/0x90
[<ffffffffa58c2956>] ? finish_wait+0x56/0x70
[<ffffffffc0798358>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffffa58c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0799eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc080339c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0804b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc07fbbae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa5a45bde>] mount_fs+0x3e/0x1b0
[<ffffffffa59daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa5a63777>] vfs_kern_mount+0x67/0x110
[<ffffffffc07fbad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc07fbed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc079a6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc07a6d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc07a7ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc07a5b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa5a45bde>] mount_fs+0x3e/0x1b0
[<ffffffffa59daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa5a63777>] vfs_kern_mount+0x67/0x110
[<ffffffffa5a65d9f>] do_mount+0x1ef/0xce0
[<ffffffffa5a65ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffffa5a66bd3>] SyS_mount+0x83/0xd0
[<ffffffffa5f74ddb>] system_call_fastpath+0x22/0x27
Code: 01 c3 55 89 c6 48 89 e5 e8 fd 1a ff ff 5d c3 90 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 53 9c 58 0f 1f 44 00 00 <48> 89 c3 fa 66 0f 1f 44 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17
Complete report #837698
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffba5817c5
Build id: 3.10.0-1062.el7.x86_64
|
0x15
|
|
2
|
prepare_to_wait
|
vmlinux |
0xffffffffb9ec5cb6
Build id: 3.10.0-1062.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc09a7488
Build id: 3.10.0-1062.el7.x86_64
|
0x88
|
|
4
|
|
vmlinux |
0xffffffffb9ec61c0
Build id: 3.10.0-1062.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc09a8e9b
Build id: 3.10.0-1062.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0a7650c
Build id: 3.10.0-1062.el7.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0a77cbe
Build id: 3.10.0-1062.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0a6ec8e
Build id: 3.10.0-1062.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffffba04d1fe
Build id: 3.10.0-1062.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffffb9fe1035
Build id: 3.10.0-1062.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffffba06b377
Build id: 3.10.0-1062.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0a6ebb6
Build id: 3.10.0-1062.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0a6efb4
Build id: 3.10.0-1062.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc09a96d7
Build id: 3.10.0-1062.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc09b5d9b
Build id: 3.10.0-1062.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc09b6d00
Build id: 3.10.0-1062.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc09b4bb0
Build id: 3.10.0-1062.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffffba04d1fe
Build id: 3.10.0-1062.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffffb9fe1035
Build id: 3.10.0-1062.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffffba06b377
Build id: 3.10.0-1062.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffffba06dacf
Build id: 3.10.0-1062.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffffba06d800
Build id: 3.10.0-1062.el7.x86_64
|
0xc0
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffffba06e903
Build id: 3.10.0-1062.el7.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffffba58bede
Build id: 3.10.0-1062.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#16 stuck for 23s! [mount.nfs:2458]
Modules linked in: fuse ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter nvidia_uvm(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) vfat fat nvidia(POE) snd_hda_codec_hdmi rndis_host cdc_ether usbnet mii iTCO_wdt iTCO_vendor_support intel_powerclamp coretemp kvm_intel kvm snd_hda_codec_realtek snd_hda_codec_generic irqbypass crc32_pclmul ghash_clmulni_intel snd_hda_intel aesni_intel snd_hda_codec lrw gf128mul glue_helper ablk_helper cryptd snd_hda_core pcspkr snd_hwdep snd_seq snd_seq_device snd_pcm ipmi_ssif snd_timer snd sg i2c_i801 soundcore mei_me mei ipmi_si ipmi_devintf ipmi_msghandler pinctrl_lewisburg pinctrl_intel acpi_pad acpi_power_meter pcc_cpufreq
acpi_cpufreq joydev sch_fq_codel ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm igb libahci crct10dif_pclmul crct10dif_common crc32c_intel libata nvme nvme_core ptp pps_core dca i2c_algo_bit drm_panel_orientation_quirks wmi
CPU: 16 PID: 2458 Comm: mount.nfs Kdump: loaded Tainted: P W OE ------------ 3.10.0-1062.el7.x86_64 #1
Hardware name: Supermicro X12DAi-N6/X12DAi-N6, BIOS 1.1b 09/10/2021
task: ffff8a87b6c3b150 ti: ffff8a87b6e64000 task.ti: ffff8a87b6e64000
RIP: 0010:[<ffffffffba5817c5>] [<ffffffffba5817c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8a87b6e67a70 EFLAGS: 00000246
RAX: 0000000000000246 RBX: ffff8aa7bc5ba728 RCX: ffff8a87b6c3b150
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff8a87b6e67a70 R08: ffffffffc09d0078 R09: ffff8aa7bcf2fe00
R10: ffff8aa7bc5ba728 R11: ffff8aa7bc4ec800 R12: ffff8a87bfe1ac40
R13: ffff8a87b6e67a70 R14: ffff8a87b6e67a80 R15: ffffe1587ff13a00
FS: 00007f5226f13880(0000) GS:ffff8a87bfe00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0257b5d000 CR3: 0000001ff7b36000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb9ec5cb6>] prepare_to_wait+0x56/0x90
[<ffffffffc09a7488>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffffb9ec61c0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc09a8e9b>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0a7650c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0a77cbe>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0a6ec8e>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffba04d1fe>] mount_fs+0x3e/0x1b0
[<ffffffffb9fe1035>] ? __alloc_percpu+0x15/0x20
[<ffffffffba06b377>] vfs_kern_mount+0x67/0x110
[<ffffffffc0a6ebb6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0a6efb4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc09a96d7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc09b5d9b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc09b6d00>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc09b4bb0>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffba04d1fe>] mount_fs+0x3e/0x1b0
[<ffffffffb9fe1035>] ? __alloc_percpu+0x15/0x20
[<ffffffffba06b377>] vfs_kern_mount+0x67/0x110
[<ffffffffba06dacf>] do_mount+0x1ef/0xce0
[<ffffffffba06d800>] ? copy_mount_options+0xc0/0x170
[<ffffffffba06e903>] SyS_mount+0x83/0xd0
[<ffffffffba58bede>] system_call_fastpath+0x25/0x2a
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #789571
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0830375
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa5
|
|
2
|
|
vmlinux |
0xffffffffb02c3d00
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x30
|
|
3
|
nfs_get_client
|
nfs |
0xffffffffc0831eab
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22b
|
|
4
|
nfs4_set_client
|
nfsv4 |
0xffffffffc075d39c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
5
|
nfs4_create_server
|
nfsv4 |
0xffffffffc075eb4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
6
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0755bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
7
|
mount_fs
|
vmlinux |
0xffffffffb0446bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
8
|
|
vmlinux |
0xffffffffb03dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
9
|
vfs_kern_mount
|
vmlinux |
0xffffffffb0464777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
10
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0755ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
11
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0755ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
12
|
|
nfs |
0xffffffffc08326e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
13
|
nfs_fs_mount
|
nfs |
0xffffffffc083ed3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
14
|
|
nfs |
0xffffffffc083fca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
15
|
|
nfs |
0xffffffffc083db40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
16
|
mount_fs
|
vmlinux |
0xffffffffb0446bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
17
|
|
vmlinux |
0xffffffffb03dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
18
|
vfs_kern_mount
|
vmlinux |
0xffffffffb0464777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
19
|
do_mount
|
vmlinux |
0xffffffffb0466d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
20
|
|
vmlinux |
0xffffffffb0466ad0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xc0
|
|
21
|
SyS_mount
|
vmlinux |
0xffffffffb0467bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
22
|
tracesys
|
vmlinux |
0xffffffffb098705b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#45 stuck for 22s! [mount.nfs:13286]
Modules linked in: nfsv3 nfs_acl uinput xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink cmac arc4 md4 nls_utf8 cifs ccm rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod vfat fat sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel
lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt iTCO_vendor_support joydev pcspkr sg i2c_i801 hpwdt lpc_ich hpilo ioatdma dca ipmi_si ipmi_devintf ipmi_msghandler pcc_cpufreq acpi_power_meter wmi auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm crct10dif_pclmul crct10dif_common crc32c_intel libahci libata tg3 drm_panel_orientation_quirks ptp pps_core
CPU: 45 PID: 13286 Comm: mount.nfs Kdump: loaded Tainted: G W L ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff9af025b50000 ti: ffff9aed98a70000 task.ti: ffff9aed98a70000
RIP: 0010:[<ffffffffc0830375>] [<ffffffffc0830375>] nfs_wait_client_init_complete+0xa5/0xe0 [nfs]
RSP: 0018:ffff9aed98a73ab8 EFLAGS: 00000206
RAX: ffffffffc0859078 RBX: 0000000000000206 RCX: 00000000846d03e7
RDX: ffff9ae85cd27ad0 RSI: 0000000000000206 RDI: 0000000000000206
RBP: ffff9aed98a73af8 R08: ffff9aed98a73ad0 R09: ffff9af5ddb6fd00
R10: ffff9aedda6eed28 R11: 0000000000000005 R12: dead000000000200
R13: ffffffffc0859078 R14: ffff9aed98a73ad0 R15: ffff9af5ddb6fd00
FS: 00007f7465cf5880(0000) GS:ffff9aeddfa40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000021 CR3: 00000006a930e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb02c3d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0831eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc075d39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc075eb4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0755bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffb0446bde>] mount_fs+0x3e/0x1b0
[<ffffffffb03dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffb0464777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0755ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0755ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc08326e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc083ed3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc083fca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc083db40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffb0446bde>] mount_fs+0x3e/0x1b0
[<ffffffffb03dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffb0464777>] vfs_kern_mount+0x67/0x110
[<ffffffffb0466d9f>] do_mount+0x1ef/0xce0
[<ffffffffb0466ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffffb0467bd3>] SyS_mount+0x83/0xd0
[<ffffffffb098705b>] tracesys+0xa3/0xc9
Code: 00 00 48 c7 c7 70 90 85 c0 e8 48 34 a9 ef 8b 43 08 85 c0 7f d1 31 db 48 8d 75 c0 48 c7 c7 70 90 85 c0 e8 8f 35 a9 ef 48 8b 4d e8 <65> 48 33 0c 25 28 00 00 00 89 d8 75 12 48 83 c4 30 5b 41 5c 5d
Complete report #767829
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
nfs_get_client
|
nfs |
0xffffffffc075fdb5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x135
|
|
2
|
nfs4_set_client
|
nfsv4 |
0xffffffffc085c39c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
3
|
nfs4_create_server
|
nfsv4 |
0xffffffffc085db4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
4
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0854bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
5
|
mount_fs
|
vmlinux |
0xffffffffa6e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
6
|
|
vmlinux |
0xffffffffa6ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
7
|
vfs_kern_mount
|
vmlinux |
0xffffffffa6e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
8
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0854ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
9
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0854ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
10
|
|
nfs |
0xffffffffc07606e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
11
|
nfs_fs_mount
|
nfs |
0xffffffffc076cd3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
12
|
|
nfs |
0xffffffffc076dca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
13
|
|
nfs |
0xffffffffc076bb40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
14
|
mount_fs
|
vmlinux |
0xffffffffa6e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
15
|
|
vmlinux |
0xffffffffa6ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
16
|
vfs_kern_mount
|
vmlinux |
0xffffffffa6e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
17
|
do_mount
|
vmlinux |
0xffffffffa6e66d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
18
|
|
vmlinux |
0xffffffffa6e66ad0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xc0
|
|
19
|
SyS_mount
|
vmlinux |
0xffffffffa6e67bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
20
|
tracesys
|
vmlinux |
0xffffffffa738705b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#44 stuck for 22s! [mount.nfs:37878]
Modules linked in: binfmt_misc nfsv3 nfs_acl joydev uinput tun devlink cmac arc4 md4 nls_utf8 cifs rpcsec_gss_krb5 nfsv4 ccm dns_resolver nfs lockd grace fscache ip_set nfnetlink bridge stp llc libcrc32c dm_mirror dm_region_hash dm_log dm_mod vfat fat sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt iTCO_vendor_support pcspkr sg hpilo hpwdt i2c_i801 lpc_ich ioatdma ipmi_si ipmi_devintf dca ipmi_msghandler pcc_cpufreq wmi acpi_power_meter auth_rpcgss sunrpc ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul sysimgblt crct10dif_common fb_sys_fops crc32c_intel ttm ahci libahci drm libata
tg3 ptp pps_core drm_panel_orientation_quirks [last unloaded: ip_tables]
CPU: 44 PID: 37878 Comm: mount.nfs Kdump: loaded Tainted: G W ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff8decce3530c0 ti: ffff8dfea2aa4000 task.ti: ffff8dfea2aa4000
RIP: 0010:[<ffffffffc075fdb5>] [<ffffffffc075fdb5>] nfs_get_client+0x135/0x470 [nfs]
RSP: 0018:ffff8dfea2aa7b08 EFLAGS: 00000246
RAX: 0000000000000006 RBX: ffffffffa6cc3956 RCX: 00000000000000d4
RDX: ffff8df279e76860 RSI: ffff8df279e768b8 RDI: ffff8df279e768b8
RBP: ffff8dfea2aa7b60 R08: ffff8dfea2aa7ad0 R09: ffff8df279e76800
R10: ffff8df5b56d9528 R11: 7fffffffffffffff R12: 0000000000000287
R13: 0000000000000287 R14: ffffffffc0787070 R15: ffff8dfea2aa7ad0
FS: 00007f2ce9c8b880(0000) GS:ffff8df27fa00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f023b385650 CR3: 0000001ea83aa000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc085c39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc085db4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0854bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa6e46bde>] mount_fs+0x3e/0x1b0
[<ffffffffa6ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa6e64777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0854ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0854ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc07606e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc076cd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc076dca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc076bb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa6e46bde>] mount_fs+0x3e/0x1b0
[<ffffffffa6ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa6e64777>] vfs_kern_mount+0x67/0x110
[<ffffffffa6e66d9f>] do_mount+0x1ef/0xce0
[<ffffffffa6e66ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffffa6e67bd3>] SyS_mount+0x83/0xd0
[<ffffffffa738705b>] tracesys+0xa3/0xc9
Code: 0f 84 98 00 00 00 41 8b 44 24 08 85 c0 78 df 0f 85 e9 00 00 00 49 8b 47 30 48 8b 40 18 49 39 84 24 e0 00 00 00 75 c7 41 8b 47 38 <41> 39 84 24 e8 00 00 00 75 b9 41 8b 47 3c 41 39 84 24 f8 00 00
Complete report #671473
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
rpc_xprt_switch_has_addr
|
sunrpc |
0xffffffffc247281c
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x10c
|
|
2
|
rpc_clnt_xprt_switch_has_addr
|
sunrpc |
0xffffffffc244e175
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x15
|
|
3
|
nfs_get_client
|
nfs |
0xffffffffc25bae02
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x182
|
|
4
|
nfs4_set_client
|
nfsv4 |
0xffffffffc262437c
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x9c
|
|
5
|
nfs4_create_server
|
nfsv4 |
0xffffffffc2625b2e
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x13e
|
|
6
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc261cbae
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x2e
|
|
7
|
mount_fs
|
vmlinux |
0xffffffff8be4620e
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x3e
|
|
8
|
|
vmlinux |
0xffffffff8bddb445
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x15
|
|
9
|
vfs_kern_mount
|
vmlinux |
0xffffffff8be63da7
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x67
|
|
10
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc261cad6
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x86
|
|
11
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc261ced4
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x44
|
|
12
|
|
nfs |
0xffffffffc25bb6e7
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x27
|
|
13
|
nfs_fs_mount
|
nfs |
0xffffffffc25c7d3b
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x4cb
|
|
14
|
|
nfs |
0xffffffffc25c8ca0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x140
|
|
15
|
|
nfs |
0xffffffffc25c6b40
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x70
|
|
16
|
mount_fs
|
vmlinux |
0xffffffff8be4620e
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x3e
|
|
17
|
|
vmlinux |
0xffffffff8bddb445
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x15
|
|
18
|
vfs_kern_mount
|
vmlinux |
0xffffffff8be63da7
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x67
|
|
19
|
do_mount
|
vmlinux |
0xffffffff8be663cf
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x1ef
|
|
20
|
|
vmlinux |
0xffffffff8be3e66a
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x1ca
|
|
21
|
|
vmlinux |
0xffffffff8be1c9ac
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x3c
|
|
22
|
SyS_mount
|
vmlinux |
0xffffffff8be67203
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x83
|
|
23
|
system_call_fastpath
|
vmlinux |
0xffffffff8c375d9b
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#37 stuck for 23s! [mount.nfs:92996]
Modules linked in: fuse xt_limit nf_log_ipv4 nf_log_common xt_LOG mpt3sas mpt2sas raid_class scsi_transport_sas mptctl mptbase rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper snd_hda_codec_hdmi cryptd iTCO_wdt iTCO_vendor_support nvidia(POE) pcspkr snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm joydev snd_timer ipmi_ssif snd soundcore sg lpc_ich i2c_i801 mei_me mei ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul i40e crct10dif_common crc32c_intel ahci drm libahci libata ptp nvme pps_core nvme_core drm_panel_orientation_quirks wmi nfit libnvdimm
CPU: 37 PID: 92996 Comm: mount.nfs Tainted: P OE ------------ 3.10.0-957.12.1.el7.x86_64 #1
Hardware name: Supermicro SYS-4029GP-TRT/X11DPG-OT-CPU, BIOS 2.1a 10/02/2018
task: ffff89fb1b790000 ti: ffff8a00cf7ec000 task.ti: ffff8a00cf7ec000
RIP: 0010:[<ffffffffc247281c>] [<ffffffffc247281c>] rpc_xprt_switch_has_addr+0x10c/0x190 [sunrpc]
RSP: 0018:ffff8a00cf7efab0 EFLAGS: 00000206
RAX: 00000000a7528796 RBX: 0000000000000246 RCX: 0000000000000002
RDX: ffff8a0b1bfb8660 RSI: ffff89fa85e65b28 RDI: ffff89f2f93f8dc0
RBP: ffff8a00cf7efae8 R08: ffff8a00cf7efad0 R09: ffff8a0b1bfb8600
R10: ffff89fa85e65b28 R11: ffffdd031f5d3600 R12: ffff89fb1b790000
R13: 0000000000000246 R14: ffffffffc25e2078 R15: ffff8a0b1bfb8600
FS: 00002b49b8b66240(0000) GS:ffff8a0b1dd40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f81f94769e5 CR3: 00000023d4e1c000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffc244e175>] rpc_clnt_xprt_switch_has_addr+0x15/0x20 [sunrpc]
[<ffffffffc25bae02>] nfs_get_client+0x182/0x470 [nfs]
[<ffffffffc262437c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc2625b2e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc261cbae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff8be4620e>] mount_fs+0x3e/0x1b0
[<ffffffff8bddb445>] ? __alloc_percpu+0x15/0x20
[<ffffffff8be63da7>] vfs_kern_mount+0x67/0x110
[<ffffffffc261cad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc261ced4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc25bb6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc25c7d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc25c8ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc25c6b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff8be4620e>] mount_fs+0x3e/0x1b0
[<ffffffff8bddb445>] ? __alloc_percpu+0x15/0x20
[<ffffffff8be63da7>] vfs_kern_mount+0x67/0x110
[<ffffffff8be663cf>] do_mount+0x1ef/0xce0
[<ffffffff8be3e66a>] ? __check_object_size+0x1ca/0x250
[<ffffffff8be1c9ac>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff8be67203>] SyS_mount+0x83/0xd0
[<ffffffff8c375d9b>] system_call_fastpath+0x22/0x27
Code: 89 f8 0f 85 9f 00 00 00 48 83 c4 10 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 00 00 00 00 8b 43 1c 41 39 44 24 04 41 0f 94 c7 <45> 84 ff 0f 84 4b ff ff ff 41 0f b7 14 24 66 83 fa 02 74 40 66
Complete report #639889
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_lock
|
vmlinux |
0xffffffffbbf6b470
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x10
|
|
2
|
|
nfs |
0xffffffffc20e5ebb
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x23b
|
|
3
|
nfs4_set_client
|
nfsv4 |
0xffffffffc21ea37c
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x9c
|
|
4
|
nfs4_create_server
|
nfsv4 |
0xffffffffc21ebb2e
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x13e
|
|
5
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc21e2bae
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x2e
|
|
6
|
mount_fs
|
vmlinux |
0xffffffffbba4624e
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x3e
|
|
7
|
|
vmlinux |
0xffffffffbb9db485
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x15
|
|
8
|
vfs_kern_mount
|
vmlinux |
0xffffffffbba63de7
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x67
|
|
9
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc21e2ad6
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x86
|
|
10
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc21e2ed4
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x44
|
|
11
|
|
nfs |
0xffffffffc20e66e7
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x27
|
|
12
|
nfs_fs_mount
|
nfs |
0xffffffffc20f2d3b
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x4cb
|
|
13
|
|
nfs |
0xffffffffc20f3ca0
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x140
|
|
14
|
|
nfs |
0xffffffffc20f1b40
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x70
|
|
15
|
mount_fs
|
vmlinux |
0xffffffffbba4624e
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x3e
|
|
16
|
|
vmlinux |
0xffffffffbb9db485
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x15
|
|
17
|
vfs_kern_mount
|
vmlinux |
0xffffffffbba63de7
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x67
|
|
18
|
do_mount
|
vmlinux |
0xffffffffbba6640f
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x1ef
|
|
19
|
|
vmlinux |
0xffffffffbba66140
Build id: 3.10.0-957.10.1.el7.x86_64
|
0xc0
|
|
20
|
SyS_mount
|
vmlinux |
0xffffffffbba67243
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x83
|
|
21
|
system_call_fastpath
|
vmlinux |
0xffffffffbbf75ddb
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#57 stuck for 23s! [mount.nfs:18727]
Modules linked in: xfs iscsi_tcp libiscsi_tcp geneve ip6_udp_tunnel udp_tunnel vhost_net vhost macvtap macvlan ebt_arp tun rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache fuse bnx2fc cnic uio fcoe libfcoe libfc scsi_transport_fc nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute openvswitch ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter softdog ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp
scsi_tgt ib_ucm iTCO_wdt iTCO_vendor_support nvidia_vgpu_vfio(POE) skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass nvidia(POE) rpcrdma rdma_ucm ib_uverbs snd_hda_codec_realtek opa_vnic ib_iser snd_hda_codec_generic ib_umad intel_wmi_thunderbolt rdma_cm ib_ipoib iw_cm libiscsi joydev ib_cm pcspkr snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq vfio_mdev snd_seq_device mdev vfio_iommu_type1 snd_pcm sg vfio snd_timer snd mei_me lpc_ich i2c_i801 soundcore mei ipmi_si ipmi_devintf ipmi_msghandler acpi_cpufreq acpi_pad acpi_power_meter bridge bonding ip_tables ext4 mbcache jbd2 dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c mlx5_ib 8021q garp mrp stp llc dm_service_time sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common
crc32_pclmul crc32c_intel hfi1 ast ghash_clmulni_intel aesni_intel lrw drm_kms_helper gf128mul glue_helper ablk_helper cryptd mlx5_core syscopyarea sysfillrect sysimgblt fb_sys_fops ttm rdmavt ixgbe drm mlxfw ib_core scsi_transport_iscsi devlink ahci libahci i2c_algo_bit libata dca drm_panel_orientation_quirks ptp pps_core mdio dm_multipath wmi nfit libnvdimm sunrpc dm_mirror dm_region_hash dm_log dm_mod
CPU: 57 PID: 18727 Comm: mount.nfs Kdump: loaded Tainted: P OE ------------ 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: Supermicro SYS-7049GP-TRT/X11DPG-QT, BIOS 2.1 07/20/2018
task: ffff91de8129d140 ti: ffff91e902344000 task.ti: ffff91e902344000
RIP: 0010:[<ffffffffbbf6b470>] [<ffffffffbbf6b470>] _raw_spin_lock+0x10/0x30
RSP: 0018:ffff91e902347b00 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffbb8c2996 RCX: 0000000000000002
RDX: 0000000000000001 RSI: ffff922222a579b8 RDI: ffff922222a579b8
RBP: ffff91e902347b60 R08: ffff91e902347ad0 R09: ffff922222a57900
R10: ffff9221e964b328 R11: 0000000000000005 R12: 0000000000000293
R13: 0000000000000293 R14: ffffffffc210d070 R15: ffff91e902347ad0
FS: 00007f1ad974f880(0000) GS:ffff91f2be340000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055abd5c27000 CR3: 00000024072e4000 CR4: 00000000007627e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffc20e5ebb>] ? nfs_get_client+0x23b/0x470 [nfs]
[<ffffffffc21ea37c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc21ebb2e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc21e2bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffbba4624e>] mount_fs+0x3e/0x1b0
[<ffffffffbb9db485>] ? __alloc_percpu+0x15/0x20
[<ffffffffbba63de7>] vfs_kern_mount+0x67/0x110
[<ffffffffc21e2ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc21e2ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc20e66e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc20f2d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc20f3ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc20f1b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffbba4624e>] mount_fs+0x3e/0x1b0
[<ffffffffbb9db485>] ? __alloc_percpu+0x15/0x20
[<ffffffffbba63de7>] vfs_kern_mount+0x67/0x110
[<ffffffffbba6640f>] do_mount+0x1ef/0xce0
[<ffffffffbba66140>] ? copy_mount_options+0xc0/0x170
[<ffffffffbba67243>] SyS_mount+0x83/0xd0
[<ffffffffbbf75ddb>] system_call_fastpath+0x22/0x27
Code: 44 00 00 85 d2 74 e4 0f 1f 40 00 eb ed 66 0f 1f 44 00 00 b8 01 00 00 00 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17 <85> c0 75 01 c3 55 89 c6 48 89 e5 e8 40 1b ff ff 5d c3 0f 1f 40
Complete report #587793
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
rpc_clnt_xprt_switch_has_addr
|
sunrpc |
0xffffffffc0696160
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
-
|
|
2
|
|
nfs |
0xffffffffc096be02
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x182
|
|
3
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0a3c39c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
4
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0a3db4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
5
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0a34bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
6
|
mount_fs
|
vmlinux |
0xffffffff82646bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
7
|
|
vmlinux |
0xffffffff825dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
8
|
vfs_kern_mount
|
vmlinux |
0xffffffff82664777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
9
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0a34ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
10
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0a34ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
11
|
|
nfs |
0xffffffffc096c6e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
12
|
nfs_fs_mount
|
nfs |
0xffffffffc0978d3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
13
|
|
nfs |
0xffffffffc0979ca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
14
|
|
nfs |
0xffffffffc0977b40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
15
|
mount_fs
|
vmlinux |
0xffffffff82646bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
16
|
|
vmlinux |
0xffffffff825dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
17
|
vfs_kern_mount
|
vmlinux |
0xffffffff82664777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
18
|
do_mount
|
vmlinux |
0xffffffff82666d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
19
|
|
vmlinux |
0xffffffff82666ad0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xc0
|
|
20
|
SyS_mount
|
vmlinux |
0xffffffff82667bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
21
|
system_call_fastpath
|
vmlinux |
0xffffffff82b86ddb
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#42 stuck for 22s! [mount.nfs:23462]
Modules linked in: joydev uinput xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nfsv3 nfs_acl cmac arc4 md4 rpcsec_gss_krb5 nfsv4 nls_utf8 cifs nfs ccm dns_resolver lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul vfat fat iTCO_wdt iTCO_vendor_support
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg hpilo hpwdt lpc_ich i2c_i801 ipmi_si ipmi_devintf ipmi_msghandler pcc_cpufreq ioatdma dca acpi_power_meter wmi auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 crct10dif_pclmul crct10dif_common crc32c_intel i2c_algo_bit drm_kms_helper ahci syscopyarea libahci sysfillrect sysimgblt fb_sys_fops tg3 libata ttm ptp pps_core drm drm_panel_orientation_quirks
CPU: 42 PID: 23462 Comm: mount.nfs Kdump: loaded Not tainted 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff94fddc3ad140 ti: ffff94fdaf758000 task.ti: ffff94fdaf758000
RIP: 0010:[<ffffffffc0696160>] [<ffffffffc0696160>] rpc_clnt_xprt_switch_has_addr+0x0/0x20 [sunrpc]
RSP: 0018:ffff94fdaf75bb00 EFLAGS: 00000287
RAX: 00000000c4016400 RBX: ffffffff824c3956 RCX: 0000000000000002
RDX: ffff9505d46e1f60 RSI: ffff9505a9785528 RDI: ffff94fdd8288c00
RBP: ffff94fdaf75bb60 R08: ffff94fdaf75bad0 R09: ffff9505d46e1f00
R10: ffff9505a9785528 R11: 0000000000000005 R12: 0000000000000293
R13: 0000000000000293 R14: ffffffffc0993070 R15: ffff94fdaf75bad0
FS: 00007fbb45e64880(0000) GS:ffff94fddf980000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f956a3379b0 CR3: 000000082f7f4000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc096be02>] ? nfs_get_client+0x182/0x470 [nfs]
[<ffffffffc0a3c39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0a3db4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0a34bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff82646bde>] mount_fs+0x3e/0x1b0
[<ffffffff825dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff82664777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0a34ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0a34ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc096c6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0978d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0979ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0977b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff82646bde>] mount_fs+0x3e/0x1b0
[<ffffffff825dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff82664777>] vfs_kern_mount+0x67/0x110
[<ffffffff82666d9f>] do_mount+0x1ef/0xce0
[<ffffffff82666ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffff82667bd3>] SyS_mount+0x83/0xd0
[<ffffffff82b86ddb>] system_call_fastpath+0x22/0x27
Code: 48 8b 93 98 05 00 00 44 89 fe 48 c7 c7 70 29 6c c0 31 c0 e8 ba 74 4d c2 44 89 f8 eb be 41 89 c7 eb ce 66 0f 1f 84 00 00 00 00 00 <0f> 1f 44 00 00 55 48 89 e5 48 8b bf 90 01 00 00 e8 8b 45 02 00
Complete report #587781
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffb257c0f5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
2
|
prepare_to_wait
|
vmlinux |
0xffffffffb1ec37f6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0807358
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x88
|
|
4
|
|
vmlinux |
0xffffffffb1ec3d00
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc0808eab
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc08ad39c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc08aeb4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc08a5bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffffb2046bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffffb1fdbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffffb2064777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc08a5ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc08a5ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc08096e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc0815d3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc0816ca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc0814b40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffffb2046bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffffb1fdbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffffb2064777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffffb2066d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffffb2066ad0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xc0
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffffb2067bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
24
|
tracesys
|
vmlinux |
0xffffffffb258705b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#68 stuck for 22s! [mount.nfs:12752]
Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) cmac arc4 md4 nfsv3 nfs_acl nls_utf8 cifs ccm rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod vfat fat sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt iTCO_vendor_support pcspkr sg i2c_i801 lpc_ich hpwdt hpilo ipmi_si ipmi_devintf ipmi_msghandler ioatdma dca wmi pcc_cpufreq acpi_power_meter auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt crct10dif_pclmul fb_sys_fops ahci crct10dif_common crc32c_intel ttm libahci drm libata tg3 ptp pps_core drm_panel_orientation_quirks
CPU: 68 PID: 12752 Comm: mount.nfs Tainted: G W L ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff8c313fa86180 ti: ffff8c30cc794000 task.ti: ffff8c30cc794000
RIP: 0010:[<ffffffffb257c0f5>] [<ffffffffb257c0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8c30cc797a70 EFLAGS: 00000246
RAX: 0000000000000246 RBX: ffff8c311d3ebb28 RCX: ffff8c313fa86180
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff8c30cc797a70 R08: ffff8c30cc317ad0 R09: ffff8c311f35b7c0
R10: ffff8c311d3ebb28 R11: 0000000000000000 R12: ffff8c311f31abc0
R13: ffff8c30cc797a70 R14: ffff8c313fa86818 R15: 00000001cc7979d8
FS: 00007fa36b8a8880(0000) GS:ffff8c311f300000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000562cf564a000 CR3: 0000000ff2f6a000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb1ec37f6>] prepare_to_wait+0x56/0x90
[<ffffffffc0807358>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffffb1ec3d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0808eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc08ad39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc08aeb4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc08a5bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffb2046bde>] mount_fs+0x3e/0x1b0
[<ffffffffb1fdbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffb2064777>] vfs_kern_mount+0x67/0x110
[<ffffffffc08a5ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc08a5ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc08096e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0815d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0816ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0814b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffb2046bde>] mount_fs+0x3e/0x1b0
[<ffffffffb1fdbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffb2064777>] vfs_kern_mount+0x67/0x110
[<ffffffffb2066d9f>] do_mount+0x1ef/0xce0
[<ffffffffb2066ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffffb2067bd3>] SyS_mount+0x83/0xd0
[<ffffffffb258705b>] tracesys+0xa3/0xc9
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #581071
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc07ad343
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x73
|
|
2
|
|
vmlinux |
0xffffffffa52c3d00
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x30
|
|
3
|
nfs_get_client
|
nfs |
0xffffffffc07aeeab
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22b
|
|
4
|
nfs4_set_client
|
nfsv4 |
0xffffffffc072339c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
5
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0724b4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
6
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc071bbae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
7
|
mount_fs
|
vmlinux |
0xffffffffa5446bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
8
|
|
vmlinux |
0xffffffffa53dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
9
|
vfs_kern_mount
|
vmlinux |
0xffffffffa5464777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
10
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc071bad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
11
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc071bed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
12
|
|
nfs |
0xffffffffc07af6e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
13
|
nfs_fs_mount
|
nfs |
0xffffffffc07bbd3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
14
|
|
nfs |
0xffffffffc07bcca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
15
|
|
nfs |
0xffffffffc07bab40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
16
|
mount_fs
|
vmlinux |
0xffffffffa5446bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
17
|
|
vmlinux |
0xffffffffa53dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
18
|
vfs_kern_mount
|
vmlinux |
0xffffffffa5464777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
19
|
do_mount
|
vmlinux |
0xffffffffa5466d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
20
|
|
vmlinux |
0xffffffffa5466ad0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xc0
|
|
21
|
SyS_mount
|
vmlinux |
0xffffffffa5467bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
22
|
system_call_fastpath
|
vmlinux |
0xffffffffa5986ddb
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [mount.nfs:57222]
Modules linked in: joydev uinput xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nfsv3 nfs_acl cmac rpcsec_gss_krb5 arc4 md4 nfsv4 nls_utf8 cifs nfs lockd grace ccm fscache dns_resolver ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass vfat crc32_pclmul fat iTCO_wdt iTCO_vendor_support
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg lpc_ich i2c_i801 hpilo hpwdt ipmi_si ipmi_devintf ipmi_msghandler ioatdma pcc_cpufreq dca wmi acpi_power_meter auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper ahci crct10dif_pclmul crct10dif_common crc32c_intel libahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata tg3 drm ptp pps_core drm_panel_orientation_quirks
CPU: 6 PID: 57222 Comm: mount.nfs Kdump: loaded Not tainted 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff9c661a54c100 ti: ffff9c660e378000 task.ti: ffff9c660e378000
RIP: 0010:[<ffffffffc07ad343>] [<ffffffffc07ad343>] nfs_wait_client_init_complete+0x73/0xe0 [nfs]
RSP: 0018:ffff9c660e37bab8 EFLAGS: 00000202
RAX: ffff9c661a54c100 RBX: 0000000000000246 RCX: ffff9c660e37bab8
RDX: ffff9c660e37bad0 RSI: 0000000000000000 RDI: ffff9c5e1bbb8c00
RBP: ffff9c660e37baf8 R08: ffff9c660e37bad0 R09: ffff9c65fd27f200
R10: ffff9c661beddd28 R11: 0000000000000005 R12: ffff9c661a54c100
R13: 0000000000000246 R14: ffff9c663ff0fad0 R15: ffff9c65fd27f200
FS: 00007f830e673880(0000) GS:ffff9c5e1f580000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8c29826000 CR3: 000000102c2e0000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa52c3d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc07aeeab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc072339c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0724b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc071bbae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa5446bde>] mount_fs+0x3e/0x1b0
[<ffffffffa53dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa5464777>] vfs_kern_mount+0x67/0x110
[<ffffffffc071bad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc071bed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc07af6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc07bbd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc07bcca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc07bab40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa5446bde>] mount_fs+0x3e/0x1b0
[<ffffffffa53dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa5464777>] vfs_kern_mount+0x67/0x110
[<ffffffffa5466d9f>] do_mount+0x1ef/0xce0
[<ffffffffa5466ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffffa5467bd3>] SyS_mount+0x83/0xd0
[<ffffffffa5986ddb>] system_call_fastpath+0x22/0x27
Code: 01 00 48 8d 51 18 48 89 45 c8 49 89 c4 48 89 55 d8 48 89 55 e0 eb 14 90 49 8b 54 24 08 48 8b 52 10 83 e2 04 75 5b e8 5d c8 1c e5 <48> 8d 75 c0 ba 82 00 00 00 48 c7 c7 70 60 7d c0 e8 48 64 b1 e4
Complete report #575844
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffa7b7c0f5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
2
|
finish_wait
|
vmlinux |
0xffffffffa74c3956
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc2fb3371
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa1
|
|
4
|
|
vmlinux |
0xffffffffa74c3d00
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc2fb4eab
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc303839c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc3039b4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc3030bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffffa7646bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffffa75dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffffa7664777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc3030ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc3030ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc2fb56e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc2fc1d3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc2fc2ca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc2fc0b40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffffa7646bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffffa75dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffffa7664777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffffa7666d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffffa7666ad0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xc0
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffffa7667bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffffa7b86ddb
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [mount.nfs:8598]
Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) nfsv3 nfs_acl cmac arc4 md4 nls_utf8 cifs rpcsec_gss_krb5 nfsv4 nfs ccm dns_resolver lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) nvidia(POE) vfat fat iTCO_wdt iTCO_vendor_support
mei_wdt hp_wmi sparse_keymap rfkill sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel snd_hda_codec_hdmi kvm irqbypass crc32_pclmul snd_hda_codec_realtek ghash_clmulni_intel snd_hda_codec_generic aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr serio_raw snd_hda_intel snd_hda_codec e1000e(OE) snd_hda_core drm_kms_helper snd_hwdep snd_seq sg snd_seq_device i2c_i801 lpc_ich snd_pcm drm snd_timer mei_me snd mei drm_panel_orientation_quirks syscopyarea ptp sysfillrect soundcore sysimgblt pps_core fb_sys_fops tpm_infineon wmi auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod sr_mod crc_t10dif crct10dif_generic cdrom crct10dif_pclmul crct10dif_common crc32c_intel ahci libahci libata
CPU: 7 PID: 8598 Comm: mount.nfs Tainted: P W OEL ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: Hewlett-Packard HP Z440 Workstation/212B, BIOS M60 v02.23 07/22/2016
task: ffff8b1ff03430c0 ti: ffff8b1fd3b68000 task.ti: ffff8b1fd3b68000
RIP: 0010:[<ffffffffa7b7c0f5>] [<ffffffffa7b7c0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8b1fd3b6ba78 EFLAGS: 00000206
RAX: ffff8b1fc820bad0 RBX: ffff8b202fee3f28 RCX: dead000000000200
RDX: ffff8b18c78bfad0 RSI: 0000000000000206 RDI: 0000000000000206
RBP: ffff8b1fd3b6ba78 R08: ffff8b1fd3b6bad0 R09: ffff8b200c6db7c0
R10: ffff8b202fee3f28 R11: 0000000000000001 R12: ffffffffc02cc25e
R13: ffff8b1fd3b6ba70 R14: 0000000000000000 R15: ffff8b1fd3b6ba18
FS: 00007f5baa9aa880(0000) GS:ffff8b200c7c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa73b173046 CR3: 00000007c621a000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa74c3956>] finish_wait+0x56/0x70
[<ffffffffc2fb3371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffffa74c3d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc2fb4eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc303839c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc3039b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc3030bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa7646bde>] mount_fs+0x3e/0x1b0
[<ffffffffa75dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa7664777>] vfs_kern_mount+0x67/0x110
[<ffffffffc3030ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc3030ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc2fb56e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc2fc1d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc2fc2ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc2fc0b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa7646bde>] mount_fs+0x3e/0x1b0
[<ffffffffa75dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa7664777>] vfs_kern_mount+0x67/0x110
[<ffffffffa7666d9f>] do_mount+0x1ef/0xce0
[<ffffffffa7666ad0>] ? copy_mount_options+0xc0/0x170
[<ffffffffa7667bd3>] SyS_mount+0x83/0xd0
[<ffffffffa7b86ddb>] system_call_fastpath+0x22/0x27
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #522295
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_lock
|
vmlinux |
0xffffffff9c56a3f0
Build id: 3.10.0-957.el7.x86_64
|
0x10
|
|
2
|
|
nfs |
0xffffffffc0a40ebb
Build id: 3.10.0-957.el7.x86_64
|
0x23b
|
|
3
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0ac439c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
4
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0ac5b4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
5
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0abcbae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
6
|
mount_fs
|
vmlinux |
0xffffffff9c045bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
7
|
|
vmlinux |
0xffffffff9bfdaff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
8
|
vfs_kern_mount
|
vmlinux |
0xffffffff9c063777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
9
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0abcad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
10
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0abced4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
11
|
|
nfs |
0xffffffffc0a416e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
12
|
nfs_fs_mount
|
nfs |
0xffffffffc0a4dd3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
13
|
|
nfs |
0xffffffffc0a4eca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
14
|
|
nfs |
0xffffffffc0a4cb40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
15
|
mount_fs
|
vmlinux |
0xffffffff9c045bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
16
|
|
vmlinux |
0xffffffff9bfdaff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
17
|
vfs_kern_mount
|
vmlinux |
0xffffffff9c063777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
18
|
do_mount
|
vmlinux |
0xffffffff9c065d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
19
|
|
vmlinux |
0xffffffff9c065ad5
Build id: 3.10.0-957.el7.x86_64
|
0xc5
|
|
20
|
SyS_mount
|
vmlinux |
0xffffffff9c066bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
21
|
system_call_fastpath
|
vmlinux |
0xffffffff9c574ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#55 stuck for 23s! [mount.nfs4:42240]
Modules linked in: nfsd mgc(OE) lustre(OE) lmv(OE) mdc(OE) fid(OE) osc(OE) lov(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) tcp_lp nfsv3 nfs_acl fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter devlink ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm i40iw rpcrdma sunrpc rdma_ucm ib_uverbs ib_iser opa_vnic ib_umad rdma_cm iw_cm ib_ipoib libiscsi scsi_transport_iscsi ib_cm squashfs loop iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp
coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd hfi1 rdmavt ib_core joydev pcspkr sg mei_me i2c_i801 lpc_ich mei ipmi_si ipmi_devintf ipmi_msghandler wmi acpi_pad acpi_power_meter sch_fq_codel binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast drm_kms_helper syscopyarea sysfillrect i40e sysimgblt fb_sys_fops ttm igb drm ptp pps_core megaraid_sas dca ahci i2c_algo_bit libahci drm_panel_orientation_quirks libata nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 55 PID: 42240 Comm: mount.nfs4 Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019
task: ffffa14bf51cd140 ti: ffffa17a87d7c000 task.ti: ffffa17a87d7c000
RIP: 0010:[<ffffffff9c56a3f0>] [<ffffffff9c56a3f0>] _raw_spin_lock+0x10/0x30
RSP: 0018:ffffa17a87d7fb00 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff9bec2956 RCX: 0000000000000003
RDX: 0000000000000001 RSI: ffffa17af1eddfb8 RDI: ffffa17af1eddfb8
RBP: ffffa17a87d7fb60 R08: ffffa17a87d7fad0 R09: ffffa17af1eddf00
R10: ffffa17a87ec6728 R11: 0000000000000005 R12: 0000000000000293
R13: 0000000000000293 R14: ffffffffc0a68070 R15: ffffa17a87d7fad0
FS: 00002aed3ca61f80(0000) GS:ffffa17afc2c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002aed3d141038 CR3: 0000005f0cff0000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffc0a40ebb>] ? nfs_get_client+0x23b/0x470 [nfs]
[<ffffffffc0ac439c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0ac5b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0abcbae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff9c045bde>] mount_fs+0x3e/0x1b0
[<ffffffff9bfdaff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9c063777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0abcad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0abced4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0a416e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0a4dd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0a4eca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0a4cb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff9c045bde>] mount_fs+0x3e/0x1b0
[<ffffffff9bfdaff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9c063777>] vfs_kern_mount+0x67/0x110
[<ffffffff9c065d9f>] do_mount+0x1ef/0xce0
[<ffffffff9c065ad5>] ? copy_mount_options+0xc5/0x170
[<ffffffff9c066bd3>] SyS_mount+0x83/0xd0
[<ffffffff9c574ddb>] system_call_fastpath+0x22/0x27
Code: 44 00 00 85 d2 74 e4 0f 1f 40 00 eb ed 66 0f 1f 44 00 00 b8 01 00 00 00 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17 <85> c0 75 01 c3 55 89 c6 48 89 e5 e8 40 1b ff ff 5d c3 0f 1f 40
Complete report #232902
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff9756b455
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x15
|
|
2
|
finish_wait
|
vmlinux |
0xffffffff96ec2ac6
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc080f371
Build id: 3.10.0-957.21.3.el7.x86_64
|
0xa1
|
|
4
|
|
vmlinux |
0xffffffff96ec2e70
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc0810eab
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc1bbf37c
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc1bc0b2e
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc1bb7bae
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff970463ce
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff96fdb605
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff97063ff7
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc1bb7ad6
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc1bb7ed4
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc08116e7
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc081dd3b
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc081eca0
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc081cb40
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff970463ce
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff96fdb605
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff97063ff7
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff9706661f
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffff97066350
Build id: 3.10.0-957.21.3.el7.x86_64
|
0xc0
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffff97067453
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffff97575ddb
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [mount.nfs:1542]
Modules linked in: devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) vfat fat intel_pmc_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm snd_hda_codec_hdmi irqbypass ppdev iTCO_wdt mei_wdt crc32_pclmul iTCO_vendor_support ghash_clmulni_intel aesni_intel snd_hda_codec_realtek
lrw gf128mul glue_helper ablk_helper cryptd snd_hda_codec_generic joydev snd_hda_intel wacom snd_hda_codec drm_kms_helper sg snd_hda_core syscopyarea pcspkr sysfillrect snd_hwdep sysimgblt snd_seq fb_sys_fops intel_wmi_thunderbolt drm snd_seq_device snd_pcm parport_pc parport snd_timer drm_panel_orientation_quirks ipmi_devintf ipmi_msghandler snd pinctrl_sunrisepoint pinctrl_intel soundcore tpm_infineon acpi_pad mei_me mei i2c_i801 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ahci crct10dif_pclmul crct10dif_common e1000e libahci libata crc32c_intel serio_raw ptp pps_core wmi video i2c_hid dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1542 Comm: mount.nfs Tainted: P OE ------------ 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./B150M-D3H-CF, BIOS F6 03/14/2016
task: ffff8f5b4ffee180 ti: ffff8f593f670000 task.ti: ffff8f593f670000
RIP: 0010:[<ffffffff9756b455>] [<ffffffff9756b455>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8f593f673a78 EFLAGS: 00000287
RAX: ffffffffc0838078 RBX: ffff8f5a7cbdb128 RCX: dead000000000200
RDX: ffff8f5b67b37ad0 RSI: 0000000000000287 RDI: 0000000000000287
RBP: ffff8f593f673a78 R08: ffff8f593f673ad0 R09: ffff8f597c80e700
R10: ffff8f5a7cbdb128 R11: 0000000000000001 R12: ffff8f5b7541ab80
R13: ffff8f593f673a78 R14: ffff8f5b4ffee818 R15: 0000000163a53128
FS: 00007f87c28ea880(0000) GS:ffff8f5b75400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdb7b51e000 CR3: 000000003f2ec000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff96ec2ac6>] finish_wait+0x56/0x70
[<ffffffffc080f371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffff96ec2e70>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0810eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc1bbf37c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc1bc0b2e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc1bb7bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff970463ce>] mount_fs+0x3e/0x1b0
[<ffffffff96fdb605>] ? __alloc_percpu+0x15/0x20
[<ffffffff97063ff7>] vfs_kern_mount+0x67/0x110
[<ffffffffc1bb7ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc1bb7ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc08116e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc081dd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc081eca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc081cb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff970463ce>] mount_fs+0x3e/0x1b0
[<ffffffff96fdb605>] ? __alloc_percpu+0x15/0x20
[<ffffffff97063ff7>] vfs_kern_mount+0x67/0x110
[<ffffffff9706661f>] do_mount+0x1ef/0xce0
[<ffffffff97066350>] ? copy_mount_options+0xc0/0x170
[<ffffffff97067453>] SyS_mount+0x83/0xd0
[<ffffffff97575ddb>] system_call_fastpath+0x22/0x27
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #230834
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff8176b45a
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x1a
|
|
2
|
prepare_to_wait
|
vmlinux |
0xffffffff810c2966
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc1b56358
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x88
|
|
4
|
|
vmlinux |
0xffffffff810c2e70
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc1b57eab
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc1bc137c
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc1bc2b2e
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc1bb9bae
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff812463ce
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff811db605
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff81263ff7
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc1bb9ad6
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc1bb9ed4
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc1b586e7
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc1b64d3b
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc1b65ca0
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc1b63b40
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff812463ce
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff811db605
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff81263ff7
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff8126661f
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffff81266350
Build id: 3.10.0-957.21.3.el7.x86_64
|
0xc0
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffff81267453
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffff81775ddb
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [mount.nfs:14166]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink fuse rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) snd_hda_codec_hdmi intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm iTCO_wdt irqbypass iTCO_vendor_support crc32_pclmul ghash_clmulni_intel
snd_hda_codec_realtek snd_hda_codec_generic drm_kms_helper aesni_intel lrw gf128mul glue_helper ablk_helper cryptd syscopyarea snd_hda_intel sysfillrect sysimgblt fb_sys_fops snd_hda_codec drm snd_hda_core snd_hwdep snd_seq joydev wacom snd_seq_device sg snd_pcm i2c_i801 pcspkr snd_timer drm_panel_orientation_quirks ipmi_devintf snd ipmi_msghandler nuvoton_cir mei_me rc_core soundcore mei lpc_ich ie31200_edac ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ahci libahci libata r8169 crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mii video dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 14166 Comm: mount.nfs Kdump: loaded Tainted: P OE ------------ 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./B85M-HDS R2.0, BIOS P1.20 07/27/2015
task: ffff88f7c0762080 ti: ffff88f8e6f20000 task.ti: ffff88f8e6f20000
RIP: 0010:[<ffffffff8176b45a>] [<ffffffff8176b45a>] _raw_spin_unlock_irqrestore+0x1a/0x20
RSP: 0018:ffff88f8e6f23a70 EFLAGS: 00000246
RAX: 0000000000000246 RBX: ffff88f7ba5d6328 RCX: ffff88f7c0762080
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff88f8e6f23a70 R08: ffff88f7a7263ad0 R09: ffff88f915028700
R10: ffff88f7ba5d6328 R11: 0000000000000000 R12: ffff88f91ec9ab80
R13: ffff88f8e6f23a78 R14: ffff88f7c0762718 R15: 00000001810e561f
FS: 00007f63e179b880(0000) GS:ffff88f91ec80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff0a02f2000 CR3: 00000001bff94000 CR4: 00000000001607e0
Call Trace:
[<ffffffff810c2966>] prepare_to_wait+0x56/0x90
[<ffffffffc1b56358>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffff810c2e70>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc1b57eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc1bc137c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc1bc2b2e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc1bb9bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff812463ce>] mount_fs+0x3e/0x1b0
[<ffffffff811db605>] ? __alloc_percpu+0x15/0x20
[<ffffffff81263ff7>] vfs_kern_mount+0x67/0x110
[<ffffffffc1bb9ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc1bb9ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc1b586e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc1b64d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc1b65ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc1b63b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff812463ce>] mount_fs+0x3e/0x1b0
[<ffffffff811db605>] ? __alloc_percpu+0x15/0x20
[<ffffffff81263ff7>] vfs_kern_mount+0x67/0x110
[<ffffffff8126661f>] do_mount+0x1ef/0xce0
[<ffffffff81266350>] ? copy_mount_options+0xc0/0x170
[<ffffffff81267453>] SyS_mount+0x83/0xd0
[<ffffffff81775ddb>] system_call_fastpath+0x22/0x27
Code: 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d 0f 1f 44 00 00 <5d> c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 c7 c2 ff ff ff
Complete report #223969
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc094a4a1
Build id: 3.10.0-1062.el7.x86_64
|
0xa1
|
|
2
|
|
vmlinux |
0xffffffffa5ac61c0
Build id: 3.10.0-1062.el7.x86_64
|
0x30
|
|
3
|
nfs_get_client
|
nfs |
0xffffffffc094be9b
Build id: 3.10.0-1062.el7.x86_64
|
0x22b
|
|
4
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0d8450c
Build id: 3.10.0-1062.el7.x86_64
|
0x9c
|
|
5
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0d85cbe
Build id: 3.10.0-1062.el7.x86_64
|
0x13e
|
|
6
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0d7cc8e
Build id: 3.10.0-1062.el7.x86_64
|
0x2e
|
|
7
|
mount_fs
|
vmlinux |
0xffffffffa5c4d1fe
Build id: 3.10.0-1062.el7.x86_64
|
0x3e
|
|
8
|
|
vmlinux |
0xffffffffa5be1035
Build id: 3.10.0-1062.el7.x86_64
|
0x15
|
|
9
|
vfs_kern_mount
|
vmlinux |
0xffffffffa5c6b377
Build id: 3.10.0-1062.el7.x86_64
|
0x67
|
|
10
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0d7cbb6
Build id: 3.10.0-1062.el7.x86_64
|
0x86
|
|
11
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0d7cfb4
Build id: 3.10.0-1062.el7.x86_64
|
0x44
|
|
12
|
|
nfs |
0xffffffffc094c6d7
Build id: 3.10.0-1062.el7.x86_64
|
0x27
|
|
13
|
nfs_fs_mount
|
nfs |
0xffffffffc0958d9b
Build id: 3.10.0-1062.el7.x86_64
|
0x4cb
|
|
14
|
|
nfs |
0xffffffffc0959d00
Build id: 3.10.0-1062.el7.x86_64
|
0x140
|
|
15
|
|
nfs |
0xffffffffc0957bb0
Build id: 3.10.0-1062.el7.x86_64
|
0x70
|
|
16
|
mount_fs
|
vmlinux |
0xffffffffa5c4d1fe
Build id: 3.10.0-1062.el7.x86_64
|
0x3e
|
|
17
|
|
vmlinux |
0xffffffffa5be1035
Build id: 3.10.0-1062.el7.x86_64
|
0x15
|
|
18
|
vfs_kern_mount
|
vmlinux |
0xffffffffa5c6b377
Build id: 3.10.0-1062.el7.x86_64
|
0x67
|
|
19
|
do_mount
|
vmlinux |
0xffffffffa5c6dacf
Build id: 3.10.0-1062.el7.x86_64
|
0x1ef
|
|
20
|
|
vmlinux |
0xffffffffa5c6d800
Build id: 3.10.0-1062.el7.x86_64
|
0xc0
|
|
21
|
SyS_mount
|
vmlinux |
0xffffffffa5c6e903
Build id: 3.10.0-1062.el7.x86_64
|
0x83
|
|
22
|
system_call_fastpath
|
vmlinux |
0xffffffffa618bede
Build id: 3.10.0-1062.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#21 stuck for 22s! [mount.nfs:3616]
Modules linked in: nf_conntrack_netlink xt_addrtype xt_CHECKSUM br_netfilter ipt_MASQUERADE nf_nat_masquerade_ipv4 tun bridge stp llc devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc nvidia_drm(POE) nvidia_modeset(POE) iTCO_wdt iTCO_vendor_support nvidia(POE) snd_hda_codec_hdmi sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm
snd_hda_codec_realtek snd_hda_codec_generic irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel snd_hda_intel lrw gf128mul snd_hda_codec glue_helper ablk_helper cryptd intel_wmi_thunderbolt pcspkr snd_hda_core snd_seq snd_hwdep snd_seq_device i2c_i801 snd_pcm lpc_ich joydev mei_me snd_timer mei sg snd ioatdma soundcore ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad pcc_cpufreq ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ast drm_kms_helper mxm_wmi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm igb libahci crct10dif_pclmul crct10dif_common libata crc32c_intel ptp pps_core dca i2c_algo_bit drm_panel_orientation_quirks nfit libnvdimm wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 21 PID: 3616 Comm: mount.nfs Kdump: loaded Tainted: P OE ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: Supermicro SYS-7048GR-TR/X10DRG-Q, BIOS 2.0b 07/26/2017
task: ffff97173ff33150 ti: ffff97173ff1c000 task.ti: ffff97173ff1c000
RIP: 0010:[<ffffffffc094a4a1>] [<ffffffffc094a4a1>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
RSP: 0018:ffff97173ff1fab8 EFLAGS: 00000206
RAX: ffffffffc0973078 RBX: 0000000000000206 RCX: dead000000000200
RDX: ffff96ff54edbad0 RSI: 0000000000000206 RDI: 0000000000000206
RBP: ffff97173ff1faf8 R08: ffff97173ff1fad0 R09: ffff96ff514d3300
R10: ffff97174177e528 R11: ffffee4d0025e000 R12: dead000000000200
R13: ffffffffc0973078 R14: ffff97173ff1fad0 R15: ffff96ff514d3300
FS: 00007fd16df36880(0000) GS:ffff97175f040000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe4c169f5b0 CR3: 0000003005aa2000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa5ac61c0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc094be9b>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0d8450c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0d85cbe>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0d7cc8e>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa5c4d1fe>] mount_fs+0x3e/0x1b0
[<ffffffffa5be1035>] ? __alloc_percpu+0x15/0x20
[<ffffffffa5c6b377>] vfs_kern_mount+0x67/0x110
[<ffffffffc0d7cbb6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0d7cfb4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc094c6d7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0958d9b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0959d00>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0957bb0>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa5c4d1fe>] mount_fs+0x3e/0x1b0
[<ffffffffa5be1035>] ? __alloc_percpu+0x15/0x20
[<ffffffffa5c6b377>] vfs_kern_mount+0x67/0x110
[<ffffffffa5c6dacf>] do_mount+0x1ef/0xce0
[<ffffffffa5c6d800>] ? copy_mount_options+0xc0/0x170
[<ffffffffa5c6e903>] SyS_mount+0x83/0xd0
[<ffffffffa618bede>] system_call_fastpath+0x25/0x2a
Code: c0 ba 82 00 00 00 48 c7 c7 70 30 97 c0 e8 d8 b7 17 e5 8b 43 08 85 c0 7f d1 31 db 48 8d 75 c0 48 c7 c7 70 30 97 c0 e8 1f b9 17 e5 <48> 8b 4d e8 65 48 33 0c 25 28 00 00 00 89 d8 75 12 48 83 c4 30
Complete report #37700
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff8cb6a0f5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
2
|
prepare_to_wait
|
vmlinux |
0xffffffff8c4c27f6
Build id: 3.10.0-957.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0938358
Build id: 3.10.0-957.el7.x86_64
|
0x88
|
|
4
|
|
vmlinux |
0xffffffff8c4c2d00
Build id: 3.10.0-957.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc0939eab
Build id: 3.10.0-957.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc09a339c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc09a4b4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc099bbae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff8c645bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff8c5daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff8c663777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc099bad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc099bed4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc093a6e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc0946d3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc0947ca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc0945b40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff8c645bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff8c5daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff8c663777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff8c665d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffff8c63e03a
Build id: 3.10.0-957.el7.x86_64
|
0x1ca
|
|
23
|
|
vmlinux |
0xffffffff8c61c4dc
Build id: 3.10.0-957.el7.x86_64
|
0x3c
|
|
24
|
SyS_mount
|
vmlinux |
0xffffffff8c666bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
25
|
system_call_fastpath
|
vmlinux |
0xffffffff8cb74ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
|
26
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8cb74d21
Build id: 3.10.0-957.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [mount.nfs:4067]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) dm_mirror dm_region_hash dm_log dm_mod snd_hda_codec_realtek snd_hda_codec_generic intel_powerclamp coretemp intel_rapl kvm_intel kvm irqbypass crc32_pclmul ppdev gpio_ich iTCO_wdt ghash_clmulni_intel iTCO_vendor_support snd_hda_intel snd_hda_codec
cryptd snd_hda_core snd_hwdep snd_seq snd_seq_device pcspkr snd_pcm snd_timer sg wmi mei_me mei snd parport_pc parport i2c_i801 lpc_ich soundcore auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic i915 crct10dif_pclmul crct10dif_common crc32c_intel serio_raw i2c_algo_bit video iosf_mbi drm_kms_helper ahci syscopyarea sysfillrect sysimgblt fb_sys_fops libahci drm libata r8169 mii drm_panel_orientation_quirks
CPU: 3 PID: 4067 Comm: mount.nfs Tainted: G OEL ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: LENOVO 3157AD8/To be filled by O.E.M., BIOS 9QKT34AUS 12/16/2011
task: ffff92a6f2f74100 ti: ffff92a56d590000 task.ti: ffff92a56d590000
RIP: 0010:[<ffffffff8cb6a0f5>] [<ffffffff8cb6a0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff92a56d593a70 EFLAGS: 00000246
RAX: 0000000000000246 RBX: ffff92a56d5a3728 RCX: ffff92a6f2f74100
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff92a56d593a70 R08: ffff92a6e8653ad0 R09: ffff92a6ffa9b780
R10: ffff92a56d5a3728 R11: 0000000000000001 R12: ffff92a6ffb9ab80
R13: ffff92a56d593a78 R14: ffff92a6f2f74798 R15: 000000016d61e1e8
FS: 00007f20da26a880(0000) GS:ffff92a6ffb80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe110e79000 CR3: 00000000ad5c2000 CR4: 00000000000607e0
Call Trace:
[<ffffffff8c4c27f6>] prepare_to_wait+0x56/0x90
[<ffffffffc0938358>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffff8c4c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0939eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc09a339c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc09a4b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc099bbae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff8c645bde>] mount_fs+0x3e/0x1b0
[<ffffffff8c5daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff8c663777>] vfs_kern_mount+0x67/0x110
[<ffffffffc099bad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc099bed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc093a6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0946d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0947ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0945b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff8c645bde>] mount_fs+0x3e/0x1b0
[<ffffffff8c5daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff8c663777>] vfs_kern_mount+0x67/0x110
[<ffffffff8c665d9f>] do_mount+0x1ef/0xce0
[<ffffffff8c63e03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff8c61c4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff8c666bd3>] SyS_mount+0x83/0xd0
[<ffffffff8cb74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff8cb74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #999785
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffa137c0f5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
2
|
finish_wait
|
vmlinux |
0xffffffffa0cc3956
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc4597371
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa1
|
|
4
|
|
vmlinux |
0xffffffffa0cc3d00
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc4598eab
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0cc939c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0ccab4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
8
|
|
vmlinux |
0xffffffffa1377dbb
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1b
|
|
9
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0cc1bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
10
|
mount_fs
|
vmlinux |
0xffffffffa0e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
11
|
|
vmlinux |
0xffffffffa0ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
12
|
vfs_kern_mount
|
vmlinux |
0xffffffffa0e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
13
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0cc1ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
14
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0cc1ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
15
|
|
nfs |
0xffffffffc45996e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
16
|
nfs_fs_mount
|
nfs |
0xffffffffc45a5d3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
17
|
|
nfs |
0xffffffffc45a6ca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
18
|
|
nfs |
0xffffffffc45a4b40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
19
|
mount_fs
|
vmlinux |
0xffffffffa0e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
20
|
|
vmlinux |
0xffffffffa0ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
21
|
vfs_kern_mount
|
vmlinux |
0xffffffffa0e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
22
|
do_mount
|
vmlinux |
0xffffffffa0e66d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
23
|
|
vmlinux |
0xffffffffa0e3f03a
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ca
|
|
24
|
|
vmlinux |
0xffffffffa0e1d4dc
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3c
|
|
25
|
SyS_mount
|
vmlinux |
0xffffffffa0e67bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
26
|
tracesys
|
vmlinux |
0xffffffffa138705b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [mount.nfs:7912]
Modules linked in: overlay cmac arc4 md4 rpcsec_gss_krb5 nfsv4 nls_utf8 cifs nfs lockd ccm dns_resolver grace ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cachefiles fscache dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) nvidia(POE) snd_hda_codec_hdmi snd_hda_codec_generic vfat ppdev fat hp_wmi sparse_keymap rfkill intel_powerclamp coretemp kvm_intel kvm
irqbypass snd_hda_intel crc32_pclmul snd_usb_audio ghash_clmulni_intel snd_hda_codec uvcvideo hid_wacom aesni_intel drm_kms_helper videobuf2_vmalloc videobuf2_memops videobuf2_core lrw videodev gf128mul glue_helper ablk_helper e1000e(OE) cryptd snd_hda_core drm joydev snd_usbmidi_lib snd_hwdep snd_rawmidi snd_seq snd_seq_device snd_pcm serio_raw ptp pps_core snd_timer drm_panel_orientation_quirks pcspkr syscopyarea snd sysfillrect sysimgblt fb_sys_fops soundcore wmi parport_pc parport video int3400_thermal tpm_crb acpi_pad auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 ahci libahci libata crct10dif_pclmul crct10dif_common crc32c_intel nvme nvme_core
CPU: 14 PID: 7912 Comm: mount.nfs Tainted: P OEL ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP HP Z1 Entry Tower G6/870C, BIOS S01 Ver. 02.18.00 12/14/2023
task: ffff8857d9f4c100 ti: ffff8857d861c000 task.ti: ffff8857d861c000
RIP: 0010:[<ffffffffa137c0f5>] [<ffffffffa137c0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8857d861fa78 EFLAGS: 00000287
RAX: ffff8857e13cfad0 RBX: ffff8857e3bced28 RCX: dead000000000200
RDX: ffff8857e6b2bad0 RSI: 0000000000000287 RDI: 0000000000000287
RBP: ffff8857d861fa78 R08: ffff8857d861fad0 R09: ffff88584ec5b7c0
R10: ffff8857e3bced28 R11: 0000000000000001 R12: ffff88584ed9abc0
R13: ffff8857d861fa70 R14: ffff8857d9f4c798 R15: 00000001d861f9d8
FS: 00007f00625de880(0000) GS:ffff88584ed80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2763fffd90 CR3: 0000000f98624000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa0cc3956>] finish_wait+0x56/0x70
[<ffffffffc4597371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffffa0cc3d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc4598eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0cc939c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0ccab4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffa1377dbb>] ? mutex_unlock+0x1b/0x20
[<ffffffffc0cc1bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa0e46bde>] mount_fs+0x3e/0x1b0
[<ffffffffa0ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa0e64777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0cc1ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0cc1ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc45996e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc45a5d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc45a6ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc45a4b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa0e46bde>] mount_fs+0x3e/0x1b0
[<ffffffffa0ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa0e64777>] vfs_kern_mount+0x67/0x110
[<ffffffffa0e66d9f>] do_mount+0x1ef/0xce0
[<ffffffffa0e3f03a>] ? __check_object_size+0x1ca/0x250
[<ffffffffa0e1d4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffffa0e67bd3>] SyS_mount+0x83/0xd0
[<ffffffffa138705b>] tracesys+0xa3/0xc9
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #935303
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_lock_irqsave
|
vmlinux |
0xffffffff89d6a450
Build id: 3.10.0-957.el7.x86_64
|
-
|
|
2
|
|
vmlinux |
0xffffffff896c2938
Build id: 3.10.0-957.el7.x86_64
|
0x38
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0b7c371
Build id: 3.10.0-957.el7.x86_64
|
0xa1
|
|
4
|
|
vmlinux |
0xffffffff896c2d00
Build id: 3.10.0-957.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc0b7deab
Build id: 3.10.0-957.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0be739c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0be8b4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0bdfbae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff89845bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff897daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff89863777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0bdfad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0bdfed4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc0b7e6e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc0b8ad3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc0b8bca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc0b89b40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff89845bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff897daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff89863777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff89865d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffff8983e03a
Build id: 3.10.0-957.el7.x86_64
|
0x1ca
|
|
23
|
|
vmlinux |
0xffffffff8981c4dc
Build id: 3.10.0-957.el7.x86_64
|
0x3c
|
|
24
|
SyS_mount
|
vmlinux |
0xffffffff89866bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
25
|
system_call_fastpath
|
vmlinux |
0xffffffff89d74ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#22 stuck for 22s! [mount.nfs:15993]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache cuse fuse rdma_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_umad(OE) sunrpc iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif joydev pcspkr mei_me lpc_ich mei i2c_i801 sg ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter wmi acpi_pad binfmt_misc knem(OE) ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) ast crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mlx5_core(OE) igb drm ahci libahci mlxfw(OE) psample dca i2c_algo_bit
auxiliary(OE) devlink libata mlx_compat(OE) ptp drm_panel_orientation_quirks pps_core nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod xpmem(OE)
CPU: 22 PID: 15993 Comm: mount.nfs Tainted: G OE ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: THTF Chaoqiang R628/T1DM-E2, BIOS T1DM56.03 09/28/2019
task: ffff8d9bdb34e180 ti: ffff8d9bce744000 task.ti: ffff8d9bce744000
RIP: 0010:[<ffffffff89d6a450>] [<ffffffff89d6a450>] _raw_spin_lock_irqsave+0x0/0x40
RSP: 0018:ffff8d9bce747a80 EFLAGS: 00000293
RAX: ffff8d9bdb34e180 RBX: ffff8d9bcea9bf28 RCX: ffff8d9bdb34e180
RDX: 0000000000000004 RSI: ffff8d9bce747ab8 RDI: ffffffffc0ba5070
RBP: ffff8d9bce747aa8 R08: ffffffffc0ba5078 R09: ffff8d9bd7b9bf00
R10: ffff8d9bcea9bf28 R11: 0000000000000005 R12: ffff8d9bcea9bf28
R13: ffff8d9bce747a70 R14: ffff8d9bddb1ab80 R15: ffff8d9bce747a70
FS: 00007f5289e28880(0000) GS:ffff8d9bddb00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f923f5c8b90 CR3: 0000002fd8722000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff896c2938>] ? finish_wait+0x38/0x70
[<ffffffffc0b7c371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffff896c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0b7deab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0be739c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0be8b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0bdfbae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff89845bde>] mount_fs+0x3e/0x1b0
[<ffffffff897daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff89863777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0bdfad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0bdfed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0b7e6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0b8ad3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0b8bca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0b89b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff89845bde>] mount_fs+0x3e/0x1b0
[<ffffffff897daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff89863777>] vfs_kern_mount+0x67/0x110
[<ffffffff89865d9f>] do_mount+0x1ef/0xce0
[<ffffffff8983e03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff8981c4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff89866bd3>] SyS_mount+0x83/0xd0
[<ffffffff89d74ddb>] system_call_fastpath+0x22/0x27
Code: 02 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17 85 c0 75 01 c3 55 89 c6 48 89 e5 e8 fd 1a ff ff 5d c3 90 66 2e 0f 1f 84 00 00 00 00 00 <0f> 1f 44 00 00 55 48 89 e5 53 9c 58 0f 1f 44 00 00 48 89 c3 fa
Complete report #935302
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff9096a0f5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
2
|
finish_wait
|
vmlinux |
0xffffffff902c2956
Build id: 3.10.0-957.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc094f371
Build id: 3.10.0-957.el7.x86_64
|
0xa1
|
|
4
|
|
vmlinux |
0xffffffff902c2d00
Build id: 3.10.0-957.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc0950eab
Build id: 3.10.0-957.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc09ba39c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc09bbb4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc09b2bae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff90445bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff903daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff90463777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc09b2ad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc09b2ed4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc09516e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc095dd3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc095eca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc095cb40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff90445bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff903daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff90463777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff90465d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffff9043e03a
Build id: 3.10.0-957.el7.x86_64
|
0x1ca
|
|
23
|
|
vmlinux |
0xffffffff9041c4dc
Build id: 3.10.0-957.el7.x86_64
|
0x3c
|
|
24
|
SyS_mount
|
vmlinux |
0xffffffff90466bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
25
|
system_call_fastpath
|
vmlinux |
0xffffffff90974ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#16 stuck for 22s! [mount.nfs:16043]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache cuse fuse rdma_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_umad(OE) sunrpc iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif pcspkr joydev i2c_i801 sg lpc_ich ipmi_si ipmi_devintf ipmi_msghandler mei_me mei wmi acpi_pad acpi_power_meter binfmt_misc knem(OE) ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) ast drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul crct10dif_common sysimgblt crc32c_intel fb_sys_fops ttm drm drm_panel_orientation_quirks mlx5_core(OE) igb ahci libahci
mlxfw(OE) psample dca auxiliary(OE) i2c_algo_bit devlink libata mlx_compat(OE) ptp pps_core nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod xpmem(OE)
CPU: 16 PID: 16043 Comm: mount.nfs Tainted: G OE ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: THTF Chaoqiang R628/T1DM-E2, BIOS T1DM56.03 09/28/2019
task: ffff8e04d588d140 ti: ffff8e1cc82ec000 task.ti: ffff8e1cc82ec000
RIP: 0010:[<ffffffff9096a0f5>] [<ffffffff9096a0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8e1cc82efa78 EFLAGS: 00000293
RAX: ffffffffc0978078 RBX: ffff8e1cd3e26b28 RCX: dead000000000200
RDX: ffffffffc0978078 RSI: 0000000000000293 RDI: 0000000000000293
RBP: ffff8e1cc82efa78 R08: ffff8e1cc82efad0 R09: ffff8e1cd2afa300
R10: ffff8e1cd3e26b28 R11: 0000000000000005 R12: ffff8e04dfa1ab80
R13: ffff8e1cc82efa70 R14: ffff8e04d588d7d8 R15: 00000001c82ef9d8
FS: 00007fd947dcb880(0000) GS:ffff8e04dfa00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005564600ee362 CR3: 0000002fd038e000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff902c2956>] finish_wait+0x56/0x70
[<ffffffffc094f371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffff902c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0950eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc09ba39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc09bbb4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc09b2bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff90445bde>] mount_fs+0x3e/0x1b0
[<ffffffff903daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff90463777>] vfs_kern_mount+0x67/0x110
[<ffffffffc09b2ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc09b2ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc09516e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc095dd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc095eca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc095cb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff90445bde>] mount_fs+0x3e/0x1b0
[<ffffffff903daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff90463777>] vfs_kern_mount+0x67/0x110
[<ffffffff90465d9f>] do_mount+0x1ef/0xce0
[<ffffffff9043e03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff9041c4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff90466bd3>] SyS_mount+0x83/0xd0
[<ffffffff90974ddb>] system_call_fastpath+0x22/0x27
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #935300
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
prepare_to_wait
|
vmlinux |
0xffffffff818c27fb
Build id: 3.10.0-957.el7.x86_64
|
0x5b
|
|
2
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0c2f358
Build id: 3.10.0-957.el7.x86_64
|
0x88
|
|
3
|
|
vmlinux |
0xffffffff818c2d00
Build id: 3.10.0-957.el7.x86_64
|
0x30
|
|
4
|
nfs_get_client
|
nfs |
0xffffffffc0c30eab
Build id: 3.10.0-957.el7.x86_64
|
0x22b
|
|
5
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0cb239c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
6
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0cb3b4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
7
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0caabae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
8
|
mount_fs
|
vmlinux |
0xffffffff81a45bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
9
|
|
vmlinux |
0xffffffff819daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
10
|
vfs_kern_mount
|
vmlinux |
0xffffffff81a63777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
11
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0caaad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
12
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0caaed4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
13
|
|
nfs |
0xffffffffc0c316e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
14
|
nfs_fs_mount
|
nfs |
0xffffffffc0c3dd3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
15
|
|
nfs |
0xffffffffc0c3eca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
16
|
|
nfs |
0xffffffffc0c3cb40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
17
|
mount_fs
|
vmlinux |
0xffffffff81a45bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
18
|
|
vmlinux |
0xffffffff819daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
19
|
vfs_kern_mount
|
vmlinux |
0xffffffff81a63777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
20
|
do_mount
|
vmlinux |
0xffffffff81a65d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
21
|
|
vmlinux |
0xffffffff81a3e03a
Build id: 3.10.0-957.el7.x86_64
|
0x1ca
|
|
22
|
|
vmlinux |
0xffffffff81a1c4dc
Build id: 3.10.0-957.el7.x86_64
|
0x3c
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffff81a66bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffff81f74ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#26 stuck for 22s! [mount.nfs:16052]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 cuse fuse dns_resolver nfs lockd grace fscache rdma_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_umad(OE) sunrpc iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif pcspkr joydev sg mei_me lpc_ich i2c_i801 mei wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad binfmt_misc knem(OE) ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) ast crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mlx5_core(OE) igb ahci mlxfw(OE) psample auxiliary(OE) devlink
dca i2c_algo_bit libahci mlx_compat(OE) drm_panel_orientation_quirks ptp pps_core libata nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod xpmem(OE)
CPU: 26 PID: 16052 Comm: mount.nfs Tainted: G OE ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: THTF T1DM/T1DM-E2, BIOS T1DM56.03 09/28/2019
task: ffff977ac92ab0c0 ti: ffff977ac62d4000 task.ti: ffff977ac62d4000
RIP: 0010:[<ffffffff818c27fb>] [<ffffffff818c27fb>] prepare_to_wait+0x5b/0x90
RSP: 0018:ffff977ac62d7a98 EFLAGS: 00000286
RAX: 0000000000000246 RBX: ffffffffc0c58078 RCX: ffff977ac92ab0c0
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff977ac62d7aa8 R08: ffffffffc0c58078 R09: ffff977ace72a400
R10: ffff9792ca77af28 R11: 0000000000000005 R12: ffff9792ca77af28
R13: 0000000000000005 R14: ffff9792ca77af28 R15: ffff977ac62d7a70
FS: 00007f012dfa6880(0000) GS:ffff9792ddc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055f8bd693002 CR3: 00000017bf626000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffc0c2f358>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffff818c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0c30eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0cb239c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0cb3b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0caabae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff81a45bde>] mount_fs+0x3e/0x1b0
[<ffffffff819daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff81a63777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0caaad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0caaed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0c316e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0c3dd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0c3eca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0c3cb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff81a45bde>] mount_fs+0x3e/0x1b0
[<ffffffff819daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff81a63777>] vfs_kern_mount+0x67/0x110
[<ffffffff81a65d9f>] do_mount+0x1ef/0xce0
[<ffffffff81a3e03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff81a1c4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff81a66bd3>] SyS_mount+0x83/0xd0
[<ffffffff81f74ddb>] system_call_fastpath+0x22/0x27
Code: 89 6d e0 48 8b 55 e0 65 48 8b 0c 25 80 0e 01 00 48 87 11 4c 89 e7 48 89 c6 48 89 55 e0 48 8b 55 e0 e8 ea 78 6a 00 48 83 c4 10 5b <41> 5c 41 5d 5d c3 0f 1f 80 00 00 00 00 49 8b 54 24 08 49 8d 74
Complete report #907861
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
nfs_get_client
|
nfs |
0xffffffffc0cabe99
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x219
|
|
2
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0d8a39c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
3
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0d8bb4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
4
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0d82bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
5
|
mount_fs
|
vmlinux |
0xffffffff87e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
6
|
|
vmlinux |
0xffffffff87ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
7
|
vfs_kern_mount
|
vmlinux |
0xffffffff87e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
8
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0d82ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
9
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0d82ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
10
|
|
nfs |
0xffffffffc0cac6e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
11
|
nfs_fs_mount
|
nfs |
0xffffffffc0cb8d3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
12
|
|
nfs |
0xffffffffc0cb9ca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
13
|
|
nfs |
0xffffffffc0cb7b40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
14
|
mount_fs
|
vmlinux |
0xffffffff87e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
15
|
|
vmlinux |
0xffffffff87ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
16
|
vfs_kern_mount
|
vmlinux |
0xffffffff87e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
17
|
do_mount
|
vmlinux |
0xffffffff87e66d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
18
|
|
vmlinux |
0xffffffff87e3f03a
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ca
|
|
19
|
|
vmlinux |
0xffffffff87e1d4dc
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3c
|
|
20
|
SyS_mount
|
vmlinux |
0xffffffff87e67bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
21
|
tracesys
|
vmlinux |
0xffffffff8838705b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mount.nfs:3586]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) tcp_lp binfmt_misc fuse uinput xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink cmac nfsv3 nfs_acl md4 nls_utf8 cifs ccm rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep cachefiles fscache dm_mirror dm_region_hash dm_log dm_mod hp_wmi sparse_keymap ppdev vfat
fat arc4 mei_wdt iTCO_wdt iTCO_vendor_support intel_pmc_core intel_powerclamp coretemp intel_rapl iwlmvm kvm mac80211 snd_hda_codec_hdmi irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul snd_hda_codec_realtek glue_helper ablk_helper cryptd snd_hda_codec_generic serio_raw iwlwifi cfg80211 uvcvideo pcspkr videobuf2_vmalloc videobuf2_memops videobuf2_core videodev joydev snd_usb_audio btusb btrtl btbcm btintel snd_usbmidi_lib bluetooth snd_rawmidi sg rfkill wmi i915 snd_hda_intel parport_pc snd_hda_codec parport e1000e(OE) snd_hda_core iosf_mbi video i2c_algo_bit snd_hwdep snd_seq drm_kms_helper tpm_infineon snd_seq_device acpi_pad drm snd_pcm mei_me snd_timer snd drm_panel_orientation_quirks syscopyarea sysfillrect mei sysimgblt fb_sys_fops soundcore ptp i2c_i801 pps_core
auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ahci libahci libata
CPU: 1 PID: 3586 Comm: mount.nfs Kdump: loaded Tainted: G W OE ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP HP EliteDesk 800 G2 DM 35W/8055, BIOS N21 Ver. 02.53 04/16/2021
task: ffff8f7558f630c0 ti: ffff8f74d2610000 task.ti: ffff8f74d2610000
RIP: 0010:[<ffffffffc0cabe99>] [<ffffffffc0cabe99>] nfs_get_client+0x219/0x470 [nfs]
RSP: 0000:ffff8f74d2613b08 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffff87cc3956 RCX: 0000000000000002
RDX: ffff8f73ee822b60 RSI: 0000000000000000 RDI: ffff8f7535eb4840
RBP: ffff8f74d2613b60 R08: ffff8f74d2613ad0 R09: ffff8f73ee822b00
R10: ffff8f73a508b128 R11: 0000000000000001 R12: 0000000000000293
R13: 0000000000000293 R14: ffffffffc0cd3070 R15: ffff8f74d2613ad0
FS: 00007f1ab95e8880(0000) GS:ffff8f756f440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000002486db0 CR3: 0000000192b3e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0d8a39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0d8bb4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0d82bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff87e46bde>] mount_fs+0x3e/0x1b0
[<ffffffff87ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff87e64777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0d82ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0d82ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0cac6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0cb8d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0cb9ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0cb7b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff87e46bde>] mount_fs+0x3e/0x1b0
[<ffffffff87ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff87e64777>] vfs_kern_mount+0x67/0x110
[<ffffffff87e66d9f>] do_mount+0x1ef/0xce0
[<ffffffff87e3f03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff87e1d4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff87e67bd3>] SyS_mount+0x83/0xd0
[<ffffffff8838705b>] tracesys+0xa3/0xc9
Code: 48 c7 c7 60 ee cc c0 31 c0 e8 8b 17 6c c7 48 89 d8 e9 4f 01 00 00 0f 1f 40 00 4c 89 4d b8 48 89 55 c0 4c 89 55 c8 f0 41 ff 04 24 <4c> 89 ef c6 07 00 0f 1f 40 00 4c 89 e7 e8 25 e4 ff ff 4c 89 e7
Complete report #852583
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_wait
|
vmlinux |
0xffffffff89cc3916
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x16
|
|
2
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0667371
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa1
|
|
3
|
|
vmlinux |
0xffffffff89cc3d00
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x30
|
|
4
|
nfs_get_client
|
nfs |
0xffffffffc0668eab
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22b
|
|
5
|
nfs4_set_client
|
nfsv4 |
0xffffffffc06ea39c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
6
|
nfs4_create_server
|
nfsv4 |
0xffffffffc06ebb4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
7
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc06e2bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
8
|
mount_fs
|
vmlinux |
0xffffffff89e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
9
|
|
vmlinux |
0xffffffff89ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
10
|
vfs_kern_mount
|
vmlinux |
0xffffffff89e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
11
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc06e2ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
12
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc06e2ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
13
|
|
nfs |
0xffffffffc06696e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
14
|
nfs_fs_mount
|
nfs |
0xffffffffc0675d3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
15
|
|
nfs |
0xffffffffc0676ca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
16
|
|
nfs |
0xffffffffc0674b40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
17
|
mount_fs
|
vmlinux |
0xffffffff89e46bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
18
|
|
vmlinux |
0xffffffff89ddbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
19
|
vfs_kern_mount
|
vmlinux |
0xffffffff89e64777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
20
|
do_mount
|
vmlinux |
0xffffffff89e66d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
21
|
|
vmlinux |
0xffffffff89e3f03a
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ca
|
|
22
|
|
vmlinux |
0xffffffff89e1d4dc
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3c
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffff89e67bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffff8a386ddb
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [mount.nfs:6424]
Modules linked in: joydev uinput xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nfsv3 cmac nfs_acl arc4 md4 nls_utf8 cifs rpcsec_gss_krb5 nfsv4 ccm dns_resolver nfs lockd grace ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cachefiles fscache dm_mirror dm_region_hash dm_log dm_mod vfat fat iTCO_wdt iTCO_vendor_support sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm
irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr i2c_i801 hpwdt hpilo ipmi_si ipmi_devintf ipmi_msghandler pcc_cpufreq lpc_ich ioatdma acpi_power_meter wmi dca auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper ahci crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt libahci fb_sys_fops ttm libata drm tg3 ptp pps_core drm_panel_orientation_quirks
CPU: 10 PID: 6424 Comm: mount.nfs Kdump: loaded Tainted: G W ------------ 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff8a7f7e7ea080 ti: ffff8a7c5ba60000 task.ti: ffff8a7c5ba60000
RIP: 0010:[<ffffffff89cc3916>] [<ffffffff89cc3916>] finish_wait+0x16/0x70
RSP: 0000:ffff8a7c5ba63a98 EFLAGS: 00000202
RAX: ffff8a7f7e7ea080 RBX: ffff8a75f9677ad0 RCX: ffff8a7f7e7ea080
RDX: 0000000000000004 RSI: ffff8a7c5ba63ab8 RDI: ffffffffc0690070
RBP: ffff8a7c5ba63aa8 R08: ffff8a75f9677ad0 R09: ffff8a8447373e00
R10: ffff8a7f58f33528 R11: 0000000000000001 R12: ffff8a7f58f33528
R13: 0000000000000001 R14: ffff8a7f58f33528 R15: ffff8a7c5ba63a70
FS: 00007f7fe37de880(0000) GS:ffff8a7c5f680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5f1d3b2a7c CR3: 00000002a0584000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0667371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffff89cc3d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0668eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc06ea39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc06ebb4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc06e2bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff89e46bde>] mount_fs+0x3e/0x1b0
[<ffffffff89ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff89e64777>] vfs_kern_mount+0x67/0x110
[<ffffffffc06e2ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc06e2ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc06696e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0675d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0676ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0674b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff89e46bde>] mount_fs+0x3e/0x1b0
[<ffffffff89ddbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff89e64777>] vfs_kern_mount+0x67/0x110
[<ffffffff89e66d9f>] do_mount+0x1ef/0xce0
[<ffffffff89e3f03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff89e1d4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff89e67bd3>] SyS_mount+0x83/0xd0
[<ffffffff8a386ddb>] system_call_fastpath+0x22/0x27
Code: df e8 ef 87 6b 00 5b 41 5c 41 5d 5d c3 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 65 48 8b 04 25 80 0e 01 00 48 89 e5 41 56 41 55 <49> 89 fd 41 54 4c 8d 66 18 53 48 c7 00 00 00 00 00 48 89 f3 4c
Complete report #786327
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff9b97c0f5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
2
|
finish_wait
|
vmlinux |
0xffffffff9b2c3956
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0a0e371
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa1
|
|
4
|
|
vmlinux |
0xffffffff9b2c3d00
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc0a0feab
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0ab139c
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0ab2b4e
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0aa9bae
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff9b446bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff9b3dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff9b464777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0aa9ad6
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0aa9ed4
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc0a106e7
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc0a1cd3b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc0a1dca0
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc0a1bb40
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff9b446bde
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff9b3dbff5
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff9b464777
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff9b466d9f
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffff9b43f03a
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x1ca
|
|
23
|
|
vmlinux |
0xffffffff9b41d4dc
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x3c
|
|
24
|
SyS_mount
|
vmlinux |
0xffffffff9b467bd3
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0x83
|
|
25
|
tracesys
|
vmlinux |
0xffffffff9b98705b
Build id: 3.10.0-957.el7.centos.plus.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 22s! [mount.nfs:61751]
Modules linked in: joydev uinput xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nfsv3 nfs_acl cmac arc4 md4 nls_utf8 cifs ccm rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod vfat fat sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul iTCO_wdt iTCO_vendor_support
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg hpilo hpwdt lpc_ich i2c_i801 ioatdma dca ipmi_si ipmi_devintf ipmi_msghandler pcc_cpufreq acpi_power_meter wmi auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 ahci i2c_algo_bit drm_kms_helper crct10dif_pclmul libahci crct10dif_common crc32c_intel syscopyarea libata sysfillrect sysimgblt fb_sys_fops tg3 ttm drm ptp pps_core drm_panel_orientation_quirks
CPU: 11 PID: 61751 Comm: mount.nfs Kdump: loaded Tainted: G W L ------------ T 3.10.0-957.el7.centos.plus.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 04/29/2021
task: ffff97b3a66a0000 ti: ffff97b741bd0000 task.ti: ffff97b741bd0000
RIP: 0010:[<ffffffff9b97c0f5>] [<ffffffff9b97c0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff97b741bd3a78 EFLAGS: 00000287
RAX: ffff97be9b79fad0 RBX: ffff97b715b8e928 RCX: dead000000000200
RDX: ffff97b78d2b3ad0 RSI: 0000000000000287 RDI: 0000000000000287
RBP: ffff97b741bd3a78 R08: ffff97b741bd3ad0 R09: ffff97bf9f01b7c0
R10: ffff97b715b8e928 R11: 0000000000000001 R12: ffff97b79f6dabc0
R13: ffff97b741bd3a78 R14: ffff97b3a66a0698 R15: 0000000141bd39d8
FS: 00007f0ae0380880(0000) GS:ffff97b79f6c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056455259128c CR3: 0000000841abe000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9b2c3956>] finish_wait+0x56/0x70
[<ffffffffc0a0e371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffff9b2c3d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0a0feab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0ab139c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0ab2b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0aa9bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff9b446bde>] mount_fs+0x3e/0x1b0
[<ffffffff9b3dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9b464777>] vfs_kern_mount+0x67/0x110
[<ffffffffc0aa9ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0aa9ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0a106e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0a1cd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0a1dca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0a1bb40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff9b446bde>] mount_fs+0x3e/0x1b0
[<ffffffff9b3dbff5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9b464777>] vfs_kern_mount+0x67/0x110
[<ffffffff9b466d9f>] do_mount+0x1ef/0xce0
[<ffffffff9b43f03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff9b41d4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff9b467bd3>] SyS_mount+0x83/0xd0
[<ffffffff9b98705b>] tracesys+0xa3/0xc9
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #334379
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffb536a175
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x15
|
|
2
|
prepare_to_wait
|
vmlinux |
0xffffffffb4cc27f6
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc1a46358
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x88
|
|
4
|
|
vmlinux |
0xffffffffb4cc2d00
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc1a47eab
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc1ab139c
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x9c
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc1ab2b4e
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc1aa9bae
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffffb4e45e4e
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffffb4ddb265
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffffb4e639e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc1aa9ad6
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc1aa9ed4
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc1a486e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc1a54d3b
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc1a55ca0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc1a53b40
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffffb4e45e4e
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffffb4ddb265
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffffb4e639e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffffb4e6600f
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffffb4e3e2aa
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x1ca
|
|
23
|
|
vmlinux |
0xffffffffb4e1c74c
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x3c
|
|
24
|
SyS_mount
|
vmlinux |
0xffffffffb4e66e43
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x83
|
|
25
|
system_call_fastpath
|
vmlinux |
0xffffffffb5374ddb
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [mount.nfs:14049]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM overlay(T) ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace ipt_REJECT nf_reject_ipv4 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6_tables iptable_filter vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) cachefiles fscache vboxdrv(OE) iTCO_wdt iTCO_vendor_support sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm snd_hda_codec_hdmi irqbypass crc32_pclmul ghash_clmulni_intel cryptd snd_hda_codec_realtek snd_usb_audio snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops
snd_hda_intel videobuf2_core joydev videodev snd_usbmidi_lib snd_hda_codec snd_rawmidi snd_hda_core intel_wmi_thunderbolt snd_hwdep snd_seq pcspkr snd_seq_device snd_pcm sg snd_timer i2c_i801 snd lpc_ich ioatdma soundcore acpi_power_meter binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul crct10dif_common ahci drm crc32c_intel igb libahci libata ptp pps_core dca drm_panel_orientation_quirks i2c_algo_bit ipmi_devintf ipmi_msghandler wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 14049 Comm: mount.nfs Tainted: P IOEL ------------ T 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: Supermicro SYS-7038A-I/X10DAI, BIOS 1.0c 01/29/2015
task: ffff972e90265140 ti: ffff972694358000 task.ti: ffff972694358000
RIP: 0010:[<ffffffffb536a175>] [<ffffffffb536a175>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff97269435ba70 EFLAGS: 00000246
RAX: 0000000000000246 RBX: ffff972e990d4728 RCX: ffff972e90265140
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff97269435ba70 R08: ffffffffc1a6f078 R09: ffff97269b774b00
R10: ffff972e990d4728 R11: 0000000000000005 R12: ffff97269f89ab80
R13: ffff97269435ba70 R14: ffff972e902657d8 R15: 00000001b4ce53df
FS: 00007fc0c452e880(0000) GS:ffff97269f880000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fee114a87c0 CR3: 0000000822328000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb4cc27f6>] prepare_to_wait+0x56/0x90
[<ffffffffc1a46358>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffffb4cc2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc1a47eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc1ab139c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc1ab2b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc1aa9bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffb4e45e4e>] mount_fs+0x3e/0x1b0
[<ffffffffb4ddb265>] ? __alloc_percpu+0x15/0x20
[<ffffffffb4e639e7>] vfs_kern_mount+0x67/0x110
[<ffffffffc1aa9ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc1aa9ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc1a486e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc1a54d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc1a55ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc1a53b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffb4e45e4e>] mount_fs+0x3e/0x1b0
[<ffffffffb4ddb265>] ? __alloc_percpu+0x15/0x20
[<ffffffffb4e639e7>] vfs_kern_mount+0x67/0x110
[<ffffffffb4e6600f>] do_mount+0x1ef/0xce0
[<ffffffffb4e3e2aa>] ? __check_object_size+0x1ca/0x250
[<ffffffffb4e1c74c>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffffb4e66e43>] SyS_mount+0x83/0xd0
[<ffffffffb5374ddb>] system_call_fastpath+0x22/0x27
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #263362
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
nfs_get_client
|
nfs |
0xffffffffc1acce80
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x210
|
|
2
|
nfs4_set_client
|
nfsv4 |
0xffffffffc1b53518
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x98
|
|
3
|
nfs4_create_server
|
nfsv4 |
0xffffffffc1b54cbe
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x13e
|
|
4
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc1b4bd9e
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x2e
|
|
5
|
mount_fs
|
vmlinux |
0xffffffff9c223c2e
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x3e
|
|
6
|
|
vmlinux |
0xffffffff9c1bb1e5
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x15
|
|
7
|
vfs_kern_mount
|
vmlinux |
0xffffffff9c241157
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x67
|
|
8
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc1b4bcc6
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x86
|
|
9
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc1b4c0c4
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x44
|
|
10
|
|
nfs |
0xffffffffc1acd6d7
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x27
|
|
11
|
nfs_fs_mount
|
nfs |
0xffffffffc1ad9cdb
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x4cb
|
|
12
|
|
nfs |
0xffffffffc1adac20
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x140
|
|
13
|
|
nfs |
0xffffffffc1ad8ae0
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x70
|
|
14
|
mount_fs
|
vmlinux |
0xffffffff9c223c2e
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x3e
|
|
15
|
|
vmlinux |
0xffffffff9c1bb1e5
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x15
|
|
16
|
vfs_kern_mount
|
vmlinux |
0xffffffff9c241157
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x67
|
|
17
|
do_mount
|
vmlinux |
0xffffffff9c24377f
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x1ef
|
|
18
|
|
vmlinux |
0xffffffff9c1faf2c
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x3c
|
|
19
|
SyS_mount
|
vmlinux |
0xffffffff9c2445b3
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x83
|
|
20
|
system_call_fastpath
|
vmlinux |
0xffffffff9c72579b
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [mount.nfs:2335]
Modules linked in: devlink ppdev rpcsec_gss_krb5 vmnet(OE) auth_rpcgss parport_pc parport nfsv4 vmw_vsock_vmci_transport vsock vmw_vmci dns_resolver nfs vmmon(OE) lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) snd_hda_codec_hdmi sb_edac intel_powerclamp coretemp mei_wdt intel_rapl iosf_mbi kvm_intel
iTCO_wdt iTCO_vendor_support kvm dcdbas irqbypass snd_hda_codec_realtek crc32_pclmul snd_hda_codec_generic ghash_clmulni_intel snd_hda_intel drm_kms_helper aesni_intel lrw joydev snd_hda_codec gf128mul glue_helper ablk_helper syscopyarea cryptd sysfillrect sysimgblt fb_sys_fops snd_hda_core drm snd_hwdep snd_seq snd_seq_device snd_pcm pcspkr ipmi_devintf i2c_i801 ipmi_msghandler mei_me snd_timer lpc_ich snd sg mei soundcore shpchp binfmt_misc ip_tables xfs sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ahci bnx2x libahci libata crct10dif_pclmul crct10dif_common e1000e igb mpt2sas crc32c_intel serio_raw dca i2c_algo_bit i2c_core mdio ptp raid_class pps_core scsi_transport_sas libcrc32c wmi
CPU: 2 PID: 2335 Comm: mount.nfs Tainted: P OEL ------------ 3.10.0-862.11.6.el7.x86_64 #1
Hardware name: Dell Inc. Precision T7610/0NK70N, BIOS A08 07/24/2014
task: ffff966f8051dee0 ti: ffff966f818ac000 task.ti: ffff966f818ac000
RIP: 0010:[<ffffffffc1acce80>] [<ffffffffc1acce80>] nfs_get_client+0x210/0x470 [nfs]
RSP: 0018:ffff966f818afb08 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffff9c0beb96 RCX: 0000000000000003
RDX: ffff966f9cad6360 RSI: ffff966f9cad63b8 RDI: ffff966f9cad63b8
RBP: ffff966f818afb60 R08: ffff966f818afad0 R09: ffff966f9cad6300
R10: ffff966fadac8f28 R11: 0000000000000005 R12: 0000000000000293
R13: 0000000000000293 R14: ffffffffc1af6070 R15: ffff966f818afad0
FS: 00007f367f470880(0000) GS:ffff966fefc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fddb1be15f0 CR3: 000000101aa9e000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc1b53518>] nfs4_set_client+0x98/0x130 [nfsv4]
[<ffffffffc1b54cbe>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc1b4bd9e>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff9c223c2e>] mount_fs+0x3e/0x1b0
[<ffffffff9c1bb1e5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9c241157>] vfs_kern_mount+0x67/0x110
[<ffffffffc1b4bcc6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc1b4c0c4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc1acd6d7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc1ad9cdb>] nfs_fs_mount+0x4cb/0xda0 [nfs]
[<ffffffffc1adac20>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc1ad8ae0>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff9c223c2e>] mount_fs+0x3e/0x1b0
[<ffffffff9c1bb1e5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9c241157>] vfs_kern_mount+0x67/0x110
[<ffffffff9c24377f>] do_mount+0x1ef/0xce0
[<ffffffff9c1faf2c>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff9c2445b3>] SyS_mount+0x83/0xd0
[<ffffffff9c72579b>] system_call_fastpath+0x22/0x27
Code: 00 00 49 8b 77 08 48 89 c2 48 c7 c7 c8 15 af c1 31 c0 e8 ea 03 c4 da 48 89 d8 e9 4f 01 00 00 0f 1f 40 00 4c 89 4d b8 48 89 55 c0 <4c> 89 55 c8 f0 41 ff 04 24 4c 89 ef c6 07 00 0f 1f 40 00 4c 89
Complete report #198449
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
nfs_get_client
|
nfs |
0xffffffffc0c70e9b
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x22b
|
|
2
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0cda518
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x98
|
|
3
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0cdbcbe
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x13e
|
|
4
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0cd2d9e
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x2e
|
|
5
|
mount_fs
|
vmlinux |
0xffffffff9e023cee
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x3e
|
|
6
|
|
vmlinux |
0xffffffff9dfbb2a5
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x15
|
|
7
|
vfs_kern_mount
|
vmlinux |
0xffffffff9e041217
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x67
|
|
8
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0cd2cc6
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x86
|
|
9
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0cd30c4
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x44
|
|
10
|
|
nfs |
0xffffffffc0c716d7
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x27
|
|
11
|
nfs_fs_mount
|
nfs |
0xffffffffc0c7dcdb
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x4cb
|
|
12
|
|
nfs |
0xffffffffc0c7ec20
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x140
|
|
13
|
|
nfs |
0xffffffffc0c7cae0
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x70
|
|
14
|
mount_fs
|
vmlinux |
0xffffffff9e023cee
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x3e
|
|
15
|
|
vmlinux |
0xffffffff9dfbb2a5
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x15
|
|
16
|
vfs_kern_mount
|
vmlinux |
0xffffffff9e041217
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x67
|
|
17
|
do_mount
|
vmlinux |
0xffffffff9e04383f
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x1ef
|
|
18
|
|
vmlinux |
0xffffffff9dffafec
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x3c
|
|
19
|
SyS_mount
|
vmlinux |
0xffffffff9e044673
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x83
|
|
20
|
system_call_fastpath
|
vmlinux |
0xffffffff9e52579b
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mount.nfs:1371]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc arc4 ath9k ath9k_common ath9k_hw ath mac80211 cfg80211 intel_powerclamp coretemp intel_rapl kvm_intel ath3k kvm btusb btrtl btbcm btintel bluetooth irqbypass crc32_pclmul ghash_clmulni_intel rfkill aesni_intel dell_wmi lrw dell_smbios gf128mul dcdbas iTCO_wdt glue_helper ablk_helper iTCO_vendor_support snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi pinctrl_sunrisepoint sparse_keymap pinctrl_intel cryptd i2c_i801 mei_me mei sg snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd pcspkr soundcore acpi_pad shpchp ip_tables xfs libcrc32c rtsx_usb_sdmmc mmc_core rtsx_usb sr_mod sd_mod cdrom crc_t10dif crct10dif_generic i915 nouveau mxm_wmi
ttm iosf_mbi i2c_algo_bit ahci drm_kms_helper libahci syscopyarea sysfillrect sysimgblt fb_sys_fops libata drm crct10dif_pclmul crct10dif_common crc32c_intel r8169 serio_raw mii wmi i2c_hid video i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1371 Comm: mount.nfs Kdump: loaded Tainted: G L ------------ 3.10.0-862.14.4.el7.x86_64 #1
Hardware name: Dell Inc. Vostro 3668/0N9Y46, BIOS 1.7.0 01/30/2018
task: ffffa098ab484f10 ti: ffffa0988b6a8000 task.ti: ffffa0988b6a8000
RIP: 0010:[<ffffffffc0c70e9b>] [<ffffffffc0c70e9b>] nfs_get_client+0x22b/0x470 [nfs]
RSP: 0018:ffffa0988b6abb08 EFLAGS: 00000286
RAX: 00000000fffffe00 RBX: ffffffff9debeb96 RCX: 0000000000000000
RDX: ffffffffc0c9a078 RSI: 0000000000000206 RDI: 0000000000000206
RBP: ffffa0988b6abb60 R08: ffffa0988b6abad0 R09: ffffa098bec19740
R10: ffffa0989d637328 R11: 0000000000000000 R12: 0000000000000206
R13: ffffffff9e51b7d7 R14: ffffa0988b6aba78 R15: ffffffff9e50dc5f
FS: 00007ff02b5ba880(0000) GS:ffffa098bec40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f061322a000 CR3: 00000004646fc000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0cda518>] nfs4_set_client+0x98/0x130 [nfsv4]
[<ffffffffc0cdbcbe>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0cd2d9e>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff9e023cee>] mount_fs+0x3e/0x1b0
[<ffffffff9dfbb2a5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9e041217>] vfs_kern_mount+0x67/0x110
[<ffffffffc0cd2cc6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0cd30c4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0c716d7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0c7dcdb>] nfs_fs_mount+0x4cb/0xda0 [nfs]
[<ffffffffc0c7ec20>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0c7cae0>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff9e023cee>] mount_fs+0x3e/0x1b0
[<ffffffff9dfbb2a5>] ? __alloc_percpu+0x15/0x20
[<ffffffff9e041217>] vfs_kern_mount+0x67/0x110
[<ffffffff9e04383f>] do_mount+0x1ef/0xce0
[<ffffffff9dffafec>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff9e044673>] SyS_mount+0x83/0xd0
[<ffffffff9e52579b>] system_call_fastpath+0x22/0x27
Code: 4f 01 00 00 0f 1f 40 00 4c 89 4d b8 48 89 55 c0 4c 89 55 c8 f0 41 ff 04 24 4c 89 ef c6 07 00 0f 1f 40 00 4c 89 e7 e8 25 e4 ff ff <4c> 89 e7 e8 9d f2 ff ff 4c 89 ef e8 85 a8 8a dd 4c 8b 55 c8 48
Complete report #160976
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_wait
|
vmlinux |
0xffffffffa7cc3ba9
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x59
|
|
2
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0b25371
Build id: 3.10.0-957.27.2.el7.x86_64
|
0xa1
|
|
3
|
|
vmlinux |
0xffffffffa7cc3f50
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x30
|
|
4
|
nfs_get_client
|
nfs |
0xffffffffc0b26eab
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x22b
|
|
5
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0c7a39c
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x9c
|
|
6
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0c7bb4e
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x13e
|
|
7
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0c72bae
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x2e
|
|
8
|
mount_fs
|
vmlinux |
0xffffffffa7e4723e
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x3e
|
|
9
|
|
vmlinux |
0xffffffffa7ddc6d5
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x15
|
|
10
|
vfs_kern_mount
|
vmlinux |
0xffffffffa7e64e67
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x67
|
|
11
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0c72ad6
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x86
|
|
12
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0c72ed4
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x44
|
|
13
|
|
nfs |
0xffffffffc0b276e7
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x27
|
|
14
|
nfs_fs_mount
|
nfs |
0xffffffffc0b33d3b
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x4cb
|
|
15
|
|
nfs |
0xffffffffc0b34ca0
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x140
|
|
16
|
|
nfs |
0xffffffffc0b32b40
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x70
|
|
17
|
mount_fs
|
vmlinux |
0xffffffffa7e4723e
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x3e
|
|
18
|
|
vmlinux |
0xffffffffa7ddc6d5
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x15
|
|
19
|
vfs_kern_mount
|
vmlinux |
0xffffffffa7e64e67
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x67
|
|
20
|
do_mount
|
vmlinux |
0xffffffffa7e6748f
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x1ef
|
|
21
|
|
vmlinux |
0xffffffffa7e3f69a
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x1ca
|
|
22
|
|
vmlinux |
0xffffffffa7e1ea2c
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x3c
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffffa7e682c3
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x83
|
|
24
|
sys_talpa_mount
|
talpa_syscallhook |
0xffffffffc08378b6
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x66
|
|
25
|
system_call_fastpath
|
vmlinux |
0xffffffffa8376ddb
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [mount.nfs:422]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache talpa_vfshook(OE) talpa_pedconnector(OE) talpa_pedevice(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) ext4 mbcache jbd2 intel_pmc_core intel_powerclamp coretemp intel_rapl iTCO_wdt iTCO_vendor_support kvm ppdev asus_wmi sparse_keymap rfkill mxm_wmi irqbypass crc32_pclmul snd_hda_codec_hdmi ghash_clmulni_intel snd_hda_codec_realtek aesni_intel snd_hda_codec_generic lrw gf128mul glue_helper
ablk_helper cryptd snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device pcspkr snd_pcm snd_timer snd soundcore i2c_i801 sg parport_pc parport pinctrl_sunrisepoint pinctrl_intel acpi_pad wmi mei_me mei sch_fq_codel auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic i915 crct10dif_pclmul crct10dif_common crc32c_intel serio_raw r8169 mii i2c_algo_bit iosf_mbi drm_kms_helper ahci video libahci syscopyarea sysfillrect sysimgblt fb_sys_fops i2c_hid drm libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 422 Comm: mount.nfs Kdump: loaded Tainted: G OE ------------ 3.10.0-957.27.2.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/H110M-R, BIOS 3401 03/20/2017
task: ffff9fe3b410e180 ti: ffff9fe3a5568000 task.ti: ffff9fe3a5568000
RIP: 0010:[<ffffffffa7cc3ba9>] [<ffffffffa7cc3ba9>] finish_wait+0x59/0x70
RSP: 0018:ffff9fe3a556ba50 EFLAGS: 00000283
RAX: ffffffffc0b4e078 RBX: ffff9fe3a556ba88 RCX: dead000000000200
RDX: ffffffffc0b4e078 RSI: 0000000000000283 RDI: 0000000000000283
RBP: ffff9fe3a556ba60 R08: ffff9fe3a556ba88 R09: ffff9fe3fc496b00
R10: ffff9fe4ce368528 R11: 0000000000000000 R12: ffff9fe4ce368528
R13: 0000000000000000 R14: ffff9fe4ce368528 R15: ffff9fe3a556ba30
FS: 00007fd2328f9880(0000) GS:ffff9fe576d80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc82f595010 CR3: 00000000740a0000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc0b25371>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffffa7cc3f50>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0b26eab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0c7a39c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc0c7bb4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0c72bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa7e4723e>] mount_fs+0x3e/0x1b0
[<ffffffffa7ddc6d5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa7e64e67>] vfs_kern_mount+0x67/0x110
[<ffffffffc0c72ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0c72ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0b276e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0b33d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0b34ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0b32b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa7e4723e>] mount_fs+0x3e/0x1b0
[<ffffffffa7ddc6d5>] ? __alloc_percpu+0x15/0x20
[<ffffffffa7e64e67>] vfs_kern_mount+0x67/0x110
[<ffffffffa7e6748f>] do_mount+0x1ef/0xce0
[<ffffffffa7e3f69a>] ? __check_object_size+0x1ca/0x250
[<ffffffffa7e1ea2c>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffffa7e682c3>] SyS_mount+0x83/0xd0
[<ffffffffc08378b6>] sys_talpa_mount+0x66/0x100 [talpa_syscallhook]
[<ffffffffa8376ddb>] system_call_fastpath+0x22/0x27
Code: 74 30 4c 89 ef e8 68 8c 6a 00 4c 89 e7 49 89 c6 e8 5d 2d 2d 00 4c 89 63 18 4c 89 63 20 4c 89 f6 4c 89 ef e8 da 88 6a 00 5b 41 5c <41> 5d 41 5e 5d c3 90 4c 3b 66 20 75 ca eb ee 0f 1f 84 00 00 00
Complete report #123804
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff84f1b445
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x15
|
|
2
|
prepare_to_wait
|
vmlinux |
0xffffffff848bea36
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0e71348
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x88
|
|
4
|
|
vmlinux |
0xffffffff848bef10
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc0e72e9b
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc0f90518
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x98
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc0f91cbe
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc0f88d9e
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff84a23cee
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff849bb2a5
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff84a41217
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc0f88cc6
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc0f890c4
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc0e736d7
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc0e7fcdb
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc0e80c20
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc0e7eae0
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff84a23cee
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff849bb2a5
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff84a41217
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff84a4383f
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x1ef
|
|
22
|
|
vmlinux |
0xffffffff849fafec
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x3c
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffff84a44673
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffff84f2579b
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [mount.nfs:1443]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek snd_hda_codec_generic arc4 ath9k snd_hda_codec_hdmi intel_powerclamp coretemp ath9k_common intel_rapl ath9k_hw kvm_intel ath snd_hda_intel kvm snd_hda_codec mac80211 iTCO_wdt iTCO_vendor_support snd_hda_core dell_wmi snd_hwdep dell_smbios sparse_keymap snd_seq irqbypass crc32_pclmul snd_seq_device ghash_clmulni_intel ath3k dcdbas btusb snd_pcm btrtl btbcm btintel bluetooth cfg80211 aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr mei_me snd_timer mei pinctrl_sunrisepoint i2c_i801 pinctrl_intel rfkill snd soundcore shpchp acpi_pad ip_tables xfs libcrc32c rtsx_usb_sdmmc mmc_core rtsx_usb sr_mod sd_mod cdrom crc_t10dif crct10dif_generic i915 nouveau mxm_wmi
ttm iosf_mbi i2c_algo_bit drm_kms_helper ahci syscopyarea sysfillrect sysimgblt fb_sys_fops libahci drm libata crct10dif_pclmul crct10dif_common crc32c_intel r8169 serio_raw mii wmi i2c_hid video i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 1443 Comm: mount.nfs Kdump: loaded Tainted: G L ------------ 3.10.0-862.14.4.el7.x86_64 #1
Hardware name: Dell Inc. Vostro 3668/0N9Y46, BIOS 1.7.0 01/30/2018
task: ffff9be198b9dee0 ti: ffff9bddb4e34000 task.ti: ffff9bddb4e34000
RIP: 0010:[<ffffffff84f1b445>] [<ffffffff84f1b445>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9bddb4e37a70 EFLAGS: 00000246
RAX: 0000000000000246 RBX: ffff9be1a9235528 RCX: ffff9be198b9dee0
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff9bddb4e37a70 R08: ffff9bddb4e27ad0 R09: ffff9be1bec99740
R10: ffff9be1a9235528 R11: 0000000000000000 R12: ffff9be1bedd8b40
R13: ffff9bddb4e37a70 R14: ffff9be198b9e578 R15: 00000001848e066d
FS: 00007fb50f9c9880(0000) GS:ffff9be1bedc0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056125f836000 CR3: 00000004683da000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff848bea36>] prepare_to_wait+0x56/0x90
[<ffffffffc0e71348>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffff848bef10>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc0e72e9b>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc0f90518>] nfs4_set_client+0x98/0x130 [nfsv4]
[<ffffffffc0f91cbe>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc0f88d9e>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff84a23cee>] mount_fs+0x3e/0x1b0
[<ffffffff849bb2a5>] ? __alloc_percpu+0x15/0x20
[<ffffffff84a41217>] vfs_kern_mount+0x67/0x110
[<ffffffffc0f88cc6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc0f890c4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc0e736d7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0e7fcdb>] nfs_fs_mount+0x4cb/0xda0 [nfs]
[<ffffffffc0e80c20>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0e7eae0>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff84a23cee>] mount_fs+0x3e/0x1b0
[<ffffffff849bb2a5>] ? __alloc_percpu+0x15/0x20
[<ffffffff84a41217>] vfs_kern_mount+0x67/0x110
[<ffffffff84a4383f>] do_mount+0x1ef/0xce0
[<ffffffff849fafec>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff84a44673>] SyS_mount+0x83/0xd0
[<ffffffff84f2579b>] system_call_fastpath+0x22/0x27
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #52216
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff816b6545
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x15
|
|
2
|
finish_wait
|
vmlinux |
0xffffffff810b4d86
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x56
|
|
3
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc054a361
Build id: 3.10.0-693.21.1.el7.x86_64
|
0xa1
|
|
4
|
|
vmlinux |
0xffffffff810b4fc0
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x30
|
|
5
|
nfs_get_client
|
nfs |
0xffffffffc054be9b
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x22b
|
|
6
|
nfs4_set_client
|
nfsv4 |
0xffffffffc05b4fd8
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x98
|
|
7
|
nfs4_create_server
|
nfsv4 |
0xffffffffc05b677e
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x13e
|
|
8
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc05ad91e
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x2e
|
|
9
|
mount_fs
|
vmlinux |
0xffffffff81209f1e
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x3e
|
|
10
|
|
vmlinux |
0xffffffff811aa685
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x15
|
|
11
|
vfs_kern_mount
|
vmlinux |
0xffffffff81226d57
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x67
|
|
12
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc05ad846
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x86
|
|
13
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc05adc44
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x44
|
|
14
|
|
nfs |
0xffffffffc054c6d7
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x27
|
|
15
|
nfs_fs_mount
|
nfs |
0xffffffffc0558c9b
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x4cb
|
|
16
|
|
nfs |
0xffffffffc0559be0
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x140
|
|
17
|
|
nfs |
0xffffffffc0557aa0
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x70
|
|
18
|
mount_fs
|
vmlinux |
0xffffffff81209f1e
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x3e
|
|
19
|
|
vmlinux |
0xffffffff811aa685
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x15
|
|
20
|
vfs_kern_mount
|
vmlinux |
0xffffffff81226d57
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x67
|
|
21
|
do_mount
|
vmlinux |
0xffffffff81229263
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x233
|
|
22
|
SyS_mount
|
vmlinux |
0xffffffff81229ea6
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x96
|
|
23
|
system_call_fastpath
|
vmlinux |
0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x1c
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mount.nfs:977]
Modules linked in: uinput ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache snd_hda_codec_generic snd_hda_intel snd_hda_codec iosf_mbi snd_hda_core crc32_pclmul snd_hwdep snd_seq snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr virtio_balloon snd_timer sg snd soundcore i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata crc32c_intel floppy virtio_pci serio_raw virtio_ring virtio i2c_core dm_mirror dm_region_hash dm_log
dm_mod
CPU: 1 PID: 977 Comm: mount.nfs Not tainted 3.10.0-693.21.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
task: ffff880134ba0000 ti: ffff880137cdc000 task.ti: ffff880137cdc000
RIP: 0010:[<ffffffff816b6545>] [<ffffffff816b6545>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff880137cdfa60 EFLAGS: 00000283
RAX: ffffffffc0575078 RBX: ffff8800b509ad28 RCX: dead000000000200
RDX: ffffffffc0575078 RSI: 0000000000000283 RDI: 0000000000000283
RBP: ffff880137cdfa60 R08: ffff880137cdfab8 R09: ffff8800b723fa00
R10: ffff8800b509ad28 R11: ffffea0002d45300 R12: ffff8800b509ad28
R13: ffff880137cdfa60 R14: 0000000000000040 R15: ffff880137cdfa58
FS: 00007f0643806880(0000) GS:ffff88013fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f01a3529000 CR3: 00000000b73b4000 CR4: 00000000001606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff810b4d86>] finish_wait+0x56/0x70
[<ffffffffc054a361>] nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffff810b4fc0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc054be9b>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc05b4fd8>] nfs4_set_client+0x98/0x130 [nfsv4]
[<ffffffffc05b677e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc05ad91e>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffff81209f1e>] mount_fs+0x3e/0x1b0
[<ffffffff811aa685>] ? __alloc_percpu+0x15/0x20
[<ffffffff81226d57>] vfs_kern_mount+0x67/0x110
[<ffffffffc05ad846>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc05adc44>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc054c6d7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0558c9b>] nfs_fs_mount+0x4cb/0xda0 [nfs]
[<ffffffffc0559be0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0557aa0>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffff81209f1e>] mount_fs+0x3e/0x1b0
[<ffffffff811aa685>] ? __alloc_percpu+0x15/0x20
[<ffffffff81226d57>] vfs_kern_mount+0x67/0x110
[<ffffffff81229263>] do_mount+0x233/0xaf0
[<ffffffff81229ea6>] SyS_mount+0x96/0xf0
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #14207
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
prepare_to_wait
|
vmlinux |
0xffffffff9fec27fb
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x5b
|
|
2
|
nfs_wait_client_init_complete
|
nfs |
0xffffffffc0939358
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x88
|
|
3
|
|
vmlinux |
0xffffffff9fec2d00
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x30
|
|
4
|
nfs_get_client
|
nfs |
0xffffffffc093aeab
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22b
|
|
5
|
nfs4_set_client
|
nfsv4 |
0xffffffffc09f939c
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x9c
|
|
6
|
nfs4_create_server
|
nfsv4 |
0xffffffffc09fab4e
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x13e
|
|
7
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc09f1bae
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x2e
|
|
8
|
mount_fs
|
vmlinux |
0xffffffffa0045e4e
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x3e
|
|
9
|
|
vmlinux |
0xffffffff9ffdb265
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x15
|
|
10
|
vfs_kern_mount
|
vmlinux |
0xffffffffa00639e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x67
|
|
11
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc09f1ad6
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x86
|
|
12
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc09f1ed4
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x44
|
|
13
|
|
nfs |
0xffffffffc093b6e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x27
|
|
14
|
nfs_fs_mount
|
nfs |
0xffffffffc0947d3b
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x4cb
|
|
15
|
|
nfs |
0xffffffffc0948ca0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x140
|
|
16
|
|
nfs |
0xffffffffc0946b40
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x70
|
|
17
|
mount_fs
|
vmlinux |
0xffffffffa0045e4e
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x3e
|
|
18
|
|
vmlinux |
0xffffffff9ffdb265
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x15
|
|
19
|
vfs_kern_mount
|
vmlinux |
0xffffffffa00639e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x67
|
|
20
|
do_mount
|
vmlinux |
0xffffffffa006600f
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x1ef
|
|
21
|
|
vmlinux |
0xffffffffa003e2aa
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x1ca
|
|
22
|
|
vmlinux |
0xffffffffa001c74c
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x3c
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffffa0066e43
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffffa0574ddb
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [mount.nfs:221230]
Modules linked in: fuse rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache devlink bnx2fc cnic uio fcoe libfcoe libfc scsi_transport_fc scsi_tgt ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support dcdbas sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass dm_service_time pcspkr sg ipmi_si ipmi_devintf ipmi_msghandler lpc_ich mei_me acpi_power_meter mei acpi_pad
bridge dm_multipath bonding ip_tables ext4 mbcache jbd2 dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel mgag200 8021q garp drm_kms_helper ghash_clmulni_intel mrp stp aesni_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm lrw llc gf128mul glue_helper ablk_helper cryptd drm drm_panel_orientation_quirks igb i2c_algo_bit ahci ixgbe libahci scsi_transport_iscsi libata megaraid_sas dca ptp pps_core mdio wmi sunrpc dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 221230 Comm: mount.nfs Kdump: loaded Not tainted 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R630/0CNCJW, BIOS 1.0.4 08/28/2014
task: ffff9616dc7a4100 ti: ffff96173f94c000 task.ti: ffff96173f94c000
RIP: 0010:[<ffffffff9fec27fb>] [<ffffffff9fec27fb>] prepare_to_wait+0x5b/0x90
RSP: 0018:ffff96173f94fa98 EFLAGS: 00000286
RAX: 0000000000000246 RBX: ffff96173f94fad0 RCX: ffff9616dc7a4100
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff96173f94faa8 R08: ffffffffc0962078 R09: ffff96173f945900
R10: ffff9616edf9c128 R11: 0000000000000005 R12: ffff9616edf9c128
R13: 0000000000000005 R14: ffff9616edf9c128 R15: ffff96173f94fa78
FS: 00007f3a46e0d880(0000) GS:ffff9616fae80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f750903a910 CR3: 0000000ff8ad2000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc0939358>] nfs_wait_client_init_complete+0x88/0xe0 [nfs]
[<ffffffff9fec2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc093aeab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc09f939c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc09fab4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc09f1bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffa0045e4e>] mount_fs+0x3e/0x1b0
[<ffffffff9ffdb265>] ? __alloc_percpu+0x15/0x20
[<ffffffffa00639e7>] vfs_kern_mount+0x67/0x110
[<ffffffffc09f1ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc09f1ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc093b6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc0947d3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc0948ca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc0946b40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffa0045e4e>] mount_fs+0x3e/0x1b0
[<ffffffff9ffdb265>] ? __alloc_percpu+0x15/0x20
[<ffffffffa00639e7>] vfs_kern_mount+0x67/0x110
[<ffffffffa006600f>] do_mount+0x1ef/0xce0
[<ffffffffa003e2aa>] ? __check_object_size+0x1ca/0x250
[<ffffffffa001c74c>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffffa0066e43>] SyS_mount+0x83/0xd0
[<ffffffffa0574ddb>] system_call_fastpath+0x22/0x27
Code: 89 6d e0 48 8b 55 e0 65 48 8b 0c 25 80 0e 01 00 48 87 11 4c 89 e7 48 89 c6 48 89 55 e0 48 8b 55 e0 e8 6a 79 6a 00 48 83 c4 10 5b <41> 5c 41 5d 5d c3 0f 1f 80 00 00 00 00 49 8b 54 24 08 49 8d 74
Complete report #3233
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_wait
|
vmlinux |
0xffffffffb26c2906
Build id: 3.10.0-957.el7.x86_64
|
0x6
|
|
2
|
?
nfs_wait_client_init_complete
|
nfs |
0xffffffffc068d371
Build id: 3.10.0-957.el7.x86_64
|
0xa1
|
|
3
|
|
vmlinux |
0xffffffffb26c2d00
Build id: 3.10.0-957.el7.x86_64
|
0x30
|
|
4
|
nfs_get_client
|
nfs |
0xffffffffc068eeab
Build id: 3.10.0-957.el7.x86_64
|
0x22b
|
|
5
|
nfs4_set_client
|
nfsv4 |
0xffffffffc06f839c
Build id: 3.10.0-957.el7.x86_64
|
0x9c
|
|
6
|
nfs4_create_server
|
nfsv4 |
0xffffffffc06f9b4e
Build id: 3.10.0-957.el7.x86_64
|
0x13e
|
|
7
|
nfs4_remote_mount
|
nfsv4 |
0xffffffffc06f0bae
Build id: 3.10.0-957.el7.x86_64
|
0x2e
|
|
8
|
mount_fs
|
vmlinux |
0xffffffffb2845bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
9
|
|
vmlinux |
0xffffffffb27daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
10
|
vfs_kern_mount
|
vmlinux |
0xffffffffb2863777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
11
|
nfs_do_root_mount
|
nfsv4 |
0xffffffffc06f0ad6
Build id: 3.10.0-957.el7.x86_64
|
0x86
|
|
12
|
nfs4_try_mount
|
nfsv4 |
0xffffffffc06f0ed4
Build id: 3.10.0-957.el7.x86_64
|
0x44
|
|
13
|
|
nfs |
0xffffffffc068f6e7
Build id: 3.10.0-957.el7.x86_64
|
0x27
|
|
14
|
nfs_fs_mount
|
nfs |
0xffffffffc069bd3b
Build id: 3.10.0-957.el7.x86_64
|
0x4cb
|
|
15
|
|
nfs |
0xffffffffc069cca0
Build id: 3.10.0-957.el7.x86_64
|
0x140
|
|
16
|
|
nfs |
0xffffffffc069ab40
Build id: 3.10.0-957.el7.x86_64
|
0x70
|
|
17
|
mount_fs
|
vmlinux |
0xffffffffb2845bde
Build id: 3.10.0-957.el7.x86_64
|
0x3e
|
|
18
|
|
vmlinux |
0xffffffffb27daff5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
19
|
vfs_kern_mount
|
vmlinux |
0xffffffffb2863777
Build id: 3.10.0-957.el7.x86_64
|
0x67
|
|
20
|
do_mount
|
vmlinux |
0xffffffffb2865d9f
Build id: 3.10.0-957.el7.x86_64
|
0x1ef
|
|
21
|
|
vmlinux |
0xffffffffb283e03a
Build id: 3.10.0-957.el7.x86_64
|
0x1ca
|
|
22
|
|
vmlinux |
0xffffffffb281c4dc
Build id: 3.10.0-957.el7.x86_64
|
0x3c
|
|
23
|
SyS_mount
|
vmlinux |
0xffffffffb2866bd3
Build id: 3.10.0-957.el7.x86_64
|
0x83
|
|
24
|
system_call_fastpath
|
vmlinux |
0xffffffffb2d74ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#18 stuck for 22s! [mount.nfs:2103]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci
i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix libahci drm crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw nfit vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 18 PID: 2103 Comm: mount.nfs Kdump: loaded Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/09/2019
task: ffff8b7335621040 ti: ffff8b72b3798000 task.ti: ffff8b72b3798000
RIP: 0010:[<ffffffffb26c2906>] [<ffffffffb26c2906>] finish_wait+0x6/0x70
RSP: 0018:ffff8b72b379baa8 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffff8b7335621040 RCX: ffff8b7335621040
RDX: 0000000000000004 RSI: ffff8b72b379bab8 RDI: ffffffffc06b6070
RBP: ffff8b72b379baf8 R08: ffffffffc06b6078 R09: ffff8b729947b400
R10: ffff8b73e9b09b28 R11: 0000000000000000 R12: ffffffffc06b6078
R13: ffff8b729947b400 R14: ffff8b73e9b09b28 R15: 0000000000000000
FS: 00007f9112dd9880(0000) GS:ffff8b743f680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555eac84f800 CR3: 00000000b3470000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc068d371>] ? nfs_wait_client_init_complete+0xa1/0xe0 [nfs]
[<ffffffffb26c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffc068eeab>] nfs_get_client+0x22b/0x470 [nfs]
[<ffffffffc06f839c>] nfs4_set_client+0x9c/0x150 [nfsv4]
[<ffffffffc06f9b4e>] nfs4_create_server+0x13e/0x3b0 [nfsv4]
[<ffffffffc06f0bae>] nfs4_remote_mount+0x2e/0x60 [nfsv4]
[<ffffffffb2845bde>] mount_fs+0x3e/0x1b0
[<ffffffffb27daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffb2863777>] vfs_kern_mount+0x67/0x110
[<ffffffffc06f0ad6>] nfs_do_root_mount+0x86/0xc0 [nfsv4]
[<ffffffffc06f0ed4>] nfs4_try_mount+0x44/0xc0 [nfsv4]
[<ffffffffc068f6e7>] ? get_nfs_version+0x27/0x90 [nfs]
[<ffffffffc069bd3b>] nfs_fs_mount+0x4cb/0xdc0 [nfs]
[<ffffffffc069cca0>] ? nfs_clone_super+0x140/0x140 [nfs]
[<ffffffffc069ab40>] ? param_set_portnr+0x70/0x70 [nfs]
[<ffffffffb2845bde>] mount_fs+0x3e/0x1b0
[<ffffffffb27daff5>] ? __alloc_percpu+0x15/0x20
[<ffffffffb2863777>] vfs_kern_mount+0x67/0x110
[<ffffffffb2865d9f>] do_mount+0x1ef/0xce0
[<ffffffffb283e03a>] ? __check_object_size+0x1ca/0x250
[<ffffffffb281c4dc>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffffb2866bd3>] SyS_mount+0x83/0xd0
[<ffffffffb2d74ddb>] system_call_fastpath+0x22/0x27
Code: 7c 24 18 49 89 c5 e8 aa 21 2d 00 4c 89 ee 48 89 df e8 ef 77 6a 00 5b 41 5c 41 5d 5d c3 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 <65> 48 8b 04 25 80 0e 01 00 48 89 e5 41 56 41 55 49 89 fd 41 54