Info

Permalink
Function
native_queued_spin_lock_slowpath
First occurrence
2021-03-20
Last occurrence
2021-03-20
State
NEW
Components
kernel
Quality
-42

Statistics

Operating system Count
CentOS 7 4
Architecture Count
x86_64 4
Related packages Count
kernel 4
 0:3.10.0-862.3.2.el7 2
 0:3.10.0-862.14.4.el7 1
 0:3.10.0-693.21.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

No reports in the last 20 weeks.

Report backtrace


Complete report #104157
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffbd70ba96
Build id: 3.10.0-862.14.4.el7.x86_64
0x126
2
queued_spin_lock_slowpath
vmlinux 0xffffffffbdd0dc5f
Build id: 3.10.0-862.14.4.el7.x86_64
0xb
3
_raw_spin_lock
vmlinux 0xffffffffbdd1b750
Build id: 3.10.0-862.14.4.el7.x86_64
0x20
4
ext4_es_lru_del
ext4 0xffffffffc048f4f2
Build id: 3.10.0-862.14.4.el7.x86_64
0x32
5
ext4_clear_inode
ext4 0xffffffffc046d751
Build id: 3.10.0-862.14.4.el7.x86_64
0x41
6
ext4_free_inode
ext4 0xffffffffc0447516
Build id: 3.10.0-862.14.4.el7.x86_64
0x116
7
? ext4_evict_inode
ext4 0xffffffffc04512e7
Build id: 3.10.0-862.14.4.el7.x86_64
0x337
8
? ext4_mark_inode_dirty
ext4 0xffffffffc044efaf
Build id: 3.10.0-862.14.4.el7.x86_64
0x6f
9
ext4_evict_inode
ext4 0xffffffffc04513a7
Build id: 3.10.0-862.14.4.el7.x86_64
0x3f7
10
evict
vmlinux 0xffffffffbd83c504
Build id: 3.10.0-862.14.4.el7.x86_64
0xb4
11
iput
vmlinux 0xffffffffbd83ce0c
Build id: 3.10.0-862.14.4.el7.x86_64
0xfc
12
__dentry_kill
vmlinux 0xffffffffbd8376f0
Build id: 3.10.0-862.14.4.el7.x86_64
0x120
13
dput
vmlinux 0xffffffffbd8377f9
Build id: 3.10.0-862.14.4.el7.x86_64
0xa9
14
SYSC_renameat2
vmlinux 0xffffffffbd8318a8
Build id: 3.10.0-862.14.4.el7.x86_64
0x518
15
? handle_mm_fault
vmlinux 0xffffffffbd7caefd
Build id: 3.10.0-862.14.4.el7.x86_64
0x39d
16
? __do_page_fault
vmlinux 0xffffffffbdd2056c
Build id: 3.10.0-862.14.4.el7.x86_64
0x1bc
17
SyS_renameat2
vmlinux 0xffffffffbd8327ae
Build id: 3.10.0-862.14.4.el7.x86_64
0xe
18
SyS_rename
vmlinux 0xffffffffbd8327ee
Build id: 3.10.0-862.14.4.el7.x86_64
0x1e
19
system_call_fastpath
vmlinux 0xffffffffbdd2579b
Build id: 3.10.0-862.14.4.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#28 stuck for 22s! [mv:451107]
Modules linked in: binfmt_misc tcp_diag inet_diag bonding sunrpc dm_mirror dm_region_hash dm_log dm_mod skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul iTCO_wdt dell_smbios iTCO_vendor_support sparse_keymap ghash_clmulni_intel dcdbas aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg mei_me mei lpc_ich shpchp i2c_i801 wmi nfit libnvdimm acpi_pad acpi_power_meter ipmi_si ipmi_devintf ipmi_msghandler ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ahci crc32c_intel drm libahci igb libata nvme megaraid_sas nvme_core i2c_algo_bit i2c_core ixgbe mdio dca i40e mlx5_core mlxfw devlink ptp pps_core
CPU: 28 PID: 451107 Comm: mv Tainted: G             L ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R640/0PHYDR, BIOS 1.6.11 11/20/2018
task: ffffa00945805ee0 ti: ffffa00d44068000 task.ti: ffffa00d44068000
RIP: 0010:[<ffffffffbd70ba96>]  [<ffffffffbd70ba96>] native_queued_spin_lock_slowpath+0x126/0x200
RSP: 0018:ffffa00d4406bb88  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffa00e30473ee2 RCX: 0000000000e10000
RDX: ffffa0351c199740 RSI: 0000000000e90101 RDI: ffffa03519a18480
RBP: ffffa00d4406bb88 R08: ffffa01d1dd99740 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000060562616 R12: ffffa00e30473ee0
R13: ffffffffbd6beed5 R14: ffffa00d4406bb00 R15: 00000000a939300f
FS:  00007fe60993b840(0000) GS:ffffa01d1dd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe608d92c80 CR3: 0000000338d34000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
 [<ffffffffbdd0dc5f>] queued_spin_lock_slowpath+0xb/0xf
 [<ffffffffbdd1b750>] _raw_spin_lock+0x20/0x30
 [<ffffffffc048f4f2>] ext4_es_lru_del+0x32/0x70 [ext4]
 [<ffffffffc046d751>] ext4_clear_inode+0x41/0x90 [ext4]
 [<ffffffffc0447516>] ext4_free_inode+0x116/0x610 [ext4]
 [<ffffffffc04512e7>] ? ext4_evict_inode+0x337/0x4b0 [ext4]
 [<ffffffffc044efaf>] ? ext4_mark_inode_dirty+0x6f/0x210 [ext4]
 [<ffffffffc04513a7>] ext4_evict_inode+0x3f7/0x4b0 [ext4]
 [<ffffffffbd83c504>] evict+0xb4/0x180
 [<ffffffffbd83ce0c>] iput+0xfc/0x190
 [<ffffffffbd8376f0>] __dentry_kill+0x120/0x180
 [<ffffffffbd8377f9>] dput+0xa9/0x160
 [<ffffffffbd8318a8>] SYSC_renameat2+0x518/0x5a0
 [<ffffffffbd7caefd>] ? handle_mm_fault+0x39d/0x9b0
 [<ffffffffbdd2056c>] ? __do_page_fault+0x1bc/0x4f0
 [<ffffffffbd8327ae>] SyS_renameat2+0xe/0x10
 [<ffffffffbd8327ee>] SyS_rename+0x1e/0x20
 [<ffffffffbdd2579b>] system_call_fastpath+0x22/0x27
Code: 0d 48 98 83 e2 30 48 81 c2 40 97 01 00 48 03 14 c5 a0 54 33 be 4c 89 02 41 8b 40 08 85 c0 75 0f 0f 1f 44 00 00 f3 90 41 8b 40 08 <85> c0 74 f6 4d 8b 08 4d 85 c9 74 04 41 0f 18 09 8b 17 0f b7 c2 
Complete report #1417381
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff810fdfe6
Build id: 3.10.0-693.21.1.el7.x86_64
0x156
2
queued_spin_lock_slowpath
vmlinux 0xffffffff816a8ff4
Build id: 3.10.0-693.21.1.el7.x86_64
0xb
3
_raw_spin_lock
vmlinux 0xffffffff816b6850
Build id: 3.10.0-693.21.1.el7.x86_64
0x20
4
ext4_es_lru_del
ext4 0xffffffffc0272a42
Build id: 3.10.0-693.21.1.el7.x86_64
0x32
5
ext4_clear_inode
ext4 0xffffffffc0250b61
Build id: 3.10.0-693.21.1.el7.x86_64
0x41
6
ext4_free_inode
ext4 0xffffffffc022ac16
Build id: 3.10.0-693.21.1.el7.x86_64
0x116
7
? ext4_evict_inode
ext4 0xffffffffc02348b7
Build id: 3.10.0-693.21.1.el7.x86_64
0x337
8
? ext4_mark_inode_dirty
ext4 0xffffffffc023257f
Build id: 3.10.0-693.21.1.el7.x86_64
0x6f
9
ext4_evict_inode
ext4 0xffffffffc0234977
Build id: 3.10.0-693.21.1.el7.x86_64
0x3f7
10
evict
vmlinux 0xffffffff8122245c
Build id: 3.10.0-693.21.1.el7.x86_64
0xac
11
iput
vmlinux 0xffffffff81222d6c
Build id: 3.10.0-693.21.1.el7.x86_64
0xfc
12
dentry_kill
vmlinux 0xffffffff8121d4c8
Build id: 3.10.0-693.21.1.el7.x86_64
0x178
13
dput
vmlinux 0xffffffff8121d56e
Build id: 3.10.0-693.21.1.el7.x86_64
0x5e
14
SYSC_renameat2
vmlinux 0xffffffff81217718
Build id: 3.10.0-693.21.1.el7.x86_64
0x518
15
? __sb_end_write
vmlinux 0xffffffff81208541
Build id: 3.10.0-693.21.1.el7.x86_64
0x31
16
? mnt_drop_write
vmlinux 0xffffffff81225cbf
Build id: 3.10.0-693.21.1.el7.x86_64
0x1f
17
? mntput
vmlinux 0xffffffff812275d4
Build id: 3.10.0-693.21.1.el7.x86_64
0x24
18
? path_put
vmlinux 0xffffffff8121000e
Build id: 3.10.0-693.21.1.el7.x86_64
0x1e
19
SyS_renameat2
vmlinux 0xffffffff812185ee
Build id: 3.10.0-693.21.1.el7.x86_64
0xe
20
SyS_rename
vmlinux 0xffffffff8121862e
Build id: 3.10.0-693.21.1.el7.x86_64
0x1e
21
system_call_fastpath
vmlinux 0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#54 stuck for 22s! [datmod:314988]
Modules linked in: udp_diag tcp_diag inet_diag binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bonding dm_mirror dm_region_hash dm_log dm_mod skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt iTCO_vendor_support vfat fat pcspkr cdc_ether usbnet mii uas sg usb_storage i2c_i801 shpchp lpc_ich mei_me mei wmi ipmi_ssif ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_pad acpi_power_meter nfsd auth_rpcgss nfs_acl
lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops i40e ttm igb drm ahci libahci dca ptp i2c_algo_bit pps_core libata megaraid_sas(OE) i2c_core
CPU: 54 PID: 314988 Comm: datmod Tainted: G           OEL ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: Lenovo ThinkSystem SR850 -[7X19CTO1WW]-/-[7X19CTO1WW]-, BIOS -[TEE164L-2.80]- 10/23/2020
task: ffff881901064f10 ti: ffff882325ff8000 task.ti: ffff882325ff8000
RIP: 0010:[<ffffffff810fdfe6>]  [<ffffffff810fdfe6>] native_queued_spin_lock_slowpath+0x156/0x200
RSP: 0018:ffff882325ffbb90  EFLAGS: 00000202
RAX: 0000000000000101 RBX: ffff880d4149be30 RCX: 0000000001b10000
RDX: 0000000001710101 RSI: 0000000000000101 RDI: ffff8840a2477c80
RBP: ffff882325ffbb90 R08: ffff884ffdd19600 R09: 0000000000000000
R10: 0000000000000000 R11: 000000006a496021 R12: ffff880d4149be30
R13: ffff88461e4109c0 R14: ffffffff810b4f85 R15: ffff882325ffbb08
FS:  00007fd098a16740(0000) GS:ffff884ffdd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fff213f4950 CR3: 0000002516a76000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff816a8ff4>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff816b6850>] _raw_spin_lock+0x20/0x30
[<ffffffffc0272a42>] ext4_es_lru_del+0x32/0x70 [ext4]
[<ffffffffc0250b61>] ext4_clear_inode+0x41/0x90 [ext4]
[<ffffffffc022ac16>] ext4_free_inode+0x116/0x610 [ext4]
[<ffffffffc02348b7>] ? ext4_evict_inode+0x337/0x4b0 [ext4]
[<ffffffffc023257f>] ? ext4_mark_inode_dirty+0x6f/0x210 [ext4]
[<ffffffffc0234977>] ext4_evict_inode+0x3f7/0x4b0 [ext4]
[<ffffffff8122245c>] evict+0xac/0x180
[<ffffffff81222d6c>] iput+0xfc/0x190
[<ffffffff8121d4c8>] dentry_kill+0x178/0x1c0
[<ffffffff8121d56e>] dput+0x5e/0xe0
[<ffffffff81217718>] SYSC_renameat2+0x518/0x5a0
[<ffffffff81208541>] ? __sb_end_write+0x31/0x60
[<ffffffff81225cbf>] ? mnt_drop_write+0x1f/0x30
[<ffffffff812275d4>] ? mntput+0x24/0x40
[<ffffffff8121000e>] ? path_put+0x1e/0x30
[<ffffffff812185ee>] SyS_renameat2+0xe/0x10
[<ffffffff8121862e>] SyS_rename+0x1e/0x20
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 8b 08 4d 85 c9 74 04 41 0f 18 09 8b 17 0f b7 c2 85 c0 74 21 83 f8 03 75 10 eb 1a 66 2e 0f 1f 84 00 00 00 00 00 85 c0 74 0c f3 90 <8b> 17 0f b7 c2 83 f8 03 75 f0 be 01 00 00 00 eb 15 66 0f 1f 84 
Complete report #653411
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffa0708b32
Build id: 3.10.0-862.3.2.el7.x86_64
0x122
2
queued_spin_lock_slowpath
vmlinux 0xffffffffa0d09450
Build id: 3.10.0-862.3.2.el7.x86_64
0xb
3
_raw_spin_lock
vmlinux 0xffffffffa0d16780
Build id: 3.10.0-862.3.2.el7.x86_64
0x20
4
ext4_es_lru_del
ext4 0xffffffffc05304a2
Build id: 3.10.0-862.3.2.el7.x86_64
0x32
5
ext4_clear_inode
ext4 0xffffffffc050e701
Build id: 3.10.0-862.3.2.el7.x86_64
0x41
6
ext4_free_inode
ext4 0xffffffffc04e8516
Build id: 3.10.0-862.3.2.el7.x86_64
0x116
7
? ext4_evict_inode
ext4 0xffffffffc04f22e7
Build id: 3.10.0-862.3.2.el7.x86_64
0x337
8
? ext4_mark_inode_dirty
ext4 0xffffffffc04effaf
Build id: 3.10.0-862.3.2.el7.x86_64
0x6f
9
ext4_evict_inode
ext4 0xffffffffc04f23a7
Build id: 3.10.0-862.3.2.el7.x86_64
0x3f7
10
evict
vmlinux 0xffffffffa0838394
Build id: 3.10.0-862.3.2.el7.x86_64
0xb4
11
iput
vmlinux 0xffffffffa0838c9c
Build id: 3.10.0-862.3.2.el7.x86_64
0xfc
12
__dentry_kill
vmlinux 0xffffffffa0833520
Build id: 3.10.0-862.3.2.el7.x86_64
0x120
13
dput
vmlinux 0xffffffffa0833629
Build id: 3.10.0-862.3.2.el7.x86_64
0xa9
14
SYSC_renameat2
vmlinux 0xffffffffa082d688
Build id: 3.10.0-862.3.2.el7.x86_64
0x518
15
? __sb_end_write
vmlinux 0xffffffffa081df11
Build id: 3.10.0-862.3.2.el7.x86_64
0x31
16
? mnt_drop_write
vmlinux 0xffffffffa083be1f
Build id: 3.10.0-862.3.2.el7.x86_64
0x1f
17
? mntput
vmlinux 0xffffffffa083d9b4
Build id: 3.10.0-862.3.2.el7.x86_64
0x24
18
SyS_renameat2
vmlinux 0xffffffffa082e58e
Build id: 3.10.0-862.3.2.el7.x86_64
0xe
19
SyS_rename
vmlinux 0xffffffffa082e5ce
Build id: 3.10.0-862.3.2.el7.x86_64
0x1e
20
system_call_fastpath
vmlinux 0xffffffffa0d2082f
Build id: 3.10.0-862.3.2.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [node:31659]
Modules linked in: binfmt_misc vsock_diag vsock sctp_diag sctp unix_diag af_packet_diag netlink_diag tcp_diag udp_diag inet_diag veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_round_robin iscsi_tcp libiscsi_tcp bonding ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm rpcrdma dm_mirror dm_region_hash dm_log sunrpc rdma_ucm ib_uverbs ib_iser ib_umad rdma_cm ib_ipoib iw_cm libiscsi scsi_transport_iscsi ib_cm iTCO_wdt iTCO_vendor_support sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm vfat
fat irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd mlx4_ib ib_core pcspkr dm_multipath sg dm_mod hpilo ioatdma hpwdt lpc_ich i2c_i801 shpchp dca wmi ipmi_si ipmi_devintf ipmi_msghandler pcc_cpufreq acpi_power_meter ip_tables ext4 mbcache jbd2 mlx4_en sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops bnx2x ttm drm mlx4_core crct10dif_pclmul crct10dif_common crc32c_intel tg3 hpsa i2c_core mdio libcrc32c ptp devlink scsi_transport_sas pps_core
CPU: 9 PID: 31659 Comm: node Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.3.2.el7.x86_64 #1
Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 12/27/2015
task: ffff8e7ccddc2f70 ti: ffff8eb625518000 task.ti: ffff8eb625518000
RIP: 0010:[<ffffffffa0708b32>]  [<ffffffffa0708b32>] native_queued_spin_lock_slowpath+0x122/0x200
RSP: 0018:ffff8eb62551bb88  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8eb62551bb18 RCX: 0000000000490000
RDX: ffff8eb9bf0d9700 RSI: 0000000000690001 RDI: ffff8e8ab6e9e480
RBP: ffff8eb62551bb88 R08: ffff8e89bfa59700 R09: 0000000000000000
R10: 00000000b41a9701 R11: ffffee2236d06a40 R12: ffff8e99d2429900
R13: ffff8e97b41a9550 R14: ffffffffa07f6a42 R15: ffff8eb62551bb00
FS:  00007fbf6f7fe700(0000) GS:ffff8e89bfa40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000029a97fa54000 CR3: 0000005ebfe9a000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa0d09450>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa0d16780>] _raw_spin_lock+0x20/0x30
[<ffffffffc05304a2>] ext4_es_lru_del+0x32/0x70 [ext4]
[<ffffffffc050e701>] ext4_clear_inode+0x41/0x90 [ext4]
[<ffffffffc04e8516>] ext4_free_inode+0x116/0x610 [ext4]
[<ffffffffc04f22e7>] ? ext4_evict_inode+0x337/0x4b0 [ext4]
[<ffffffffc04effaf>] ? ext4_mark_inode_dirty+0x6f/0x210 [ext4]
[<ffffffffc04f23a7>] ext4_evict_inode+0x3f7/0x4b0 [ext4]
[<ffffffffa0838394>] evict+0xb4/0x180
[<ffffffffa0838c9c>] iput+0xfc/0x190
[<ffffffffa0833520>] __dentry_kill+0x120/0x180
[<ffffffffa0833629>] dput+0xa9/0x160
[<ffffffffa082d688>] SYSC_renameat2+0x518/0x5a0
[<ffffffffa081df11>] ? __sb_end_write+0x31/0x60
[<ffffffffa083be1f>] ? mnt_drop_write+0x1f/0x30
[<ffffffffa083d9b4>] ? mntput+0x24/0x40
[<ffffffffa082e58e>] SyS_renameat2+0xe/0x10
[<ffffffffa082e5ce>] SyS_rename+0x1e/0x20
[<ffffffffa0d2082f>] system_call_fastpath+0x1c/0x21
Code: 13 48 c1 ea 0d 48 98 83 e2 30 48 81 c2 00 97 01 00 48 03 14 c5 e0 52 33 a1 4c 89 02 41 8b 40 08 85 c0 75 0f 0f 1f 44 00 00 f3 90 <41> 8b 40 08 85 c0 74 f6 4d 8b 08 4d 85 c9 74 04 41 0f 18 09 8b