Info

Permalink
Function
_raw_spin_unlock_irqrestore
First occurrence
2020-09-26
Last occurrence
2026-04-06
State
NEW
Components
kernel
Quality
-41

Statistics

Operating system Count
CentOS 7 150
Architecture Count
x86_64 150
Related packages Count
kernel 150
 0:3.10.0-514.16.1.el7 87
 0:3.10.0-327.el7 24
 0:3.10.0-693.el7 16
 0:3.10.0-693.21.1.el7 7
 0:3.10.0-693.11.1.el7 4
 0:3.10.0-514.10.2.el7 2
 0:3.10.0-514.6.1.el7 2
 0:3.10.0-514.el7 2
 0:3.10.0-693.5.2.el7 2
 0:3.10.0-514.26.2.el7 1
 0:3.10.0-693.17.1.el7 1
 0:3.10.0-123.13.2.el7 1
 0:3.10.0-123.20.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1174582
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00503be
Build id: 3.10.0-693.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc0072729
Build id: 3.10.0-693.el7.x86_64 		0x249
3
mptspi_qcmd
mptspi 0xffffffffc00ad2b0
Build id: 3.10.0-693.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64 		0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64 		0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64 		0x33
7
queue_unplugged
vmlinux 0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64 		0x14
10
__xfs_buf_delwri_submit
xfs 0xffffffffc02a3ea9
Build id: 3.10.0-693.el7.x86_64 		0x1b9
11
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffc02a4bcf
Build id: 3.10.0-693.el7.x86_64 		0x2f
12
? xfs_trans_ail_cursor_first
xfs 0xffffffffc02d2a10
Build id: 3.10.0-693.el7.x86_64 		0x90
13
xfs_buf_delwri_submit_nowait
xfs 0xffffffffc02a4bcf
Build id: 3.10.0-693.el7.x86_64 		0x2f
14
xfsaild
xfs 0xffffffffc02d2c6f
Build id: 3.10.0-693.el7.x86_64 		0x25f
15
? xfs_trans_ail_cursor_first
xfs 0xffffffffc02d2a10
Build id: 3.10.0-693.el7.x86_64 		0x90
16
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64 		0xcf
17
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40
18
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64 		0x58
19
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [xfsaild/dm-0:431]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmhgfs(OE) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device crc32_pclmul snd_pcm ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_timer sg pcspkr nfit joydev snd soundcore libnvdimm i2c_piix4 vmw_vmci shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables
xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata e1000 crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi serio_raw mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 431 Comm: xfsaild/dm-0 Tainted: G           OEL ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff880231472f70 ti: ffff88022fba4000 task.ti: ffff88022fba4000
RIP: 0010:[<ffffffffc00503be>]  [<ffffffffc00503be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff88022fba7b10  EFLAGS: 00000246
RAX: ffffc90000d00000 RBX: ffff880198fd2f00 RCX: 000000000000000d
RDX: ffff880035482400 RSI: ffff8802375b2000 RDI: 000000000000000e
RBP: ffff88022fba7b20 R08: 0000000000000002 R09: ffff8802375b60d8
R10: ffff8801f37def40 R11: ffff8801f37def40 R12: 0000000100000001
R13: ffff88021d39f480 R14: ffff88022fba7a84 R15: ffffffff812ff13b
FS:  0000000000000000(0000) GS:ffff880235600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f126b51b000 CR3: 00000000019f2000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8802375b2d30 000000000000003c ffff88022fba7bc8 ffffffffc0072729
ffff8802375b2008 040000002fba7b88 000000000000000d ffff8801f37def40
0000000000000034 ffff8802375b2188 000000600000000d ffff880230102d10
Call Trace:
[<ffffffffc0072729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00ad2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffffc02a3ea9>] __xfs_buf_delwri_submit+0x1b9/0x250 [xfs]
[<ffffffffc02a4bcf>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc02d2a10>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffc02a4bcf>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc02d2c6f>] xfsaild+0x25f/0x6f0 [xfs]
[<ffffffffc02d2a10>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 05
Complete report #2686
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffa012f3be
Build id: 3.10.0-327.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffa014a729
Build id: 3.10.0-327.el7.x86_64 		0x249
3
mptspi_qcmd
mptspi 0xffffffffa015e2b0
Build id: 3.10.0-327.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff81417b1a
Build id: 3.10.0-327.el7.x86_64 		0xaa
5
scsi_request_fn
vmlinux 0xffffffff81420aa1
Build id: 3.10.0-327.el7.x86_64 		0x501
6
__blk_run_queue
vmlinux 0xffffffff812c73e3
Build id: 3.10.0-327.el7.x86_64 		0x33
7
queue_unplugged
vmlinux 0xffffffff812c749a
Build id: 3.10.0-327.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812cbd18
Build id: 3.10.0-327.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812cc124
Build id: 3.10.0-327.el7.x86_64 		0x14
10
__xfs_buf_delwri_submit
xfs 0xffffffffa023aa79
Build id: 3.10.0-327.el7.x86_64 		0x1e9
11
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa023b67f
Build id: 3.10.0-327.el7.x86_64 		0x2f
12
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0266470
Build id: 3.10.0-327.el7.x86_64 		0x90
13
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa023b67f
Build id: 3.10.0-327.el7.x86_64 		0x2f
14
xfsaild
xfs 0xffffffffa02666b0
Build id: 3.10.0-327.el7.x86_64 		0x240
15
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0266470
Build id: 3.10.0-327.el7.x86_64 		0x90
16
kthread
vmlinux 0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64 		0xcf
17
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64 		0x140
18
ret_from_fork
vmlinux 0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64 		0x58
19
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64 		0x140 
BUG: soft lockup - CPU#1 stuck for 150s! [xfsaild/dm-0:400]
Modules linked in: ppdev coretemp vmw_balloon pcspkr sg vmw_vmci shpchp parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi crc32c_intel serio_raw vmwgfx drm_kms_helper ttm mptspi scsi_transport_spi mptscsih vmxnet3 mptbase ata_piix drm i2c_core libata floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 400 Comm: xfsaild/dm-0 Tainted: G             L ------------   3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff880036887300 ti: ffff88003589c000 task.ti: ffff88003589c000
RIP: 0010:[<ffffffffa012f3be>]  [<ffffffffa012f3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff88003589fb00  EFLAGS: 00000246
RAX: ffffc90008400000 RBX: ffff880036a74500 RCX: 0000000000000014
RDX: ffff8800369cd400 RSI: ffff880078f22000 RDI: 000000000000000e
RBP: ffff88003589fb10 R08: 0000000000000002 R09: 000000007b270fff
R10: ffff88007a322fc0 R11: ffff88007a322fc0 R12: 0000000100000001
R13: ffff88006e814480 R14: ffff88003589fa74 R15: ffffffff812d06bb
FS:  0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f8d3dac9000 CR3: 000000007a7b7000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880078f22d30 0000000000000038 ffff88003589fbb8 ffffffffa014a729
ffff880078f22008 040000003589fb78 0000000000000014 ffff88007a322fc0
0000000000000050 ffff880078f22188 0000006000000014 ffff8800361c2fb0
Call Trace:
[<ffffffffa014a729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa015e2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff81417b1a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff81420aa1>] scsi_request_fn+0x501/0x770
[<ffffffff812c73e3>] __blk_run_queue+0x33/0x40
[<ffffffff812c749a>] queue_unplugged+0x2a/0xa0
[<ffffffff812cbd18>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812cc124>] blk_finish_plug+0x14/0x40
[<ffffffffa023aa79>] __xfs_buf_delwri_submit+0x1e9/0x250 [xfs]
[<ffffffffa023b67f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0266470>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa023b67f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa02666b0>] xfsaild+0x240/0x5e0 [xfs]
[<ffffffffa0266470>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 13
Complete report #65606
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff816ab915
Build id: 3.10.0-693.11.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0052632
Build id: 3.10.0-693.11.1.el7.x86_64 		0x442
3
? scsi_test_unit_ready
vmlinux 0xffffffff8146d0e0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146535a
Build id: 3.10.0-693.11.1.el7.x86_64 		0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e11f
Build id: 3.10.0-693.11.1.el7.x86_64 		0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f7113
Build id: 3.10.0-693.11.1.el7.x86_64 		0x33
7
queue_unplugged
vmlinux 0xffffffff812f71ca
Build id: 3.10.0-693.11.1.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fa658
Build id: 3.10.0-693.11.1.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812faa64
Build id: 3.10.0-693.11.1.el7.x86_64 		0x14
10
__xfs_buf_delwri_submit
xfs 0xffffffffc0260ec9
Build id: 3.10.0-693.11.1.el7.x86_64 		0x1b9
11
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffc0261bef
Build id: 3.10.0-693.11.1.el7.x86_64 		0x2f
12
? xfs_trans_ail_cursor_first
xfs 0xffffffffc028f9d0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x90
13
xfs_buf_delwri_submit_nowait
xfs 0xffffffffc0261bef
Build id: 3.10.0-693.11.1.el7.x86_64 		0x2f
14
xfsaild
xfs 0xffffffffc028fc2f
Build id: 3.10.0-693.11.1.el7.x86_64 		0x25f
15
? xfs_trans_ail_cursor_first
xfs 0xffffffffc028f9d0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x90
16
kthread
vmlinux 0xffffffff810b099f
Build id: 3.10.0-693.11.1.el7.x86_64 		0xcf
17
? insert_kthread_work
vmlinux 0xffffffff810b08d0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x40
18
ret_from_fork
vmlinux 0xffffffff816b4fd8
Build id: 3.10.0-693.11.1.el7.x86_64 		0x58
19
? insert_kthread_work
vmlinux 0xffffffff810b08d0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 33s! [xfsaild/dm-2:793]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon nfit pcspkr joydev sg libnvdimm parport_pc vmw_vmci shpchp parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx ahci drm_kms_helper libahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel
drm serio_raw libata vmxnet3 vmw_pvscsi i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 793 Comm: xfsaild/dm-2 Tainted: G             L ------------   3.10.0-693.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff880a032d6eb0 ti: ffff8809ff344000 task.ti: ffff8809ff344000
RIP: 0010:[<ffffffff816ab915>]  [<ffffffff816ab915>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8809ff347b78  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff8809ff347b38 RCX: 0000000035de5fff
RDX: ffffc900069a0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8809ff347b78 R08: 0000000000000001 R09: 0000000000000000
R10: ffff88094223c540 R11: ffff880a03393000 R12: 0000000000000020
R13: ffff8809bfeda180 R14: ffffffff8146bd09 R15: ffff8809ff347af0
FS:  0000000000000000(0000) GS:ffff880a2fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5aceb32100 CR3: 00000009f8302000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8809ff347bf0 ffffffffc0052632 ffff8809c0612900 ffff880035cdd000
ffff880035d13d40 ffff880a032277e8 ffffffff8146d0e0 0000000000000006
ffff880a03227754 0000000000000286 ffff88094223c540 0000000000000000
Call Trace:
[<ffffffffc0052632>] pvscsi_queue+0x442/0x7e1 [vmw_pvscsi]
[<ffffffff8146d0e0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8146535a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e11f>] scsi_request_fn+0x4df/0x680
[<ffffffff812f7113>] __blk_run_queue+0x33/0x40
[<ffffffff812f71ca>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa658>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812faa64>] blk_finish_plug+0x14/0x40
[<ffffffffc0260ec9>] __xfs_buf_delwri_submit+0x1b9/0x250 [xfs]
[<ffffffffc0261bef>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc028f9d0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffc0261bef>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc028fc2f>] xfsaild+0x25f/0x6f0 [xfs]
[<ffffffffc028f9d0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b099f>] kthread+0xcf/0xe0
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4fd8>] ret_from_fork+0x58/0x90
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #196894
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff815ea06b
Build id: 3.10.0-123.20.1.el7.x86_64 		0x1b
2
mptspi_qcmd
mptspi 0xffffffffa01902ed
Build id: 3.10.0-123.20.1.el7.x86_64 		0x8d
3
scsi_dispatch_cmd
vmlinux 0xffffffff813d4017
Build id: 3.10.0-123.20.1.el7.x86_64 		0xb7
4
scsi_request_fn
vmlinux 0xffffffff813dcad9
Build id: 3.10.0-123.20.1.el7.x86_64 		0x349
5
__blk_run_queue
vmlinux 0xffffffff81290ae3
Build id: 3.10.0-123.20.1.el7.x86_64 		0x33
6
queue_unplugged
vmlinux 0xffffffff81290b9a
Build id: 3.10.0-123.20.1.el7.x86_64 		0x2a
7
blk_flush_plug_list
vmlinux 0xffffffff81294588
Build id: 3.10.0-123.20.1.el7.x86_64 		0x1d8
8
blk_finish_plug
vmlinux 0xffffffff81294984
Build id: 3.10.0-123.20.1.el7.x86_64 		0x14
9
__xfs_buf_delwri_submit
xfs 0xffffffffa01c0bb9
Build id: 3.10.0-123.20.1.el7.x86_64 		0x1c9
10
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa01c168f
Build id: 3.10.0-123.20.1.el7.x86_64 		0x2f
11
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0222c90
Build id: 3.10.0-123.20.1.el7.x86_64 		0x90
12
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa01c168f
Build id: 3.10.0-123.20.1.el7.x86_64 		0x2f
13
xfsaild
xfs 0xffffffffa0222ed0
Build id: 3.10.0-123.20.1.el7.x86_64 		0x240
14
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0222c90
Build id: 3.10.0-123.20.1.el7.x86_64 		0x90
15
kthread
vmlinux 0xffffffff81085aff
Build id: 3.10.0-123.20.1.el7.x86_64 		0xcf
16
? kthread_create_on_node
vmlinux 0xffffffff81085a30
Build id: 3.10.0-123.20.1.el7.x86_64 		0x140
17
ret_from_fork
vmlinux 0xffffffff815f316c
Build id: 3.10.0-123.20.1.el7.x86_64 		0x7c
18
? kthread_create_on_node
vmlinux 0xffffffff81085a30
Build id: 3.10.0-123.20.1.el7.x86_64 		0x140 
BUG: soft lockup - CPU#1 stuck for 23s! [xfsaild/dm-1:383]
Modules linked in: binfmt_misc ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ip_tables sg coretemp ppdev crct10dif_pclmul crc32_pclmul crc32c_intel vmw_balloon ghash_clmulni_intel cryptd serio_raw pcspkr parport_pc parport i2c_piix4 vmw_vmci shpchp mperf nfsd auth_rpcgss nfs_acl lockd sunrpc uinput xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_common ata_generic pata_acpi mptspi vmwgfx scsi_transport_spi mptscsih e1000 mptbase ttm drm ata_piix
libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 383 Comm: xfsaild/dm-1 Tainted: G        W   --------------   3.10.0-123.20.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff8801364ee660 ti: ffff8800362f8000 task.ti: ffff8800362f8000
RIP: 0010:[<ffffffff815ea06b>]  [<ffffffff815ea06b>] _raw_spin_unlock_irqrestore+0x1b/0x40
RSP: 0018:ffff8800362f9ba0  EFLAGS: 00000296
RAX: 0000000000000000 RBX: 0000000000000009 RCX: 0000000000000009
RDX: ffff880036b79800 RSI: 0000000000000296 RDI: 0000000000000296
RBP: ffff8800362f9ba8 R08: 0000000000000002 R09: 0000000000000000
R10: ffff8800a73ad6c0 R11: 0000000000000000 R12: ffff8801360e8008
R13: ffffffffa0159729 R14: ffff8800362f9ba8 R15: 000000000000003c
FS:  0000000000000000(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007ff439143000 CR3: 00000000b7d95000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800a73ad6c0 ffff8800362f9be8 ffffffffa01902ed ffff88013606c3c8
ffff8800a73ad6c0 ffff88013606c3c8 ffff8801360eb000 ffff880036a96048
ffff8800a73ad6c0 ffff8800362f9c10 ffffffff813d4017 ffff880036a96000
Call Trace:
[<ffffffffa01902ed>] mptspi_qcmd+0x8d/0x120 [mptspi]
[<ffffffff813d4017>] scsi_dispatch_cmd+0xb7/0x290
[<ffffffff813dcad9>] scsi_request_fn+0x349/0x5a0
[<ffffffff81290ae3>] __blk_run_queue+0x33/0x40
[<ffffffff81290b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff81294588>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff81294984>] blk_finish_plug+0x14/0x40
[<ffffffffa01c0bb9>] __xfs_buf_delwri_submit+0x1c9/0x230 [xfs]
[<ffffffffa01c168f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0222c90>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa01c168f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0222ed0>] xfsaild+0x240/0x5e0 [xfs]
[<ffffffffa0222c90>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff81085aff>] kthread+0xcf/0xe0
[<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
[<ffffffff815f316c>] ret_from_fork+0x7c/0xb0
[<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
Code: 08 e8 8a c4 a7 ff 5d c3 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 53 48 89 f3 0f 1f 44 00 00 66 83 07 02 48 89 df 57 9d <0f> 1f 44 00 00 5b 5d c3 0f 1f 44 00 00 8b 37 f0 66 83 07 02 f6
Complete report #60681
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff815e988b
Build id: 3.10.0-123.13.2.el7.x86_64 		0x1b
2
mptspi_qcmd
mptspi 0xffffffffa004a2ed
Build id: 3.10.0-123.13.2.el7.x86_64 		0x8d
3
scsi_dispatch_cmd
vmlinux 0xffffffff813d3ae7
Build id: 3.10.0-123.13.2.el7.x86_64 		0xb7
4
scsi_request_fn
vmlinux 0xffffffff813dc599
Build id: 3.10.0-123.13.2.el7.x86_64 		0x349
5
__blk_run_queue
vmlinux 0xffffffff812905f3
Build id: 3.10.0-123.13.2.el7.x86_64 		0x33
6
queue_unplugged
vmlinux 0xffffffff812906aa
Build id: 3.10.0-123.13.2.el7.x86_64 		0x2a
7
blk_flush_plug_list
vmlinux 0xffffffff81294078
Build id: 3.10.0-123.13.2.el7.x86_64 		0x1d8
8
blk_finish_plug
vmlinux 0xffffffff81294474
Build id: 3.10.0-123.13.2.el7.x86_64 		0x14
9
__xfs_buf_delwri_submit
xfs 0xffffffffa01a7bb9
Build id: 3.10.0-123.13.2.el7.x86_64 		0x1c9
10
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa01a868f
Build id: 3.10.0-123.13.2.el7.x86_64 		0x2f
11
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0209c90
Build id: 3.10.0-123.13.2.el7.x86_64 		0x90
12
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa01a868f
Build id: 3.10.0-123.13.2.el7.x86_64 		0x2f
13
xfsaild
xfs 0xffffffffa0209ed0
Build id: 3.10.0-123.13.2.el7.x86_64 		0x240
14
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0209c90
Build id: 3.10.0-123.13.2.el7.x86_64 		0x90
15
kthread
vmlinux 0xffffffff81085aff
Build id: 3.10.0-123.13.2.el7.x86_64 		0xcf
16
? kthread_create_on_node
vmlinux 0xffffffff81085a30
Build id: 3.10.0-123.13.2.el7.x86_64 		0x140
17
ret_from_fork
vmlinux 0xffffffff815f296c
Build id: 3.10.0-123.13.2.el7.x86_64 		0x7c
18
? kthread_create_on_node
vmlinux 0xffffffff81085a30
Build id: 3.10.0-123.13.2.el7.x86_64 		0x140 
BUG: soft lockup - CPU#0 stuck for 24s! [xfsaild/dm-1:375]
Modules linked in: tcp_diag inet_diag ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter ip_tables sg coretemp crct10dif_pclmul crc32_pclmul crc32c_intel ppdev vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper serio_raw ablk_helper cryptd parport_pc parport pcspkr vmw_vmci shpchp i2c_piix4 mperf nfsd auth_rpcgss nfs_acl lockd sunrpc uinput xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_common vmwgfx ttm drm ata_piix
mptspi scsi_transport_spi libata vmxnet3 mptscsih mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 375 Comm: xfsaild/dm-1 Not tainted 3.10.0-123.13.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff8801b3635b00 ti: ffff8801b30d0000 task.ti: ffff8801b30d0000
RIP: 0010:[<ffffffff815e988b>]  [<ffffffff815e988b>] _raw_spin_unlock_irqrestore+0x1b/0x40
RSP: 0018:ffff8801b30d1ba0  EFLAGS: 00000296
RAX: 0000000000000000 RBX: 0000000000000028 RCX: 0000000000000028
RDX: ffff8801b304aa00 RSI: 0000000000000296 RDI: 0000000000000296
RBP: ffff8801b30d1ba8 R08: 0000000000000003 R09: 0000000000000000
R10: ffff8800b8075540 R11: 0000000000000000 R12: ffff8800368eb008
R13: ffffffffa0085729 R14: ffff8801b30d1ba8 R15: 0000000000000060
FS:  0000000000000000(0000) GS:ffff8801bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fa0cf1e1008 CR3: 00000001b4311000 CR4: 00000000000407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800b8075540 ffff8801b30d1be8 ffffffffa004a2ed ffff8801b37ec000
ffff8800b8075540 ffff8801b37ec000 ffff8800368ee000 ffff8801b37a3048
ffff8800b8075540 ffff8801b30d1c10 ffffffff813d3ae7 ffff8801b37a3000
Call Trace:
[<ffffffffa004a2ed>] mptspi_qcmd+0x8d/0x120 [mptspi]
[<ffffffff813d3ae7>] scsi_dispatch_cmd+0xb7/0x290
[<ffffffff813dc599>] scsi_request_fn+0x349/0x5a0
[<ffffffff812905f3>] __blk_run_queue+0x33/0x40
[<ffffffff812906aa>] queue_unplugged+0x2a/0xa0
[<ffffffff81294078>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff81294474>] blk_finish_plug+0x14/0x40
[<ffffffffa01a7bb9>] __xfs_buf_delwri_submit+0x1c9/0x230 [xfs]
[<ffffffffa01a868f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0209c90>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa01a868f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0209ed0>] xfsaild+0x240/0x5e0 [xfs]
[<ffffffffa0209c90>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff81085aff>] kthread+0xcf/0xe0
[<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
[<ffffffff815f296c>] ret_from_fork+0x7c/0xb0
[<ffffffff81085a30>] ? kthread_create_on_node+0x140/0x140
Code: 08 e8 6a cc a7 ff 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 53 48 89 f3 66 66 66 66 90 66 83 07 02 48 89 df 57 9d <66> 66 90 66 90 5b 5d c3 0f 1f 44 00 00 8b 37 3e 66 83 07 02 f6