Info

Permalink
Function
mpt_put_msg_frame
First occurrence
2020-09-27
Last occurrence
2026-02-27
State
NEW
Components
kernel
Quality
-52

Statistics

Operating system Count
CentOS 7 1,398
Architecture Count
x86_64 1,398
Related packages Count
kernel 1,398
 0:3.10.0-693.5.2.el7 1,212
 0:3.10.0-327.36.1.el7 59
 0:3.10.0-514.10.2.el7 45
 0:3.10.0-327.36.2.el7 17
 0:3.10.0-514.26.2.el7 16
 0:3.10.0-693.11.6.el7 14
 0:3.10.0-327.el7 13
 0:3.10.0-693.el7 7
 0:3.10.0-693.21.1.el7 5
 0:3.10.0-514.16.1.el7 4
 0:3.10.0-514.21.2.el7 3
 0:3.10.0-514.el7 2
 0:3.10.0-693.2.2.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #2985
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00783be
Build id: 3.10.0-693.11.6.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc00d6729
Build id: 3.10.0-693.11.6.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffc00932b0
Build id: 3.10.0-693.11.6.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff814675aa
Build id: 3.10.0-693.11.6.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8147036f
Build id: 3.10.0-693.11.6.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f9373
Build id: 3.10.0-693.11.6.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812f942a
Build id: 3.10.0-693.11.6.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fc8b8
Build id: 3.10.0-693.11.6.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff812fcb3a
Build id: 3.10.0-693.11.6.el7.x86_64
0x22a
10
generic_make_request
vmlinux 0xffffffff812face5
Build id: 3.10.0-693.11.6.el7.x86_64
0x105
11
? __bio_add_page
vmlinux 0xffffffff8123cb18
Build id: 3.10.0-693.11.6.el7.x86_64
0x208
12
submit_bio
vmlinux 0xffffffff812faf60
Build id: 3.10.0-693.11.6.el7.x86_64
0x70
13
_xfs_buf_ioapply
xfs 0xffffffffc0276393
Build id: 3.10.0-693.11.6.el7.x86_64
0x2f3
14
? __xfs_buf_delwri_submit
xfs 0xffffffffc0277ef8
Build id: 3.10.0-693.11.6.el7.x86_64
0x168
15
xfs_buf_submit
xfs 0xffffffffc0277bdc
Build id: 3.10.0-693.11.6.el7.x86_64
0x6c
16
__xfs_buf_delwri_submit
xfs 0xffffffffc0277ef8
Build id: 3.10.0-693.11.6.el7.x86_64
0x168
17
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffc0278c6f
Build id: 3.10.0-693.11.6.el7.x86_64
0x2f
18
? xfs_trans_ail_cursor_first
xfs 0xffffffffc02a6a50
Build id: 3.10.0-693.11.6.el7.x86_64
0x90
19
xfs_buf_delwri_submit_nowait
xfs 0xffffffffc0278c6f
Build id: 3.10.0-693.11.6.el7.x86_64
0x2f
20
xfsaild
xfs 0xffffffffc02a6caf
Build id: 3.10.0-693.11.6.el7.x86_64
0x25f
21
? xfs_trans_ail_cursor_first
xfs 0xffffffffc02a6a50
Build id: 3.10.0-693.11.6.el7.x86_64
0x90
22
kthread
vmlinux 0xffffffff810b252f
Build id: 3.10.0-693.11.6.el7.x86_64
0xcf
23
? insert_kthread_work
vmlinux 0xffffffff810b2460
Build id: 3.10.0-693.11.6.el7.x86_64
0x40
24
ret_from_fork
vmlinux 0xffffffff816b8798
Build id: 3.10.0-693.11.6.el7.x86_64
0x58
25
? insert_kthread_work
vmlinux 0xffffffff810b2460
Build id: 3.10.0-693.11.6.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [xfsaild/dm-0:411]
Modules linked in: ppdev coretemp pcspkr joydev vmw_balloon sg parport_pc parport vmw_vmci shpchp i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm mptspi scsi_transport_spi mptscsih ata_piix libata mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 411 Comm: xfsaild/dm-0 Tainted: G             L ------------   3.10.0-693.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff880035c8dee0 ti: ffff880035d2c000 task.ti: ffff880035d2c000
RIP: 0010:[<ffffffffc00783be>]  [<ffffffffc00783be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880035d2f958  EFLAGS: 00000246
RAX: ffffc90000780000 RBX: ffff88009167d3b0 RCX: 000000000000000d
RDX: ffff8800366d4000 RSI: ffff880135d9a000 RDI: 000000000000000e
RBP: ffff880035d2f968 R08: 0000000000000003 R09: ffff880135d9f0d8
R10: ffff88000752b180 R11: ffff88000752b180 R12: ffffffff816b9bff
R13: ffffffff816b9c06 R14: ffffffff816b9c0d R15: ffffffff816b9c14
FS:  0000000000000000(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f45637c74f0 CR3: 00000000b8d78000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc00d6729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00932b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff814675aa>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8147036f>] scsi_request_fn+0x4df/0x680
[<ffffffff812f9373>] __blk_run_queue+0x33/0x40
[<ffffffff812f942a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fc8b8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fcb3a>] blk_queue_bio+0x22a/0x3a0
[<ffffffff812face5>] generic_make_request+0x105/0x310
[<ffffffff8123cb18>] ? __bio_add_page+0x208/0x2b0
[<ffffffff812faf60>] submit_bio+0x70/0x150
[<ffffffffc0276393>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffc0277ef8>] ? __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffc0277bdc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc0277ef8>] __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffc0278c6f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc02a6a50>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffc0278c6f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc02a6caf>] xfsaild+0x25f/0x6f0 [xfs]
[<ffffffffc02a6a50>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b252f>] kthread+0xcf/0xe0
[<ffffffff810b2460>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b8798>] ret_from_fork+0x58/0x90
[<ffffffff810b2460>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 08 
Complete report #196862
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8168e29b
Build id: 3.10.0-514.16.1.el7.x86_64
0x1b
2
pvscsi_queue
vmw_pvscsi 0xffffffffa005758a
Build id: 3.10.0-514.16.1.el7.x86_64
0x43a
3
? scsi_sg_free
vmlinux 0xffffffff8145a1b0
Build id: 3.10.0-514.16.1.el7.x86_64
0x60
4
scsi_dispatch_cmd
vmlinux 0xffffffff8145220a
Build id: 3.10.0-514.16.1.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8145b311
Build id: 3.10.0-514.16.1.el7.x86_64
0x501
6
__blk_run_queue
vmlinux 0xffffffff812eb6e3
Build id: 3.10.0-514.16.1.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812eb79a
Build id: 3.10.0-514.16.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812f05f8
Build id: 3.10.0-514.16.1.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff812f086a
Build id: 3.10.0-514.16.1.el7.x86_64
0x21a
10
generic_make_request
vmlinux 0xffffffff812eeb89
Build id: 3.10.0-514.16.1.el7.x86_64
0x109
11
submit_bio
vmlinux 0xffffffff812eecd1
Build id: 3.10.0-514.16.1.el7.x86_64
0x71
12
_xfs_buf_ioapply
xfs 0xffffffffa025ea43
Build id: 3.10.0-514.16.1.el7.x86_64
0x2f3
13
? __xfs_buf_delwri_submit
xfs 0xffffffffa02605e8
Build id: 3.10.0-514.16.1.el7.x86_64
0x168
14
xfs_buf_submit
xfs 0xffffffffa0260300
Build id: 3.10.0-514.16.1.el7.x86_64
0x70
15
__xfs_buf_delwri_submit
xfs 0xffffffffa02605e8
Build id: 3.10.0-514.16.1.el7.x86_64
0x168
16
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa026138f
Build id: 3.10.0-514.16.1.el7.x86_64
0x2f
17
? xfs_trans_ail_cursor_first
xfs 0xffffffffa028edd0
Build id: 3.10.0-514.16.1.el7.x86_64
0x90
18
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa026138f
Build id: 3.10.0-514.16.1.el7.x86_64
0x2f
19
xfsaild
xfs 0xffffffffa028f01d
Build id: 3.10.0-514.16.1.el7.x86_64
0x24d
20
? xfs_trans_ail_cursor_first
xfs 0xffffffffa028edd0
Build id: 3.10.0-514.16.1.el7.x86_64
0x90
21
kthread
vmlinux 0xffffffff810b0a4f
Build id: 3.10.0-514.16.1.el7.x86_64
0xcf
22
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.16.1.el7.x86_64
0x140
23
ret_from_fork
vmlinux 0xffffffff816970d8
Build id: 3.10.0-514.16.1.el7.x86_64
0x58
24
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.16.1.el7.x86_64
0x140
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [xfsaild/dm-1:448]
Modules linked in: binfmt_misc vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg i2c_piix4 vmw_vmci shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_piix vmwgfx drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm vmw_pvscsi i2c_core e1000 libata floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 448 Comm: xfsaild/dm-1 Tainted: G           OEL ------------   3.10.0-514.16.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880035c44e70 ti: ffff8800353ec000 task.ti: ffff8800353ec000
RIP: 0010:[<ffffffff8168e29b>]  [<ffffffff8168e29b>] _raw_spin_unlock_irqrestore+0x1b/0x40
RSP: 0018:ffff8800353ef9d0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8133ba38 RCX: 00000002317b9fff
RDX: ffffc90000c70000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8800353ef9d8 R08: 0000000000000001 R09: 0000000000000000
R10: ffff880234793d40 R11: ffff88023279b800 R12: ffff880234793d00
R13: 0000000000000020 R14: ffffffff81458d7f R15: ffff8800353ef950
FS:  0000000000000000(0000) GS:ffff880236640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f75214ae000 CR3: 00000001c4bed000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 ffff8800353efa50 ffffffffa005758a ffff88021325e000
ffff880231dd8000 ffff88023167a140 ffff88023279cfe8 ffffffff8145a1b0
0000000000000002 ffff88023279cf54 0000000000000286 ffff880234793d40
Call Trace:
[<ffffffffa005758a>] pvscsi_queue+0x43a/0x7e0 [vmw_pvscsi]
[<ffffffff8145a1b0>] ? scsi_sg_free+0x60/0x60
[<ffffffff8145220a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8145b311>] scsi_request_fn+0x501/0x770
[<ffffffff812eb6e3>] __blk_run_queue+0x33/0x40
[<ffffffff812eb79a>] queue_unplugged+0x2a/0xa0
[<ffffffff812f05f8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812f086a>] blk_queue_bio+0x21a/0x3a0
[<ffffffff812eeb89>] generic_make_request+0x109/0x1e0
[<ffffffff812eecd1>] submit_bio+0x71/0x150
[<ffffffffa025ea43>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffa02605e8>] ? __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa0260300>] xfs_buf_submit+0x70/0x1f0 [xfs]
[<ffffffffa02605e8>] __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa026138f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa028edd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa026138f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa028f01d>] xfsaild+0x24d/0x660 [xfs]
[<ffffffffa028edd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff816970d8>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: 08 e8 2a 0c a0 ff 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 53 48 89 f3 66 66 66 66 90 66 83 07 02 48 89 df 57 9d <66> 66 90 66 90 5b 5d c3 0f 1f 44 00 00 8b 37 f0 66 83 07 02 f6 
Complete report #754629
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffa00663be
Build id: 3.10.0-327.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffa00d6729
Build id: 3.10.0-327.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffa005f2b0
Build id: 3.10.0-327.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff81417b1a
Build id: 3.10.0-327.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff81420aa1
Build id: 3.10.0-327.el7.x86_64
0x501
6
__blk_run_queue
vmlinux 0xffffffff812c73e3
Build id: 3.10.0-327.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812c749a
Build id: 3.10.0-327.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812cbd18
Build id: 3.10.0-327.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff812cbf8a
Build id: 3.10.0-327.el7.x86_64
0x21a
10
generic_make_request
vmlinux 0xffffffff812c7212
Build id: 3.10.0-327.el7.x86_64
0xe2
11
submit_bio
vmlinux 0xffffffff812c72d1
Build id: 3.10.0-327.el7.x86_64
0x71
12
_xfs_buf_ioapply
xfs 0xffffffffa0235173
Build id: 3.10.0-327.el7.x86_64
0x2f3
13
? __xfs_buf_delwri_submit
xfs 0xffffffffa0236a28
Build id: 3.10.0-327.el7.x86_64
0x198
14
xfs_buf_submit
xfs 0xffffffffa0236734
Build id: 3.10.0-327.el7.x86_64
0x64
15
__xfs_buf_delwri_submit
xfs 0xffffffffa0236a28
Build id: 3.10.0-327.el7.x86_64
0x198
16
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa023767f
Build id: 3.10.0-327.el7.x86_64
0x2f
17
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0262470
Build id: 3.10.0-327.el7.x86_64
0x90
18
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa023767f
Build id: 3.10.0-327.el7.x86_64
0x2f
19
xfsaild
xfs 0xffffffffa02626b0
Build id: 3.10.0-327.el7.x86_64
0x240
20
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0262470
Build id: 3.10.0-327.el7.x86_64
0x90
21
kthread
vmlinux 0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64
0xcf
22
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
0x140
23
ret_from_fork
vmlinux 0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64
0x58
24
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64
0x140
BUG: soft lockup - CPU#2 stuck for 146s! [xfsaild/dm-0:427]
Modules linked in: nf_conntrack_ftp binfmt_misc udp_diag nf_conntrack_netlink tcp_diag inet_diag sysmon_edr(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun arc4 ip_vs md4 nls_utf8 cifs dns_resolver ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr sg vmw_vmci parport_pc parport i2c_piix4
shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul drm_kms_helper crct10dif_common crc32c_intel ttm ata_piix serio_raw mptspi drm scsi_transport_spi vmxnet3 mptscsih libata mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 427 Comm: xfsaild/dm-0 Tainted: G           OEL ------------   3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880424635080 ti: ffff880035918000 task.ti: ffff880035918000
RIP: 0010:[<ffffffffa00663be>]  [<ffffffffa00663be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff88003591b968  EFLAGS: 00000246
RAX: ffffc90009b40000 RBX: ffff8803e321daa0 RCX: 0000000000000009
RDX: ffff880423844400 RSI: ffff880424004000 RDI: 000000000000000e
RBP: ffff88003591b978 R08: 0000000000000002 R09: ffff88042420b0d8
R10: ffff8800a37cac40 R11: ffff8800a37cac40 R12: ffff8803a3855e00
R13: ffff88003591b8dc R14: ffffffff812d06bb R15: ffff88003591b920
FS:  0000000000000000(0000) GS:ffff88043fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f2ae9fd8000 CR3: 00000000b8aa3000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880424004d30 000000000000003c ffff88003591ba20 ffffffffa00d6729
ffff880424004008 040000003591b9e0 0000000000000009 ffff8800a37cac40
0000000000000024 ffff880424004188 0000006000000009 ffff880423bc2b90
Call Trace:
[<ffffffffa00d6729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa005f2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff81417b1a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff81420aa1>] scsi_request_fn+0x501/0x770
[<ffffffff812c73e3>] __blk_run_queue+0x33/0x40
[<ffffffff812c749a>] queue_unplugged+0x2a/0xa0
[<ffffffff812cbd18>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812cbf8a>] blk_queue_bio+0x21a/0x3a0
[<ffffffff812c7212>] generic_make_request+0xe2/0x130
[<ffffffff812c72d1>] submit_bio+0x71/0x150
[<ffffffffa0235173>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffa0236a28>] ? __xfs_buf_delwri_submit+0x198/0x250 [xfs]
[<ffffffffa0236734>] xfs_buf_submit+0x64/0x1c0 [xfs]
[<ffffffffa0236a28>] __xfs_buf_delwri_submit+0x198/0x250 [xfs]
[<ffffffffa023767f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0262470>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa023767f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa02626b0>] xfsaild+0x240/0x5e0 [xfs]
[<ffffffffa0262470>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 43 07 
Complete report #195109
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffa00b43be
Build id: 3.10.0-327.36.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffa00d6729
Build id: 3.10.0-327.36.1.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffa00ad2b0
Build id: 3.10.0-327.36.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff81417f4a
Build id: 3.10.0-327.36.1.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff81420ed1
Build id: 3.10.0-327.36.1.el7.x86_64
0x501
6
__blk_run_queue
vmlinux 0xffffffff812c75b3
Build id: 3.10.0-327.36.1.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812c766a
Build id: 3.10.0-327.36.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812cbee8
Build id: 3.10.0-327.36.1.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff812cc15a
Build id: 3.10.0-327.36.1.el7.x86_64
0x21a
10
generic_make_request
vmlinux 0xffffffff812c73e2
Build id: 3.10.0-327.36.1.el7.x86_64
0xe2
11
submit_bio
vmlinux 0xffffffff812c74a1
Build id: 3.10.0-327.36.1.el7.x86_64
0x71
12
_xfs_buf_ioapply
xfs 0xffffffffa0254183
Build id: 3.10.0-327.36.1.el7.x86_64
0x2f3
13
? __xfs_buf_delwri_submit
xfs 0xffffffffa0255a38
Build id: 3.10.0-327.36.1.el7.x86_64
0x198
14
xfs_buf_submit
xfs 0xffffffffa0255744
Build id: 3.10.0-327.36.1.el7.x86_64
0x64
15
__xfs_buf_delwri_submit
xfs 0xffffffffa0255a38
Build id: 3.10.0-327.36.1.el7.x86_64
0x198
16
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa025668f
Build id: 3.10.0-327.36.1.el7.x86_64
0x2f
17
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0281520
Build id: 3.10.0-327.36.1.el7.x86_64
0x90
18
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa025668f
Build id: 3.10.0-327.36.1.el7.x86_64
0x2f
19
xfsaild
xfs 0xffffffffa0281760
Build id: 3.10.0-327.36.1.el7.x86_64
0x240
20
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0281520
Build id: 3.10.0-327.36.1.el7.x86_64
0x90
21
kthread
vmlinux 0xffffffff810a5b8f
Build id: 3.10.0-327.36.1.el7.x86_64
0xcf
22
? kthread_create_on_node
vmlinux 0xffffffff810a5ac0
Build id: 3.10.0-327.36.1.el7.x86_64
0x140
23
ret_from_fork
vmlinux 0xffffffff81646958
Build id: 3.10.0-327.36.1.el7.x86_64
0x58
24
? kthread_create_on_node
vmlinux 0xffffffff810a5ac0
Build id: 3.10.0-327.36.1.el7.x86_64
0x140
BUG: soft lockup - CPU#0 stuck for 40s! [xfsaild/dm-1:397]
Modules linked in: binfmt_misc fuse snapapi26(POE) vmw_vsock_vmci_transport vsock iptable_nat nf_nat_ipv4 nf_nat ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr i2c_piix4 vmw_balloon sg shpchp parport_pc vmw_vmci parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper ttm ata_piix drm e1000 mptspi scsi_transport_spi mptscsih mptbase i2c_core libata floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: edd]
CPU: 0 PID: 397 Comm: xfsaild/dm-1 Tainted: P           OEL ------------   3.10.0-327.36.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013
task: ffff880035372280 ti: ffff880034834000 task.ti: ffff880034834000
RIP: 0010:[<ffffffffa00b43be>]  [<ffffffffa00b43be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880034837968  EFLAGS: 00000246
RAX: ffffc900082c0000 RBX: ffff88003b93adb0 RCX: 000000000000000e
RDX: ffff880036002400 RSI: ffff8800392dc000 RDI: 000000000000000e
RBP: ffff880034837978 R08: 0000000000000002 R09: 000000003b799fff
R10: ffff88003b623a80 R11: ffff88003b623a80 R12: ffff88001dd9d300
R13: ffff8800348378dc R14: ffffffff812d08fb R15: ffff880034837920
FS:  0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f27b80da000 CR3: 000000003c619000 CR4: 00000000000407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800392dcd30 0000000000000038 ffff880034837a20 ffffffffa00d6729
ffff8800392dc008 04000000348379e0 000000000000000e ffff88003b623a80
0000000000000038 ffff8800392dc188 000000600000000e ffff880035902d70
Call Trace:
[<ffffffffa00d6729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa00ad2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff81417f4a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff81420ed1>] scsi_request_fn+0x501/0x770
[<ffffffff812c75b3>] __blk_run_queue+0x33/0x40
[<ffffffff812c766a>] queue_unplugged+0x2a/0xa0
[<ffffffff812cbee8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812cc15a>] blk_queue_bio+0x21a/0x3a0
[<ffffffff812c73e2>] generic_make_request+0xe2/0x130
[<ffffffff812c74a1>] submit_bio+0x71/0x150
[<ffffffffa0254183>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffa0255a38>] ? __xfs_buf_delwri_submit+0x198/0x250 [xfs]
[<ffffffffa0255744>] xfs_buf_submit+0x64/0x1c0 [xfs]
[<ffffffffa0255a38>] __xfs_buf_delwri_submit+0x198/0x250 [xfs]
[<ffffffffa025668f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0281520>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa025668f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0281760>] xfsaild+0x240/0x5e0 [xfs]
[<ffffffffa0281520>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810a5b8f>] kthread+0xcf/0xe0
[<ffffffff810a5ac0>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81646958>] ret_from_fork+0x58/0x90
[<ffffffff810a5ac0>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 0c 
Complete report #122906
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffa00bd3be
Build id: 3.10.0-514.21.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffa00a4729
Build id: 3.10.0-514.21.2.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffa01042b0
Build id: 3.10.0-514.21.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8145257a
Build id: 3.10.0-514.21.2.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8145b681
Build id: 3.10.0-514.21.2.el7.x86_64
0x501
6
__blk_run_queue
vmlinux 0xffffffff812eb943
Build id: 3.10.0-514.21.2.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812eb9fa
Build id: 3.10.0-514.21.2.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812f0858
Build id: 3.10.0-514.21.2.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff812f0aca
Build id: 3.10.0-514.21.2.el7.x86_64
0x21a
10
generic_make_request
vmlinux 0xffffffff812eede9
Build id: 3.10.0-514.21.2.el7.x86_64
0x109
11
submit_bio
vmlinux 0xffffffff812eef31
Build id: 3.10.0-514.21.2.el7.x86_64
0x71
12
_xfs_buf_ioapply
xfs 0xffffffffa02c5a43
Build id: 3.10.0-514.21.2.el7.x86_64
0x2f3
13
? __xfs_buf_delwri_submit
xfs 0xffffffffa02c75e8
Build id: 3.10.0-514.21.2.el7.x86_64
0x168
14
xfs_buf_submit
xfs 0xffffffffa02c7300
Build id: 3.10.0-514.21.2.el7.x86_64
0x70
15
__xfs_buf_delwri_submit
xfs 0xffffffffa02c75e8
Build id: 3.10.0-514.21.2.el7.x86_64
0x168
16
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa02c838f
Build id: 3.10.0-514.21.2.el7.x86_64
0x2f
17
? xfs_trans_ail_cursor_first
xfs 0xffffffffa02f5dd0
Build id: 3.10.0-514.21.2.el7.x86_64
0x90
18
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa02c838f
Build id: 3.10.0-514.21.2.el7.x86_64
0x2f
19
xfsaild
xfs 0xffffffffa02f601d
Build id: 3.10.0-514.21.2.el7.x86_64
0x24d
20
? xfs_trans_ail_cursor_first
xfs 0xffffffffa02f5dd0
Build id: 3.10.0-514.21.2.el7.x86_64
0x90
21
kthread
vmlinux 0xffffffff810b0a4f
Build id: 3.10.0-514.21.2.el7.x86_64
0xcf
22
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.21.2.el7.x86_64
0x140
23
ret_from_fork
vmlinux 0xffffffff81697698
Build id: 3.10.0-514.21.2.el7.x86_64
0x58
24
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.21.2.el7.x86_64
0x140
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 323s! [xfsaild/dm-0:412]
Modules linked in: ipt_REJECT nf_reject_ipv4 ip_set nfnetlink tun nf_conntrack_ftp iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_nat iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_mac xt_conntrack nf_conntrack xt_multiport iptable_filter iptable_raw vmw_vsock_vmci_transport vsock intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon sg parport_pc i2c_piix4 parport shpchp vmw_vmci nfsd nfs_acl lockd grace auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi crct10dif_pclmul crct10dif_common vmwgfx drm_kms_helper syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops serio_raw ttm drm i2c_core mptspi
scsi_transport_spi mptscsih e1000 mptbase ata_piix libata floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 412 Comm: xfsaild/dm-0 Tainted: G        W    L ------------   3.10.0-514.21.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 06/22/2012
task: ffff8800358dce70 ti: ffff880035928000 task.ti: ffff880035928000
RIP: 0010:[<ffffffffa00bd3be>]  [<ffffffffa00bd3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88003592b970  EFLAGS: 00000246
RAX: ffffc90000840000 RBX: ffff88003592b940 RCX: 0000000000000007
RDX: ffff88013580bc00 RSI: ffff880036bd9000 RDI: 000000000000000e
RBP: ffff88003592b980 R08: 0000000000000002 R09: ffff880036bda0d8
R10: ffff8801030d4fc0 R11: ffff8801030d4fc0 R12: 0000000100000001
R13: ffff88004b439b00 R14: ffff88003592b8e4 R15: ffffffff812f536b
FS:  0000000000000000(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f78a640c040 CR3: 000000004e810000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880036bd9d30 000000000000003c ffff88003592ba28 ffffffffa00a4729
ffff880036bd9008 040000003592b9e8 0000000000000007 ffff8801030d4fc0
000000000000001c ffff880036bd9188 0000006000000007 ffff880036242ad0
Call Trace:
[<ffffffffa00a4729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa01042b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8145257a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8145b681>] scsi_request_fn+0x501/0x770
[<ffffffff812eb943>] __blk_run_queue+0x33/0x40
[<ffffffff812eb9fa>] queue_unplugged+0x2a/0xa0
[<ffffffff812f0858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812f0aca>] blk_queue_bio+0x21a/0x3a0
[<ffffffff812eede9>] generic_make_request+0x109/0x1e0
[<ffffffff812eef31>] submit_bio+0x71/0x150
[<ffffffffa02c5a43>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffa02c75e8>] ? __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa02c7300>] xfs_buf_submit+0x70/0x1f0 [xfs]
[<ffffffffa02c75e8>] __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa02c838f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa02f5dd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa02c838f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa02f601d>] xfsaild+0x24d/0x660 [xfs]
[<ffffffffa02f5dd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697698>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 b3 0c 
Complete report #23409
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff816b6545
Build id: 3.10.0-693.21.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc005966a
Build id: 3.10.0-693.21.1.el7.x86_64
0x44a
3
? scsi_test_unit_ready
vmlinux 0xffffffff81475750
Build id: 3.10.0-693.21.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146d8f0
Build id: 3.10.0-693.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8147678f
Build id: 3.10.0-693.21.1.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812fd1d9
Build id: 3.10.0-693.21.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff812fd29a
Build id: 3.10.0-693.21.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff81300778
Build id: 3.10.0-693.21.1.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff813009fa
Build id: 3.10.0-693.21.1.el7.x86_64
0x22a
10
generic_make_request
vmlinux 0xffffffff812feb7b
Build id: 3.10.0-693.21.1.el7.x86_64
0x10b
11
? __bio_add_page
vmlinux 0xffffffff8123fa4b
Build id: 3.10.0-693.21.1.el7.x86_64
0x20b
12
submit_bio
vmlinux 0xffffffff812fee00
Build id: 3.10.0-693.21.1.el7.x86_64
0x70
13
_xfs_buf_ioapply
xfs 0xffffffffc025de23
Build id: 3.10.0-693.21.1.el7.x86_64
0x2f3
14
? __xfs_buf_delwri_submit
xfs 0xffffffffc025f9d8
Build id: 3.10.0-693.21.1.el7.x86_64
0x168
15
xfs_buf_submit
xfs 0xffffffffc025f6bc
Build id: 3.10.0-693.21.1.el7.x86_64
0x6c
16
__xfs_buf_delwri_submit
xfs 0xffffffffc025f9d8
Build id: 3.10.0-693.21.1.el7.x86_64
0x168
17
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffc026075f
Build id: 3.10.0-693.21.1.el7.x86_64
0x2f
18
? xfs_trans_ail_cursor_first
xfs 0xffffffffc028e780
Build id: 3.10.0-693.21.1.el7.x86_64
0x90
19
xfs_buf_delwri_submit_nowait
xfs 0xffffffffc026075f
Build id: 3.10.0-693.21.1.el7.x86_64
0x2f
20
xfsaild
xfs 0xffffffffc028e9df
Build id: 3.10.0-693.21.1.el7.x86_64
0x25f
21
? xfs_trans_ail_cursor_first
xfs 0xffffffffc028e780
Build id: 3.10.0-693.21.1.el7.x86_64
0x90
22
kthread
vmlinux 0xffffffff810b4031
Build id: 3.10.0-693.21.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
24
ret_from_fork
vmlinux 0xffffffff816c0577
Build id: 3.10.0-693.21.1.el7.x86_64
0x77
25
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 30s! [xfsaild/dm-0:516]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev nfit sg libnvdimm shpchp parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel
ahci serio_raw libahci ata_piix libata vmxnet3 i2c_core vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 516 Comm: xfsaild/dm-0 Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff880230e11fa0 ti: ffff880231774000 task.ti: ffff880231774000
RIP: 0010:[<ffffffff816b6545>]  [<ffffffff816b6545>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8802317779c0  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff81350010 RCX: 0000000000001000
RDX: ffffc90000e90000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8802317779c0 R08: 0000000000000000 R09: ffff88022ef01b00
R10: ffff88021de7ba40 R11: ffff880035e29800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8802237e1b00 R15: ffffffff81474359
FS:  0000000000000000(0000) GS:ffff88023fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fc6b334d550 CR3: 000000022ee0c000 CR4: 00000000000607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc005966a>] pvscsi_queue+0x44a/0x7e9 [vmw_pvscsi]
[<ffffffff81475750>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8146d8f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8147678f>] scsi_request_fn+0x4df/0x680
[<ffffffff812fd1d9>] __blk_run_queue+0x39/0x50
[<ffffffff812fd29a>] queue_unplugged+0x2a/0xa0
[<ffffffff81300778>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff813009fa>] blk_queue_bio+0x22a/0x3a0
[<ffffffff812feb7b>] generic_make_request+0x10b/0x320
[<ffffffff8123fa4b>] ? __bio_add_page+0x20b/0x2b0
[<ffffffff812fee00>] submit_bio+0x70/0x150
[<ffffffffc025de23>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffc025f9d8>] ? __xfs_buf_delwri_submit+0x168/0x260 [xfs]
[<ffffffffc025f6bc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc025f9d8>] __xfs_buf_delwri_submit+0x168/0x260 [xfs]
[<ffffffffc026075f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc028e780>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffc026075f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffc028e9df>] xfsaild+0x25f/0x6f0 [xfs]
[<ffffffffc028e780>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b4031>] kthread+0xd1/0xe0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #23160
Frame # Function Binary Source or offset Line
1
mptscsih_qcmd
mptscsih 0xffffffffa004f729
Build id: 3.10.0-514.10.2.el7.x86_64
0x249
2
mptspi_qcmd
mptspi 0xffffffffa01b12b0
Build id: 3.10.0-514.10.2.el7.x86_64
0x50
3
scsi_dispatch_cmd
vmlinux 0xffffffff81451cba
Build id: 3.10.0-514.10.2.el7.x86_64
0xaa
4
scsi_request_fn
vmlinux 0xffffffff8145adc1
Build id: 3.10.0-514.10.2.el7.x86_64
0x501
5
__blk_run_queue
vmlinux 0xffffffff812eb253
Build id: 3.10.0-514.10.2.el7.x86_64
0x33
6
queue_unplugged
vmlinux 0xffffffff812eb30a
Build id: 3.10.0-514.10.2.el7.x86_64
0x2a
7
blk_flush_plug_list
vmlinux 0xffffffff812f0168
Build id: 3.10.0-514.10.2.el7.x86_64
0x1d8
8
blk_queue_bio
vmlinux 0xffffffff812f03da
Build id: 3.10.0-514.10.2.el7.x86_64
0x21a
9
generic_make_request
vmlinux 0xffffffff812ee6f9
Build id: 3.10.0-514.10.2.el7.x86_64
0x109
10
submit_bio
vmlinux 0xffffffff812ee841
Build id: 3.10.0-514.10.2.el7.x86_64
0x71
11
_xfs_buf_ioapply
xfs 0xffffffffa02dbfd3
Build id: 3.10.0-514.10.2.el7.x86_64
0x2f3
12
? __xfs_buf_delwri_submit
xfs 0xffffffffa02ddb78
Build id: 3.10.0-514.10.2.el7.x86_64
0x168
13
xfs_buf_submit
xfs 0xffffffffa02dd890
Build id: 3.10.0-514.10.2.el7.x86_64
0x70
14
__xfs_buf_delwri_submit
xfs 0xffffffffa02ddb78
Build id: 3.10.0-514.10.2.el7.x86_64
0x168
15
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa02de91f
Build id: 3.10.0-514.10.2.el7.x86_64
0x2f
16
? xfs_trans_ail_cursor_first
xfs 0xffffffffa030c360
Build id: 3.10.0-514.10.2.el7.x86_64
0x90
17
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa02de91f
Build id: 3.10.0-514.10.2.el7.x86_64
0x2f
18
xfsaild
xfs 0xffffffffa030c5ad
Build id: 3.10.0-514.10.2.el7.x86_64
0x24d
19
? xfs_trans_ail_cursor_first
xfs 0xffffffffa030c360
Build id: 3.10.0-514.10.2.el7.x86_64
0x90
20
kthread
vmlinux 0xffffffff810b06ff
Build id: 3.10.0-514.10.2.el7.x86_64
0xcf
21
? kthread_create_on_node
vmlinux 0xffffffff810b0630
Build id: 3.10.0-514.10.2.el7.x86_64
0x140
22
ret_from_fork
vmlinux 0xffffffff81696a58
Build id: 3.10.0-514.10.2.el7.x86_64
0x58
23
? kthread_create_on_node
vmlinux 0xffffffff810b0630
Build id: 3.10.0-514.10.2.el7.x86_64
0x140
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 26s! [java:30573]
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880f35674d30 0000000000000054 ffff880cb5643a28 ffffffffa004f729
ffff880f35674008 04000000b56439e8 000000000000003d ffff880c18f279c0
00000000000000f4 ffff880f35674188 000000600000003d ffff880a00000007
Call Trace:
[<ffffffffa004f729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa01b12b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff81451cba>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8145adc1>] scsi_request_fn+0x501/0x770
[<ffffffff812eb253>] __blk_run_queue+0x33/0x40
[<ffffffff812eb30a>] queue_unplugged+0x2a/0xa0
[<ffffffff812f0168>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812f03da>] blk_queue_bio+0x21a/0x3a0
[<ffffffff812ee6f9>] generic_make_request+0x109/0x1e0
[<ffffffff812ee841>] submit_bio+0x71/0x150
[<ffffffffa02dbfd3>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffa02ddb78>] ? __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa02dd890>] xfs_buf_submit+0x70/0x1f0 [xfs]
[<ffffffffa02ddb78>] __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa02de91f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa030c360>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa02de91f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa030c5ad>] xfsaild+0x24d/0x660 [xfs]
[<ffffffffa030c360>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b06ff>] kthread+0xcf/0xe0
[<ffffffff810b0630>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696a58>] ret_from_fork+0x58/0x90
[<ffffffff810b0630>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 08 
Complete report #11041
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffa002b3be
Build id: 3.10.0-327.36.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffa00b4729
Build id: 3.10.0-327.36.2.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffa01402b0
Build id: 3.10.0-327.36.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff81417f4a
Build id: 3.10.0-327.36.2.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff81420ed1
Build id: 3.10.0-327.36.2.el7.x86_64
0x501
6
__blk_run_queue
vmlinux 0xffffffff812c75b3
Build id: 3.10.0-327.36.2.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812c766a
Build id: 3.10.0-327.36.2.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812cbee8
Build id: 3.10.0-327.36.2.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff812cc15a
Build id: 3.10.0-327.36.2.el7.x86_64
0x21a
10
generic_make_request
vmlinux 0xffffffff812c73e2
Build id: 3.10.0-327.36.2.el7.x86_64
0xe2
11
submit_bio
vmlinux 0xffffffff812c74a1
Build id: 3.10.0-327.36.2.el7.x86_64
0x71
12
? bio_alloc_bioset
vmlinux 0xffffffff81217f6d
Build id: 3.10.0-327.36.2.el7.x86_64
0x1fd
13
_xfs_buf_ioapply
xfs 0xffffffffa0228183
Build id: 3.10.0-327.36.2.el7.x86_64
0x2f3
14
? __xfs_buf_delwri_submit
xfs 0xffffffffa0229a38
Build id: 3.10.0-327.36.2.el7.x86_64
0x198
15
xfs_buf_submit
xfs 0xffffffffa0229744
Build id: 3.10.0-327.36.2.el7.x86_64
0x64
16
__xfs_buf_delwri_submit
xfs 0xffffffffa0229a38
Build id: 3.10.0-327.36.2.el7.x86_64
0x198
17
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa022a68f
Build id: 3.10.0-327.36.2.el7.x86_64
0x2f
18
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0255520
Build id: 3.10.0-327.36.2.el7.x86_64
0x90
19
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa022a68f
Build id: 3.10.0-327.36.2.el7.x86_64
0x2f
20
xfsaild
xfs 0xffffffffa0255760
Build id: 3.10.0-327.36.2.el7.x86_64
0x240
21
? xfs_trans_ail_cursor_first
xfs 0xffffffffa0255520
Build id: 3.10.0-327.36.2.el7.x86_64
0x90
22
kthread
vmlinux 0xffffffff810a5b8f
Build id: 3.10.0-327.36.2.el7.x86_64
0xcf
23
? finish_task_switch
vmlinux 0xffffffff810b4883
Build id: 3.10.0-327.36.2.el7.x86_64
0x53
24
? kthread_create_on_node
vmlinux 0xffffffff810a5ac0
Build id: 3.10.0-327.36.2.el7.x86_64
0x140
25
ret_from_fork
vmlinux 0xffffffff81646a58
Build id: 3.10.0-327.36.2.el7.x86_64
0x58
26
? kthread_create_on_node
vmlinux 0xffffffff810a5ac0
Build id: 3.10.0-327.36.2.el7.x86_64
0x140
BUG: soft lockup - CPU#0 stuck for 23s! [xfsaild/sda3:466]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT ip6t_REJECT xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables tun bridge stp llc vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod coretemp snd_hda_codec_generic crc32_pclmul ghash_clmulni_intel snd_hda_intel aesni_intel snd_hda_codec lrw gf128mul snd_hda_core glue_helper ablk_helper snd_hwdep cryptd snd_seq ppdev snd_seq_device vmw_balloon snd_pcm pcspkr snd_timer snd
sg soundcore vmw_vmci shpchp i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common ttm crc32c_intel ata_piix libata serio_raw mptspi drm scsi_transport_spi mptscsih e1000 mptbase i2c_core floppy [last unloaded: ip_tables]
CPU: 0 PID: 466 Comm: xfsaild/sda3 Tainted: G             L ------------   3.10.0-327.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff880035340000 ti: ffff880035e28000 task.ti: ffff880035e28000
RIP: 0010:[<ffffffffa002b3be>]  [<ffffffffa002b3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880035e2b968  EFLAGS: 00000246
RAX: ffffc90008940000 RBX: ffff8800b42b6398 RCX: 0000000000000020
RDX: ffff88003595d400 RSI: ffff880036187000 RDI: 000000000000000e
RBP: ffff880035e2b978 R08: 0000000000000002 R09: 000000007697dfff
R10: ffff880034c26bc0 R11: ffff880034c26bc0 R12: ffff880077f2c480
R13: ffff880035e2b8dc R14: ffffffff812d08fb R15: ffff880035e2b920
FS:  0000000000000000(0000) GS:ffff8800bb400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f9de7c8b000 CR3: 00000000b6960000 CR4: 00000000000407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880036187d30 0000000000000040 ffff880035e2ba20 ffffffffa00b4729
ffff880036187008 0400000035e2b9e0 0000000000000020 ffff880034c26bc0
0000000000000080 ffff880036187188 0000006000000020 ffff880034c83430
Call Trace:
[<ffffffffa00b4729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa01402b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff81417f4a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff81420ed1>] scsi_request_fn+0x501/0x770
[<ffffffff812c75b3>] __blk_run_queue+0x33/0x40
[<ffffffff812c766a>] queue_unplugged+0x2a/0xa0
[<ffffffff812cbee8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812cc15a>] blk_queue_bio+0x21a/0x3a0
[<ffffffff812c73e2>] generic_make_request+0xe2/0x130
[<ffffffff812c74a1>] submit_bio+0x71/0x150
[<ffffffff81217f6d>] ? bio_alloc_bioset+0x1fd/0x350
[<ffffffffa0228183>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffa0229a38>] ? __xfs_buf_delwri_submit+0x198/0x250 [xfs]
[<ffffffffa0229744>] xfs_buf_submit+0x64/0x1c0 [xfs]
[<ffffffffa0229a38>] __xfs_buf_delwri_submit+0x198/0x250 [xfs]
[<ffffffffa022a68f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0255520>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa022a68f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa0255760>] xfsaild+0x240/0x5e0 [xfs]
[<ffffffffa0255520>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810a5b8f>] kthread+0xcf/0xe0
[<ffffffff810b4883>] ? finish_task_switch+0x53/0x170
[<ffffffff810a5ac0>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81646a58>] ret_from_fork+0x58/0x90
[<ffffffff810a5ac0>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 93 03 
Complete report #6482
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffa012d3be
Build id: 3.10.0-514.26.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffa010a729
Build id: 3.10.0-514.26.2.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffa011c2b0
Build id: 3.10.0-514.26.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8145269a
Build id: 3.10.0-514.26.2.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8145b7a1
Build id: 3.10.0-514.26.2.el7.x86_64
0x501
6
__blk_run_queue
vmlinux 0xffffffff812eb9c3
Build id: 3.10.0-514.26.2.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812eba7a
Build id: 3.10.0-514.26.2.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812f08d8
Build id: 3.10.0-514.26.2.el7.x86_64
0x1d8
9
blk_queue_bio
vmlinux 0xffffffff812f0b4a
Build id: 3.10.0-514.26.2.el7.x86_64
0x21a
10
generic_make_request
vmlinux 0xffffffff812eee69
Build id: 3.10.0-514.26.2.el7.x86_64
0x109
11
submit_bio
vmlinux 0xffffffff812eefb1
Build id: 3.10.0-514.26.2.el7.x86_64
0x71
12
_xfs_buf_ioapply
xfs 0xffffffffa0284a43
Build id: 3.10.0-514.26.2.el7.x86_64
0x2f3
13
? __xfs_buf_delwri_submit
xfs 0xffffffffa02865e8
Build id: 3.10.0-514.26.2.el7.x86_64
0x168
14
xfs_buf_submit
xfs 0xffffffffa0286300
Build id: 3.10.0-514.26.2.el7.x86_64
0x70
15
__xfs_buf_delwri_submit
xfs 0xffffffffa02865e8
Build id: 3.10.0-514.26.2.el7.x86_64
0x168
16
? xfs_buf_delwri_submit_nowait
xfs 0xffffffffa028738f
Build id: 3.10.0-514.26.2.el7.x86_64
0x2f
17
? xfs_trans_ail_cursor_first
xfs 0xffffffffa02b4dd0
Build id: 3.10.0-514.26.2.el7.x86_64
0x90
18
xfs_buf_delwri_submit_nowait
xfs 0xffffffffa028738f
Build id: 3.10.0-514.26.2.el7.x86_64
0x2f
19
xfsaild
xfs 0xffffffffa02b501d
Build id: 3.10.0-514.26.2.el7.x86_64
0x24d
20
? xfs_trans_ail_cursor_first
xfs 0xffffffffa02b4dd0
Build id: 3.10.0-514.26.2.el7.x86_64
0x90
21
kthread
vmlinux 0xffffffff810b0a4f
Build id: 3.10.0-514.26.2.el7.x86_64
0xcf
22
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.26.2.el7.x86_64
0x140
23
ret_from_fork
vmlinux 0xffffffff81697758
Build id: 3.10.0-514.26.2.el7.x86_64
0x58
24
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.26.2.el7.x86_64
0x140
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [xfsaild/dm-0:427]
Modules linked in: nfnetlink_queue nfnetlink_log bluetooth rfkill ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg pcspkr vmw_balloon parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul
crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect mptspi sysimgblt fb_sys_fops scsi_transport_spi ttm mptscsih vmxnet3 mptbase drm ata_piix libata i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 427 Comm: xfsaild/dm-0 Tainted: G             L ------------   3.10.0-514.26.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88017ce8edd0 ti: ffff880035b70000 task.ti: ffff880035b70000
RIP: 0010:[<ffffffffa012d3be>]  [<ffffffffa012d3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880035b73970  EFLAGS: 00000246
RAX: ffffc90000e40000 RBX: ffff880035b73940 RCX: 0000000000000013
RDX: ffff880233718800 RSI: ffff88003656a000 RDI: 000000000000000e
RBP: ffff880035b73980 R08: 0000000000000002 R09: ffff88023fa140d8
R10: ffff8801f7348540 R11: ffff8801f7348540 R12: 0000000100000001
R13: ffff8800b4350780 R14: ffff880035b738e4 R15: ffffffff812f53eb
FS:  0000000000000000(0000) GS:ffff88023fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f94aca07640 CR3: 00000000b8f23000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88003656ad30 000000000000003c ffff880035b73a28 ffffffffa010a729
ffff88003656a008 0400000035b739e8 0000000000000013 ffff8801f7348540
000000000000004c ffff88003656a188 0000006000000013 ffff880232cc2f50
Call Trace:
[<ffffffffa010a729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa011c2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8145269a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8145b7a1>] scsi_request_fn+0x501/0x770
[<ffffffff812eb9c3>] __blk_run_queue+0x33/0x40
[<ffffffff812eba7a>] queue_unplugged+0x2a/0xa0
[<ffffffff812f08d8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812f0b4a>] blk_queue_bio+0x21a/0x3a0
[<ffffffff812eee69>] generic_make_request+0x109/0x1e0
[<ffffffff812eefb1>] submit_bio+0x71/0x150
[<ffffffffa0284a43>] _xfs_buf_ioapply+0x2f3/0x460 [xfs]
[<ffffffffa02865e8>] ? __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa0286300>] xfs_buf_submit+0x70/0x1f0 [xfs]
[<ffffffffa02865e8>] __xfs_buf_delwri_submit+0x168/0x250 [xfs]
[<ffffffffa028738f>] ? xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa02b4dd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffa028738f>] xfs_buf_delwri_submit_nowait+0x2f/0x50 [xfs]
[<ffffffffa02b501d>] xfsaild+0x24d/0x660 [xfs]
[<ffffffffa02b4dd0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697758>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 b3 13