Info

Permalink
Function
mpt_put_msg_frame
First occurrence
2020-09-24
Last occurrence
2026-04-10
State
NEW
Components
kernel
Quality
-41

Statistics

Operating system Count
CentOS 7 131
Architecture Count
x86_64 131
Related packages Count
kernel 131
 0:3.10.0-957.el7 40
 0:3.10.0-862.el7 33
 0:3.10.0-1062.el7 21
 0:3.10.0-957.27.2.el7 8
 0:3.10.0-957.10.1.el7 4
 0:3.10.0-862.9.1.el7 4
 0:3.10.0-1062.4.3.el7 4
 0:3.10.0-957.12.2.el7 2
 0:3.10.0-957.1.3.el7 2
 0:3.10.0-957.5.1.el7 2
 0:3.10.0-862.2.3.el7 2
 0:3.10.0-1062.9.1.el7 2
 0:3.10.0-862.14.4.el7 2
 0:3.10.0-862.3.2.el7 1
 0:3.10.0-693.11.1.el7 1
 0:3.10.0-1062.4.1.el7 1
 0:3.10.0-957.21.3.el7 1
 0:3.10.0-1160.24.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #174696
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04683be
Build id: 3.10.0-1160.24.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc048371e
Build id: 3.10.0-1160.24.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04f12e0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff860e3fa0
Build id: 3.10.0-1160.24.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff860ed6cc
Build id: 3.10.0-1160.24.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff85f53939
Build id: 3.10.0-1160.24.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff85f539fa
Build id: 3.10.0-1160.24.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff85f57338
Build id: 3.10.0-1160.24.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff85f577a4
Build id: 3.10.0-1160.24.1.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff85dcb481
Build id: 3.10.0-1160.24.1.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff85dcbaa1
Build id: 3.10.0-1160.24.1.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff85dc0c15
Build id: 3.10.0-1160.24.1.el7.x86_64
0x105
13
? release_sock
vmlinux 0xffffffff8623ba18
Build id: 3.10.0-1160.24.1.el7.x86_64
0x128
14
__xfs_filemap_fault
xfs 0xffffffffc06dd91e
Build id: 3.10.0-1160.24.1.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc06ddb1c
Build id: 3.10.0-1160.24.1.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffff85dee08a
Build id: 3.10.0-1160.24.1.el7.x86_64
0x8a
17
? mntput
vmlinux 0xffffffff85e71e64
Build id: 3.10.0-1160.24.1.el7.x86_64
0x24
18
do_read_fault.isra.63
vmlinux 0xffffffff85dee63c
Build id: 3.10.0-1160.24.1.el7.x86_64
0x4c
19
handle_mm_fault
vmlinux 0xffffffff85df5e80
Build id: 3.10.0-1160.24.1.el7.x86_64
0xa20
20
? sock_aio_read
vmlinux 0xffffffff86235731
Build id: 3.10.0-1160.24.1.el7.x86_64
0x21
21
? do_sync_read
vmlinux 0xffffffff85e4d233
Build id: 3.10.0-1160.24.1.el7.x86_64
0x93
22
__do_page_fault
vmlinux 0xffffffff86390653
Build id: 3.10.0-1160.24.1.el7.x86_64
0x213
23
do_page_fault
vmlinux 0xffffffff86390975
Build id: 3.10.0-1160.24.1.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffff8638c778
Build id: 3.10.0-1160.24.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [node_exporter:2204]
Modules linked in: xt_nat veth xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c
sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit libnvdimm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel mptspi libata scsi_transport_spi serio_raw mptscsih mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 2204 Comm: node_exporter Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8d20f77f2100 ti: ffff8d2066658000 task.ti: ffff8d2066658000
RIP: 0010:[<ffffffffc04683be>]  [<ffffffffc04683be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8d206665b918  EFLAGS: 00000246
RAX: ffffa19dc0800000 RBX: ffff8d20189f9b20 RCX: 0000000000000004
RDX: ffff8d1ff5ebd200 RSI: ffff8d20eece8000 RDI: 000000000000000e
RBP: ffff8d206665b928 R08: 0000000000000003 R09: 000000011c731fff
R10: ffff8d1ff5edcc00 R11: ffff8d20f0f1e380 R12: ffff8d2042e72600
R13: ffff8d206665b88c R14: ffffffff85f5c35b R15: ffff8d206665b8d0
FS:  000000c0003b6410(0000) GS:ffff8d20ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000008f3061 CR3: 00000000b3810000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc048371e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04f12e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff860e3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff860ed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffff85f53939>] __blk_run_queue+0x39/0x50
[<ffffffff85f539fa>] queue_unplugged+0x2a/0xa0
[<ffffffff85f57338>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff85f577a4>] blk_finish_plug+0x14/0x40
[<ffffffff85dcb481>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff85dcbaa1>] ra_submit+0x21/0x30
[<ffffffff85dc0c15>] filemap_fault+0x105/0x420
[<ffffffff8623ba18>] ? release_sock+0x128/0x180
[<ffffffffc06dd91e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc06ddb1c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff85dee08a>] __do_fault.isra.61+0x8a/0x100
[<ffffffff85e71e64>] ? mntput+0x24/0x40
[<ffffffff85dee63c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffff85df5e80>] handle_mm_fault+0xa20/0xfb0
[<ffffffff86235731>] ? sock_aio_read+0x21/0x30
[<ffffffff85e4d233>] ? do_sync_read+0x93/0xe0
[<ffffffff86390653>] __do_page_fault+0x213/0x500
[<ffffffff86390975>] do_page_fault+0x35/0x90
[<ffffffff8638c778>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 47 
Complete report #5829
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00903be
Build id: 3.10.0-693.11.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc00ab729
Build id: 3.10.0-693.11.1.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffc00452b0
Build id: 3.10.0-693.11.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146535a
Build id: 3.10.0-693.11.1.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e11f
Build id: 3.10.0-693.11.1.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f7113
Build id: 3.10.0-693.11.1.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812f71ca
Build id: 3.10.0-693.11.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fa658
Build id: 3.10.0-693.11.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812faa64
Build id: 3.10.0-693.11.1.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff8118f41e
Build id: 3.10.0-693.11.1.el7.x86_64
0x1de
11
ra_submit
vmlinux 0xffffffff8118faa1
Build id: 3.10.0-693.11.1.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff81184e55
Build id: 3.10.0-693.11.1.el7.x86_64
0x105
13
? avc_has_perm_noaudit
vmlinux 0xffffffff812b4503
Build id: 3.10.0-693.11.1.el7.x86_64
0xc3
14
xfs_filemap_fault
xfs 0xffffffffc029b92f
Build id: 3.10.0-693.11.1.el7.x86_64
0x5f
15
__do_fault
vmlinux 0xffffffff811ad0d2
Build id: 3.10.0-693.11.1.el7.x86_64
0x52
16
do_read_fault.isra.44
vmlinux 0xffffffff811ad57b
Build id: 3.10.0-693.11.1.el7.x86_64
0x4b
17
handle_mm_fault
vmlinux 0xffffffff811b1e81
Build id: 3.10.0-693.11.1.el7.x86_64
0x691
18
__do_page_fault
vmlinux 0xffffffff816b0074
Build id: 3.10.0-693.11.1.el7.x86_64
0x154
19
do_page_fault
vmlinux 0xffffffff816b03a5
Build id: 3.10.0-693.11.1.el7.x86_64
0x35
20
page_fault
vmlinux 0xffffffff816ac5c8
Build id: 3.10.0-693.11.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 192s! [chronyd:2124]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 rpcsec_gss_krb5 nfsv4 nfs fscache arc4 md4 nls_utf8 cifs dns_resolver vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg i2c_piix4 parport_pc vmw_vmci parport shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul
crct10dif_common crc32c_intel mptspi ata_piix scsi_transport_spi libata mptscsih serio_raw mptbase e1000 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 2124 Comm: chronyd Tainted: G             L ------------   3.10.0-693.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/09/2012
task: ffff880139364f10 ti: ffff880035f30000 task.ti: ffff880035f30000
RIP: 0010:[<ffffffffc00903be>]  [<ffffffffc00903be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880035f339c8  EFLAGS: 00000246
RAX: ffffc90000800000 RBX: ffff8800a7faec00 RCX: 0000000000000014
RDX: ffff880035ed2200 RSI: ffff880134a9b000 RDI: 000000000000000e
RBP: ffff880035f339d8 R08: 0000000000000002 R09: ffff880134b3e0d8
R10: ffff8800aa30a8c0 R11: ffff8800aa30a8c0 R12: ffff880006722a80
R13: ffff880035f3393c R14: ffffffff812ff07b R15: ffff880035f33980
FS:  00007fe47f6de740(0000) GS:ffff88013fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559b3540aff0 CR3: 00000000b40ce000 CR4: 00000000000007f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880134a9bd30 000000000000003c ffff880035f33a80 ffffffffc00ab729
ffff880134a9b008 0000000035f33a40 0000000000000014 ffff8800aa30a8c0
0000000000000050 ffff880134a9b188 0000006000000014 ffff8800356c2fb0
Call Trace:
[<ffffffffc00ab729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00452b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146535a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e11f>] scsi_request_fn+0x4df/0x680
[<ffffffff812f7113>] __blk_run_queue+0x33/0x40
[<ffffffff812f71ca>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa658>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812faa64>] blk_finish_plug+0x14/0x40
[<ffffffff8118f41e>] __do_page_cache_readahead+0x1de/0x250
[<ffffffff8118faa1>] ra_submit+0x21/0x30
[<ffffffff81184e55>] filemap_fault+0x105/0x410
[<ffffffff812b4503>] ? avc_has_perm_noaudit+0xc3/0x110
[<ffffffffc029b92f>] xfs_filemap_fault+0x5f/0xf0 [xfs]
[<ffffffff811ad0d2>] __do_fault+0x52/0xe0
[<ffffffff811ad57b>] do_read_fault.isra.44+0x4b/0x130
[<ffffffff811b1e81>] handle_mm_fault+0x691/0xfa0
[<ffffffff816b0074>] __do_page_fault+0x154/0x450
[<ffffffff816b03a5>] do_page_fault+0x35/0x90
[<ffffffff816ac5c8>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 09 
Complete report #683464
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01d83be
Build id: 3.10.0-1062.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc023171e
Build id: 3.10.0-1062.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc022a2e0
Build id: 3.10.0-1062.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffadcdc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffadce5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffadb4cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffadb4d01a
Build id: 3.10.0-1062.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffadb50858
Build id: 3.10.0-1062.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffadb50cc4
Build id: 3.10.0-1062.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffad9c91b1
Build id: 3.10.0-1062.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffad9c9801
Build id: 3.10.0-1062.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffad9be6a5
Build id: 3.10.0-1062.el7.x86_64
0x105
13
? task_rq_unlock
vmlinux 0xffffffffad8d20a0
Build id: 3.10.0-1062.el7.x86_64
0x20
14
? down_read
vmlinux 0xffffffffadf7e342
Build id: 3.10.0-1062.el7.x86_64
0x12
15
__xfs_filemap_fault
xfs 0xffffffffc047555e
Build id: 3.10.0-1062.el7.x86_64
0x7e
16
xfs_filemap_fault
xfs 0xffffffffc047575c
Build id: 3.10.0-1062.el7.x86_64
0x2c
17
__do_fault.isra.61
vmlinux 0xffffffffad9ea29a
Build id: 3.10.0-1062.el7.x86_64
0x8a
18
do_read_fault.isra.63
vmlinux 0xffffffffad9ea84c
Build id: 3.10.0-1062.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffffad9ef2fa
Build id: 3.10.0-1062.el7.x86_64
0x22a
20
handle_mm_fault
vmlinux 0xffffffffad9f200d
Build id: 3.10.0-1062.el7.x86_64
0x39d
21
__do_page_fault
vmlinux 0xffffffffadf86633
Build id: 3.10.0-1062.el7.x86_64
0x213
22
do_page_fault
vmlinux 0xffffffffadf86955
Build id: 3.10.0-1062.el7.x86_64
0x35
23
page_fault
vmlinux 0xffffffffadf82768
Build id: 3.10.0-1062.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [rtkit-daemon:890]
Modules linked in: binfmt_misc tcp_lp nls_utf8 isofs fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun bridge stp llc devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev snd_ens1371 vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_timer snd lrw gf128mul glue_helper
sg ablk_helper soundcore pcspkr cryptd joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx ata_piix drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata mptspi drm e1000 nfit scsi_transport_spi mptscsih libnvdimm mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 890 Comm: rtkit-daemon Kdump: loaded Tainted: G        W    L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff947f759220e0 ti: ffff947fb8958000 task.ti: ffff947fb8958000
RIP: 0010:[<ffffffffc01d83be>]  [<ffffffffc01d83be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff947fb895b890  EFLAGS: 00000246
RAX: ffffa90b81880000 RBX: ffff947fb7e8b900 RCX: 000000000000000a
RDX: ffff947f7598f600 RSI: ffff947fb501c000 RDI: 000000000000000e
RBP: ffff947fb895b8a0 R08: 0000000000000002 R09: 000000000ae7dfff
R10: ffff947f75e600d8 R11: ffff947fa4d34540 R12: 0000000100000001
R13: ffff947f4339e580 R14: ffff947fb895b804 R15: ffffffffadb557ab
FS:  00007f7fb971a700(0000) GS:ffff947fbb640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000556c2992a310 CR3: 0000000076f68000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc023171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc022a2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffadcdc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffadce5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffadb4cf59>] __blk_run_queue+0x39/0x50
[<ffffffffadb4d01a>] queue_unplugged+0x2a/0xa0
[<ffffffffadb50858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffadb50cc4>] blk_finish_plug+0x14/0x40
[<ffffffffad9c91b1>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffad9c9801>] ra_submit+0x21/0x30
[<ffffffffad9be6a5>] filemap_fault+0x105/0x490
[<ffffffffad8d20a0>] ? task_rq_unlock+0x20/0x20
[<ffffffffadf7e342>] ? down_read+0x12/0x40
[<ffffffffc047555e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc047575c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffad9ea29a>] __do_fault.isra.61+0x8a/0x100
[<ffffffffad9ea84c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffffad9ef2fa>] handle_pte_fault+0x22a/0xe20
[<ffffffffad9f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffadf86633>] __do_page_fault+0x213/0x500
[<ffffffffadf86955>] do_page_fault+0x35/0x90
[<ffffffffadf82768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 1e 
Complete report #622455
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04443be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc042c71e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04882e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8269b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff826a45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff82518ad9
Build id: 3.10.0-862.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff82518b9a
Build id: 3.10.0-862.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8251c358
Build id: 3.10.0-862.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8251c7c4
Build id: 3.10.0-862.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff823a1241
Build id: 3.10.0-862.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff823a18e1
Build id: 3.10.0-862.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff82396265
Build id: 3.10.0-862.el7.x86_64
0x105
13
? update_curr
vmlinux 0xffffffff822d816c
Build id: 3.10.0-862.el7.x86_64
0x14c
14
? account_entity_dequeue
vmlinux 0xffffffff822d4ade
Build id: 3.10.0-862.el7.x86_64
0xae
15
? down_read
vmlinux 0xffffffff82912062
Build id: 3.10.0-862.el7.x86_64
0x12
16
xfs_filemap_fault
xfs 0xffffffffc062d85f
Build id: 3.10.0-862.el7.x86_64
0x5f
17
__do_fault.isra.58
vmlinux 0xffffffff823c05fa
Build id: 3.10.0-862.el7.x86_64
0x8a
18
do_read_fault.isra.60
vmlinux 0xffffffff823c0b5c
Build id: 3.10.0-862.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffff823c52dc
Build id: 3.10.0-862.el7.x86_64
0x2dc
20
? schedule_hrtimeout_range_clock
vmlinux 0xffffffff82911f7e
Build id: 3.10.0-862.el7.x86_64
0xbe
21
? hrtimer_get_res
vmlinux 0xffffffff822be870
Build id: 3.10.0-862.el7.x86_64
0x50
22
handle_mm_fault
vmlinux 0xffffffff823c747d
Build id: 3.10.0-862.el7.x86_64
0x39d
23
__do_page_fault
vmlinux 0xffffffff8291a597
Build id: 3.10.0-862.el7.x86_64
0x197
24
do_page_fault
vmlinux 0xffffffff8291a925
Build id: 3.10.0-862.el7.x86_64
0x35
25
page_fault
vmlinux 0xffffffff82916768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [qmgr:1275]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper snd_ens1371 ablk_helper snd_rawmidi ppdev snd_ac97_codec cryptd ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd vmw_balloon soundcore sg joydev pcspkr nfit libnvdimm parport_pc parport i2c_piix4 shpchp vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm serio_raw mptspi scsi_transport_spi mptscsih mptbase e1000 ata_piix libata i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1275 Comm: qmgr Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8ac678761fa0 ti: ffff8ac674e2c000 task.ti: ffff8ac674e2c000
RIP: 0010:[<ffffffffc04443be>]  [<ffffffffc04443be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8ac674e2f8e0  EFLAGS: 00000246
RAX: ffffaf3180380000 RBX: ffff8ac605cc63e0 RCX: 0000000000000010
RDX: ffff8ac635b6d400 RSI: ffff8ac635903000 RDI: 000000000000000e
RBP: ffff8ac674e2f8f0 R08: 0000000000000003 R09: 000000005ddfcfff
R10: ffff8ac6350710d8 R11: ffff8ac601a78380 R12: 0000000100000020
R13: ffff8ac606abad00 R14: ffff8ac674e2f854 R15: ffffffff825212cb
FS:  00007f089d801880(0000) GS:ffff8ac67b640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005632f46e1d60 CR3: 0000000074e8c000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc042c71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04882e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8269b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff826a45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff82518ad9>] __blk_run_queue+0x39/0x50
[<ffffffff82518b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff8251c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8251c7c4>] blk_finish_plug+0x14/0x40
[<ffffffff823a1241>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff823a18e1>] ra_submit+0x21/0x30
[<ffffffff82396265>] filemap_fault+0x105/0x490
[<ffffffff822d816c>] ? update_curr+0x14c/0x1e0
[<ffffffff822d4ade>] ? account_entity_dequeue+0xae/0xd0
[<ffffffff82912062>] ? down_read+0x12/0x40
[<ffffffffc062d85f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffff823c05fa>] __do_fault.isra.58+0x8a/0x100
[<ffffffff823c0b5c>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffff823c52dc>] handle_pte_fault+0x2dc/0xc30
[<ffffffff82911f7e>] ? schedule_hrtimeout_range_clock+0xbe/0x150
[<ffffffff822be870>] ? hrtimer_get_res+0x50/0x50
[<ffffffff823c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8291a597>] __do_page_fault+0x197/0x4f0
[<ffffffff8291a925>] do_page_fault+0x35/0x90
[<ffffffff82916768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 45 
Complete report #353931
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03b23be
Build id: 3.10.0-957.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc030071e
Build id: 3.10.0-957.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04302e0
Build id: 3.10.0-957.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff952d0960
Build id: 3.10.0-957.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff952d9efc
Build id: 3.10.0-957.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff95142ff9
Build id: 3.10.0-957.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff951430ba
Build id: 3.10.0-957.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff951468e8
Build id: 3.10.0-957.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff95146d54
Build id: 3.10.0-957.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff94fc3601
Build id: 3.10.0-957.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff94fc3c51
Build id: 3.10.0-957.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff94fb8ba5
Build id: 3.10.0-957.el7.x86_64
0x105
13
? security_inode_permission
vmlinux 0xffffffff950f8b92
Build id: 3.10.0-957.el7.x86_64
0x22
14
? down_read
vmlinux 0xffffffff95566e72
Build id: 3.10.0-957.el7.x86_64
0x12
15
__xfs_filemap_fault
xfs 0xffffffffc04bcd0e
Build id: 3.10.0-957.el7.x86_64
0x7e
16
xfs_filemap_fault
xfs 0xffffffffc04bcf0c
Build id: 3.10.0-957.el7.x86_64
0x2c
17
__do_fault.isra.59
vmlinux 0xffffffff94fe41da
Build id: 3.10.0-957.el7.x86_64
0x8a
18
do_read_fault.isra.61
vmlinux 0xffffffff94fe478c
Build id: 3.10.0-957.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffff94fe9134
Build id: 3.10.0-957.el7.x86_64
0x2f4
20
? user_path_at_empty
vmlinux 0xffffffff95053cd2
Build id: 3.10.0-957.el7.x86_64
0x72
21
? __check_object_size
vmlinux 0xffffffff9503e03a
Build id: 3.10.0-957.el7.x86_64
0x1ca
22
handle_mm_fault
vmlinux 0xffffffff94febc6d
Build id: 3.10.0-957.el7.x86_64
0x39d
23
__do_page_fault
vmlinux 0xffffffff9556f5e3
Build id: 3.10.0-957.el7.x86_64
0x203
24
do_page_fault
vmlinux 0xffffffff9556f915
Build id: 3.10.0-957.el7.x86_64
0x35
25
? error_swapgs
vmlinux 0xffffffff9556ba96
Build id: 3.10.0-957.el7.x86_64
0xa7
26
page_fault
vmlinux 0xffffffff9556b758
Build id: 3.10.0-957.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [vmtoolsd:5691]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sunrpc ppdev coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptspi e1000 scsi_transport_spi nfit mptscsih libnvdimm mptbase floppy vmwgfx ata_generic pata_acpi drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 5691 Comm: vmtoolsd Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8a4f75a6b0c0 ti: ffff8a4fb6dc0000 task.ti: ffff8a4fb6dc0000
RIP: 0010:[<ffffffffc03b23be>]  [<ffffffffc03b23be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8a4fb6dc38a0  EFLAGS: 00000246
RAX: ffffb990c03c0000 RBX: ffff8a4f44fee540 RCX: 000000000000000f
RDX: ffff8a4f75f73600 RSI: ffff8a4f74c4c000 RDI: 000000000000000e
RBP: ffff8a4fb6dc38b0 R08: 0000000000000003 R09: 000000001d3b3fff
R10: ffff8a4f7503e0d8 R11: ffff8a4f85605500 R12: 0000000100000003
R13: ffff8a4f4dffee80 R14: ffff8a4fb6dc3814 R15: ffffffff9514b83b
FS:  00007f8caf7cd880(0000) GS:ffff8a4fbc640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8caf36e050 CR3: 0000000074c18000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc030071e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04302e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff952d0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff952d9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff95142ff9>] __blk_run_queue+0x39/0x50
[<ffffffff951430ba>] queue_unplugged+0x2a/0xa0
[<ffffffff951468e8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff95146d54>] blk_finish_plug+0x14/0x40
[<ffffffff94fc3601>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff94fc3c51>] ra_submit+0x21/0x30
[<ffffffff94fb8ba5>] filemap_fault+0x105/0x490
[<ffffffff950f8b92>] ? security_inode_permission+0x22/0x30
[<ffffffff95566e72>] ? down_read+0x12/0x40
[<ffffffffc04bcd0e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc04bcf0c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff94fe41da>] __do_fault.isra.59+0x8a/0x100
[<ffffffff94fe478c>] do_read_fault.isra.61+0x4c/0x1b0
[<ffffffff94fe9134>] handle_pte_fault+0x2f4/0xd10
[<ffffffff95053cd2>] ? user_path_at_empty+0x72/0xc0
[<ffffffff9503e03a>] ? __check_object_size+0x1ca/0x250
[<ffffffff94febc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9556f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9556f915>] do_page_fault+0x35/0x90
[<ffffffff9556ba96>] ? error_swapgs+0xa7/0xbd
[<ffffffff9556b758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 3c 
Complete report #276355
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03f63be
Build id: 3.10.0-1062.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc044571e
Build id: 3.10.0-1062.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04ad2e0
Build id: 3.10.0-1062.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff868dc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff868e5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff8674cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8674d01a
Build id: 3.10.0-1062.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff86750858
Build id: 3.10.0-1062.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff86750cc4
Build id: 3.10.0-1062.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff865c91b1
Build id: 3.10.0-1062.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff865c9801
Build id: 3.10.0-1062.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff865be6a5
Build id: 3.10.0-1062.el7.x86_64
0x105
13
? get_page_from_freelist
vmlinux 0xffffffff865c60ef
Build id: 3.10.0-1062.el7.x86_64
0x40f
14
? down_read
vmlinux 0xffffffff86b7e342
Build id: 3.10.0-1062.el7.x86_64
0x12
15
__xfs_filemap_fault
xfs 0xffffffffc069e55e
Build id: 3.10.0-1062.el7.x86_64
0x7e
16
xfs_filemap_fault
xfs 0xffffffffc069e75c
Build id: 3.10.0-1062.el7.x86_64
0x2c
17
__do_fault.isra.61
vmlinux 0xffffffff865ea29a
Build id: 3.10.0-1062.el7.x86_64
0x8a
18
do_read_fault.isra.63
vmlinux 0xffffffff865ea84c
Build id: 3.10.0-1062.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffff865ef2fa
Build id: 3.10.0-1062.el7.x86_64
0x22a
20
? alloc_pages_current
vmlinux 0xffffffff86614c68
Build id: 3.10.0-1062.el7.x86_64
0x98
21
handle_mm_fault
vmlinux 0xffffffff865f200d
Build id: 3.10.0-1062.el7.x86_64
0x39d
22
__do_page_fault
vmlinux 0xffffffff86b86633
Build id: 3.10.0-1062.el7.x86_64
0x213
23
do_page_fault
vmlinux 0xffffffff86b86955
Build id: 3.10.0-1062.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffff86b82768
Build id: 3.10.0-1062.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [gnome-session-c:1853]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun bridge stp llc devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer sg snd soundcore
i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel nfit ata_piix libnvdimm mptspi libata scsi_transport_spi serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1853 Comm: gnome-session-c Not tainted 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff95e5f43ad230 ti: ffff95e5ddc18000 task.ti: ffff95e5ddc18000
RIP: 0010:[<ffffffffc03f63be>]  [<ffffffffc03f63be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff95e5ddc1b890  EFLAGS: 00010246
RAX: ffffaa12c07c0000 RBX: ffff95e5e033c960 RCX: 0000000000000007
RDX: ffff95e5b5b57a00 RSI: ffff95e5b6246000 RDI: 000000000000000e
RBP: ffff95e5ddc1b8a0 R08: 0000000000000003 R09: 000000005bc3dfff
R10: ffff95e5b59360d8 R11: ffff95e5b59f16c0 R12: 000000010000000c
R13: ffff95e5e0284f00 R14: ffff95e5ddc1b804 R15: ffffffff867557ab
FS:  00007f322066aa40(0000) GS:ffff95e5fa000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f321868a4cc CR3: 000000005dcf8000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc044571e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04ad2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff868dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff868e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff8674cf59>] __blk_run_queue+0x39/0x50
[<ffffffff8674d01a>] queue_unplugged+0x2a/0xa0
[<ffffffff86750858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff86750cc4>] blk_finish_plug+0x14/0x40
[<ffffffff865c91b1>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff865c9801>] ra_submit+0x21/0x30
[<ffffffff865be6a5>] filemap_fault+0x105/0x490
[<ffffffff865c60ef>] ? get_page_from_freelist+0x40f/0xaa0
[<ffffffff86b7e342>] ? down_read+0x12/0x40
[<ffffffffc069e55e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc069e75c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff865ea29a>] __do_fault.isra.61+0x8a/0x100
[<ffffffff865ea84c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffff865ef2fa>] handle_pte_fault+0x22a/0xe20
[<ffffffff86614c68>] ? alloc_pages_current+0x98/0x110
[<ffffffff865f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff86b86633>] __do_page_fault+0x213/0x500
[<ffffffff86b86955>] do_page_fault+0x35/0x90
[<ffffffff86b82768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 43 40 
Complete report #241626
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02323be
Build id: 3.10.0-957.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02a171e
Build id: 3.10.0-957.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc028a2e0
Build id: 3.10.0-957.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff97ed0960
Build id: 3.10.0-957.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff97ed9efc
Build id: 3.10.0-957.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff97d42ff9
Build id: 3.10.0-957.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff97d430ba
Build id: 3.10.0-957.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff97d468e8
Build id: 3.10.0-957.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff97d46d54
Build id: 3.10.0-957.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff97bc3601
Build id: 3.10.0-957.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff97bc3c51
Build id: 3.10.0-957.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff97bb8ba5
Build id: 3.10.0-957.el7.x86_64
0x105
13
? schedule_timeout
vmlinux 0xffffffff981656a1
Build id: 3.10.0-957.el7.x86_64
0x221
14
? down_read
vmlinux 0xffffffff98166e72
Build id: 3.10.0-957.el7.x86_64
0x12
15
__xfs_filemap_fault
xfs 0xffffffffc048ed0e
Build id: 3.10.0-957.el7.x86_64
0x7e
16
xfs_filemap_fault
xfs 0xffffffffc048ef0c
Build id: 3.10.0-957.el7.x86_64
0x2c
17
__do_fault.isra.59
vmlinux 0xffffffff97be41da
Build id: 3.10.0-957.el7.x86_64
0x8a
18
do_read_fault.isra.61
vmlinux 0xffffffff97be478c
Build id: 3.10.0-957.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffff97be9134
Build id: 3.10.0-957.el7.x86_64
0x2f4
20
handle_mm_fault
vmlinux 0xffffffff97bebc6d
Build id: 3.10.0-957.el7.x86_64
0x39d
21
__do_page_fault
vmlinux 0xffffffff9816f5e3
Build id: 3.10.0-957.el7.x86_64
0x203
22
do_page_fault
vmlinux 0xffffffff9816f915
Build id: 3.10.0-957.el7.x86_64
0x35
23
page_fault
vmlinux 0xffffffff9816b758
Build id: 3.10.0-957.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [java:129042]
Modules linked in: binfmt_misc tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev btusb btrtl btbcm btintel bluetooth uvcvideo snd_ens1371 snd_rawmidi snd_ac97_codec videobuf2_vmalloc videobuf2_memops videobuf2_core videodev ac97_bus snd_seq snd_seq_device snd_pcm rfkill snd_timer snd soundcore sg parport_pc parport vmw_vmci i2c_piix4
ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi e1000 scsi_transport_spi mptscsih libata mptbase nfit drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 129042 Comm: java Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff927f43322080 ti: ffff927f47704000 task.ti: ffff927f47704000
RIP: 0010:[<ffffffffc02323be>]  [<ffffffffc02323be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff927f477078a0  EFLAGS: 00000246
RAX: ffffa17980200000 RBX: ffff927f77138e00 RCX: 0000000000000000
RDX: ffff927f75667200 RSI: ffff927f75f94000 RDI: 000000000000000e
RBP: ffff927f477078b0 R08: 0000000000000003 R09: 0000000031939fff
R10: ffff927f753d80d8 R11: ffff927f75fdcc40 R12: 0000000100000071
R13: ffff927f43097480 R14: ffff927f47707814 R15: ffffffff97d4b83b
FS:  00007f117609f700(0000) GS:ffff927f7c600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f118c341cb6 CR3: 0000000000348000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc02a171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc028a2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff97ed0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff97ed9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff97d42ff9>] __blk_run_queue+0x39/0x50
[<ffffffff97d430ba>] queue_unplugged+0x2a/0xa0
[<ffffffff97d468e8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff97d46d54>] blk_finish_plug+0x14/0x40
[<ffffffff97bc3601>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff97bc3c51>] ra_submit+0x21/0x30
[<ffffffff97bb8ba5>] filemap_fault+0x105/0x490
[<ffffffff981656a1>] ? schedule_timeout+0x221/0x2d0
[<ffffffff98166e72>] ? down_read+0x12/0x40
[<ffffffffc048ed0e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc048ef0c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff97be41da>] __do_fault.isra.59+0x8a/0x100
[<ffffffff97be478c>] do_read_fault.isra.61+0x4c/0x1b0
[<ffffffff97be9134>] handle_pte_fault+0x2f4/0xd10
[<ffffffff97bebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9816f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9816f915>] do_page_fault+0x35/0x90
[<ffffffff9816b758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 24 
Complete report #143640
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02373be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03a471e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03462e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8bc9b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8bca45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff8bb18ad9
Build id: 3.10.0-862.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8bb18b9a
Build id: 3.10.0-862.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8bb1c358
Build id: 3.10.0-862.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8bb1c7c4
Build id: 3.10.0-862.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff8b9a1241
Build id: 3.10.0-862.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff8b9a18e1
Build id: 3.10.0-862.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff8b996265
Build id: 3.10.0-862.el7.x86_64
0x105
13
? update_curr
vmlinux 0xffffffff8b8d80a6
Build id: 3.10.0-862.el7.x86_64
0x86
14
? down_read
vmlinux 0xffffffff8bf12062
Build id: 3.10.0-862.el7.x86_64
0x12
15
xfs_filemap_fault
xfs 0xffffffffc048285f
Build id: 3.10.0-862.el7.x86_64
0x5f
16
__do_fault.isra.58
vmlinux 0xffffffff8b9c05fa
Build id: 3.10.0-862.el7.x86_64
0x8a
17
? check_preempt_curr
vmlinux 0xffffffff8b8cb8ca
Build id: 3.10.0-862.el7.x86_64
0x8a
18
do_read_fault.isra.60
vmlinux 0xffffffff8b9c0b5c
Build id: 3.10.0-862.el7.x86_64
0x4c
19
? ttwu_do_activate.constprop.93
vmlinux 0xffffffff8b8cba9d
Build id: 3.10.0-862.el7.x86_64
0x5d
20
handle_pte_fault
vmlinux 0xffffffff8b9c52dc
Build id: 3.10.0-862.el7.x86_64
0x2dc
21
? wake_up_q
vmlinux 0xffffffff8b8cee3b
Build id: 3.10.0-862.el7.x86_64
0x5b
22
? futex_wake
vmlinux 0xffffffff8b9033a0
Build id: 3.10.0-862.el7.x86_64
0x90
23
handle_mm_fault
vmlinux 0xffffffff8b9c747d
Build id: 3.10.0-862.el7.x86_64
0x39d
24
__do_page_fault
vmlinux 0xffffffff8bf1a597
Build id: 3.10.0-862.el7.x86_64
0x197
25
do_page_fault
vmlinux 0xffffffff8bf1a925
Build id: 3.10.0-862.el7.x86_64
0x35
26
page_fault
vmlinux 0xffffffff8bf16768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 72s! [dockerd-current:15677]
Modules linked in: tcp_lp fuse veth xt_multiport xt_set iptable_raw ip_set_hash_ip ip_set_hash_net ip_set nf_conntrack_netlink nfnetlink xt_statistic xt_nat ip_vs_sh ip_vs_wrr ip_vs_rr tcp_diag inet_diag ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 rpcsec_gss_krb5 xt_comment xt_mark xt_CHECKSUM iptable_mangle xt_addrtype ipt_MASQUERADE nf_nat_masquerade_ipv4 br_netfilter iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 ip_vs nf_conntrack tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink cfg80211 rfkill overlay(T) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 aesni_intel vmw_balloon
snd_rawmidi snd_ac97_codec lrw gf128mul glue_helper ablk_helper cryptd ac97_bus snd_seq snd_seq_device pcspkr snd_pcm joydev snd_timer snd sg nfit soundcore vmw_vmci i2c_piix4 libnvdimm shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih drm crc32c_intel e1000 libata mptbase serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 15677 Comm: dockerd-current Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff89cc1179cf10 ti: ffff89cb106b8000 task.ti: ffff89cb106b8000
RIP: 0010:[<ffffffffc02373be>]  [<ffffffffc02373be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff89cb106bb8e0  EFLAGS: 00000246
RAX: ffffb31903480000 RBX: ffff89c838f75d80 RCX: 0000000000000000
RDX: ffff89cde02e2c00 RSI: ffff89cde1b7d000 RDI: 000000000000000e
RBP: ffff89cb106bb8f0 R08: 0000000000000003 R09: 00000001600fdfff
R10: ffff89cde70bf0d8 R11: ffff89cd7dfb3100 R12: 0000000100000005
R13: ffff89cdcb76d380 R14: ffff89cb106bb854 R15: ffffffff8bb212cb
FS:  00007f77e5ffb700(0000) GS:ffff89cde5700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000812287 CR3: 0000000621680000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc03a471e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03462e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8bc9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8bca45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff8bb18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff8bb18b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff8bb1c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8bb1c7c4>] blk_finish_plug+0x14/0x40
[<ffffffff8b9a1241>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff8b9a18e1>] ra_submit+0x21/0x30
[<ffffffff8b996265>] filemap_fault+0x105/0x490
[<ffffffff8b8d80a6>] ? update_curr+0x86/0x1e0
[<ffffffff8bf12062>] ? down_read+0x12/0x40
[<ffffffffc048285f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffff8b9c05fa>] __do_fault.isra.58+0x8a/0x100
[<ffffffff8b8cb8ca>] ? check_preempt_curr+0x8a/0xa0
[<ffffffff8b9c0b5c>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffff8b8cba9d>] ? ttwu_do_activate.constprop.93+0x5d/0x70
[<ffffffff8b9c52dc>] handle_pte_fault+0x2dc/0xc30
[<ffffffff8b8cee3b>] ? wake_up_q+0x5b/0x80
[<ffffffff8b9033a0>] ? futex_wake+0x90/0x180
[<ffffffff8b9c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8bf1a597>] __do_page_fault+0x197/0x4f0
[<ffffffff8bf1a925>] do_page_fault+0x35/0x90
[<ffffffff8bf16768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 24 
Complete report #127751
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc020d3be
Build id: 3.10.0-957.21.3.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc025271e
Build id: 3.10.0-957.21.3.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02282e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb76d1700
Build id: 3.10.0-957.21.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb76dac9c
Build id: 3.10.0-957.21.3.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffb7543849
Build id: 3.10.0-957.21.3.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb754390a
Build id: 3.10.0-957.21.3.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb7547148
Build id: 3.10.0-957.21.3.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb75475b4
Build id: 3.10.0-957.21.3.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffb73c3b51
Build id: 3.10.0-957.21.3.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffb73c41a1
Build id: 3.10.0-957.21.3.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffb73b9165
Build id: 3.10.0-957.21.3.el7.x86_64
0x105
13
? wake_up_state
vmlinux 0xffffffffb72d6b60
Build id: 3.10.0-957.21.3.el7.x86_64
0x20
14
? down_read
vmlinux 0xffffffffb79681b2
Build id: 3.10.0-957.21.3.el7.x86_64
0x12
15
__xfs_filemap_fault
xfs 0xffffffffc0450a9e
Build id: 3.10.0-957.21.3.el7.x86_64
0x7e
16
xfs_filemap_fault
xfs 0xffffffffc0450c9c
Build id: 3.10.0-957.21.3.el7.x86_64
0x2c
17
__do_fault.isra.59
vmlinux 0xffffffffb73e47ea
Build id: 3.10.0-957.21.3.el7.x86_64
0x8a
18
do_read_fault.isra.61
vmlinux 0xffffffffb73e4d9c
Build id: 3.10.0-957.21.3.el7.x86_64
0x4c
19
? sock_aio_write
vmlinux 0xffffffffb781957d
Build id: 3.10.0-957.21.3.el7.x86_64
0x15d
20
handle_pte_fault
vmlinux 0xffffffffb73e9744
Build id: 3.10.0-957.21.3.el7.x86_64
0x2f4
21
handle_mm_fault
vmlinux 0xffffffffb73ec27d
Build id: 3.10.0-957.21.3.el7.x86_64
0x39d
22
__do_page_fault
vmlinux 0xffffffffb7970603
Build id: 3.10.0-957.21.3.el7.x86_64
0x203
23
do_page_fault
vmlinux 0xffffffffb7970925
Build id: 3.10.0-957.21.3.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffffb796c768
Build id: 3.10.0-957.21.3.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [gcd:3528]
Modules linked in: fuse vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi iosf_mbi snd_ac97_codec crc32_pclmul ppdev ac97_bus snd_seq vmw_balloon snd_seq_device ghash_clmulni_intel cryptd snd_pcm joydev pcspkr snd_timer snd soundcore sg vmw_vmci i2c_piix4 parport_pc parport binfmt_misc sch_fq_codel ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm serio_raw libata mptspi scsi_transport_spi mptscsih e1000 mptbase nfit drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3528 Comm: gcd Not tainted 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff965e04439040 ti: ffff965df3f2c000 task.ti: ffff965df3f2c000
RIP: 0010:[<ffffffffc020d3be>]  [<ffffffffc020d3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff965df3f2f8a0  EFLAGS: 00000246
RAX: ffffae5fc0440000 RBX: ffff965dc5a9ec00 RCX: 000000000000001d
RDX: ffff965df567d200 RSI: ffff965df5745000 RDI: 000000000000000e
RBP: ffff965df3f2f8b0 R08: 0000000000000002 R09: 0000000066e81fff
R10: ffff965e38f2d0d8 R11: ffff965e1469a840 R12: 0000000100000001
R13: ffff965e024ca280 R14: ffff965df3f2f814 R15: ffffffffb754c09b
FS:  00007f69cf82e740(0000) GS:ffff965e39400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000416490 CR3: 000000000185e000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc025271e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02282e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb76d1700>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb76dac9c>] scsi_request_fn+0x4cc/0x680
[<ffffffffb7543849>] __blk_run_queue+0x39/0x50
[<ffffffffb754390a>] queue_unplugged+0x2a/0xa0
[<ffffffffb7547148>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb75475b4>] blk_finish_plug+0x14/0x40
[<ffffffffb73c3b51>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffb73c41a1>] ra_submit+0x21/0x30
[<ffffffffb73b9165>] filemap_fault+0x105/0x490
[<ffffffffb72d6b60>] ? wake_up_state+0x20/0x20
[<ffffffffb79681b2>] ? down_read+0x12/0x40
[<ffffffffc0450a9e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc0450c9c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffb73e47ea>] __do_fault.isra.59+0x8a/0x100
[<ffffffffb73e4d9c>] do_read_fault.isra.61+0x4c/0x1b0
[<ffffffffb781957d>] ? sock_aio_write+0x15d/0x180
[<ffffffffb73e9744>] handle_pte_fault+0x2f4/0xd10
[<ffffffffb73ec27d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb7970603>] __do_page_fault+0x203/0x4f0
[<ffffffffb7970925>] do_page_fault+0x35/0x90
[<ffffffffb796c768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 b3 21 
Complete report #67867
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc016b3be
Build id: 3.10.0-1062.4.3.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc019771e
Build id: 3.10.0-1062.4.3.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc01af2e0
Build id: 3.10.0-1062.4.3.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff900de1a0
Build id: 3.10.0-1062.4.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff900e780c
Build id: 3.10.0-1062.4.3.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff8ff4e7f9
Build id: 3.10.0-1062.4.3.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8ff4e8ba
Build id: 3.10.0-1062.4.3.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8ff52108
Build id: 3.10.0-1062.4.3.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8ff52574
Build id: 3.10.0-1062.4.3.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff8fdca7b1
Build id: 3.10.0-1062.4.3.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff8fdcae01
Build id: 3.10.0-1062.4.3.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff8fdbf7b5
Build id: 3.10.0-1062.4.3.el7.x86_64
0x105
13
? down_read
vmlinux 0xffffffff9037ec82
Build id: 3.10.0-1062.4.3.el7.x86_64
0x12
14
__xfs_filemap_fault
xfs 0xffffffffc03f755e
Build id: 3.10.0-1062.4.3.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc03f775c
Build id: 3.10.0-1062.4.3.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffff8fdeb8ca
Build id: 3.10.0-1062.4.3.el7.x86_64
0x8a
17
do_read_fault.isra.63
vmlinux 0xffffffff8fdebe7c
Build id: 3.10.0-1062.4.3.el7.x86_64
0x4c
18
handle_pte_fault
vmlinux 0xffffffff8fdf092a
Build id: 3.10.0-1062.4.3.el7.x86_64
0x22a
19
handle_mm_fault
vmlinux 0xffffffff8fdf363d
Build id: 3.10.0-1062.4.3.el7.x86_64
0x39d
20
__do_page_fault
vmlinux 0xffffffff90387653
Build id: 3.10.0-1062.4.3.el7.x86_64
0x213
21
do_page_fault
vmlinux 0xffffffff90387975
Build id: 3.10.0-1062.4.3.el7.x86_64
0x35
22
page_fault
vmlinux 0xffffffff90383778
Build id: 3.10.0-1062.4.3.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 37s! [java:1866]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr pcc_cpufreq parport_pc parport vmw_vmci i2c_piix4 sch_fq_codel ip_tables xfs libcrc32c sr_mod
cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common nfit crc32c_intel libata serio_raw libnvdimm mptspi scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 1866 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-1062.4.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff892e57ad62a0 ti: ffff892e59ea4000 task.ti: ffff892e59ea4000
RIP: 0010:[<ffffffffc016b3be>]  [<ffffffffc016b3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff892e59ea7890  EFLAGS: 00000246
RAX: ffffb4df80700000 RBX: ffff892e71341240 RCX: 000000000000000d
RDX: ffff892df4fb6600 RSI: ffff892df5e48000 RDI: 000000000000000e
RBP: ffff892e59ea78a0 R08: 0000000000000003 R09: 000000000f4dbfff
R10: ffff892df582c0d8 R11: ffff892e50e0e300 R12: 0000000100000003
R13: ffff892e597a1c80 R14: ffff892e59ea7804 R15: ffffffff8ff5705b
FS:  00007f33311eb700(0000) GS:ffff892e7a840000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f33686a2630 CR3: 00000000a52fa000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc019771e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01af2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff900de1a0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff900e780c>] scsi_request_fn+0x4cc/0x680
[<ffffffff8ff4e7f9>] __blk_run_queue+0x39/0x50
[<ffffffff8ff4e8ba>] queue_unplugged+0x2a/0xa0
[<ffffffff8ff52108>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8ff52574>] blk_finish_plug+0x14/0x40
[<ffffffff8fdca7b1>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff8fdcae01>] ra_submit+0x21/0x30
[<ffffffff8fdbf7b5>] filemap_fault+0x105/0x490
[<ffffffff9037ec82>] ? down_read+0x12/0x40
[<ffffffffc03f755e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc03f775c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff8fdeb8ca>] __do_fault.isra.61+0x8a/0x100
[<ffffffff8fdebe7c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffff8fdf092a>] handle_pte_fault+0x22a/0xe20
[<ffffffff8fdf363d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff90387653>] __do_page_fault+0x213/0x500
[<ffffffff90387975>] do_page_fault+0x35/0x90
[<ffffffff90383778>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 93 17 
Complete report #20809
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01643be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc01b371e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc01322e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffaa89b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffaa8a45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffaa718ad9
Build id: 3.10.0-862.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffaa718b9a
Build id: 3.10.0-862.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffaa71c358
Build id: 3.10.0-862.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffaa71c7c4
Build id: 3.10.0-862.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffaa5a1241
Build id: 3.10.0-862.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffaa5a18e1
Build id: 3.10.0-862.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffaa596265
Build id: 3.10.0-862.el7.x86_64
0x105
13
? down_read
vmlinux 0xffffffffaab12062
Build id: 3.10.0-862.el7.x86_64
0x12
14
xfs_filemap_fault
xfs 0xffffffffc03b485f
Build id: 3.10.0-862.el7.x86_64
0x5f
15
__do_fault.isra.58
vmlinux 0xffffffffaa5c05fa
Build id: 3.10.0-862.el7.x86_64
0x8a
16
do_read_fault.isra.60
vmlinux 0xffffffffaa5c0b5c
Build id: 3.10.0-862.el7.x86_64
0x4c
17
handle_pte_fault
vmlinux 0xffffffffaa5c52dc
Build id: 3.10.0-862.el7.x86_64
0x2dc
18
? futex_wake
vmlinux 0xffffffffaa5033a0
Build id: 3.10.0-862.el7.x86_64
0x90
19
handle_mm_fault
vmlinux 0xffffffffaa5c747d
Build id: 3.10.0-862.el7.x86_64
0x39d
20
__do_page_fault
vmlinux 0xffffffffaab1a597
Build id: 3.10.0-862.el7.x86_64
0x197
21
do_page_fault
vmlinux 0xffffffffaab1a925
Build id: 3.10.0-862.el7.x86_64
0x35
22
page_fault
vmlinux 0xffffffffaab16768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 27s! [mysqld:2068]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm fuse tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink vmw_vsock_vmci_transport vmw_vmci vsock bnep ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter iptable_raw iptable_security xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul snd_ens1371 glue_helper ablk_helper cryptd snd_rawmidi btusb snd_ac97_codec btrtl btbcm btintel vmw_balloon ac97_bus snd_seq bluetooth snd_seq_device pcspkr snd_pcm joydev snd_timer snd soundcore rfkill sg i2c_piix4 shpchp nfit libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod
cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix mptspi serio_raw libata scsi_transport_spi mptscsih e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 2068 Comm: mysqld Kdump: loaded Tainted: G        W    L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff97e7629c0fd0 ti: ffff97e75fe80000 task.ti: ffff97e75fe80000
RIP: 0010:[<ffffffffc01643be>]  [<ffffffffc01643be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff97e75fe838e0  EFLAGS: 00000246
RAX: ffffab3900380000 RBX: ffff97e763e266a0 RCX: 0000000000000011
RDX: ffff97e735a41000 RSI: ffff97e73584c000 RDI: 000000000000000e
RBP: ffff97e75fe838f0 R08: 0000000000000003 R09: 000000006017efff
R10: ffff97e7356fb0d8 R11: ffff97e75562da40 R12: 0000000100000006
R13: ffff97e70fc6a100 R14: ffff97e75fe83854 R15: ffffffffaa7212cb
FS:  00007f854b627700(0000) GS:ffff97e77b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8567fb3370 CR3: 0000000075ade000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc01b371e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01322e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffaa89b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffaa8a45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffffaa718ad9>] __blk_run_queue+0x39/0x50
[<ffffffffaa718b9a>] queue_unplugged+0x2a/0xa0
[<ffffffffaa71c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffaa71c7c4>] blk_finish_plug+0x14/0x40
[<ffffffffaa5a1241>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffaa5a18e1>] ra_submit+0x21/0x30
[<ffffffffaa596265>] filemap_fault+0x105/0x490
[<ffffffffaab12062>] ? down_read+0x12/0x40
[<ffffffffc03b485f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffffaa5c05fa>] __do_fault.isra.58+0x8a/0x100
[<ffffffffaa5c0b5c>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffffaa5c52dc>] handle_pte_fault+0x2dc/0xc30
[<ffffffffaa5033a0>] ? futex_wake+0x90/0x180
[<ffffffffaa5c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffaab1a597>] __do_page_fault+0x197/0x4f0
[<ffffffffaab1a925>] do_page_fault+0x35/0x90
[<ffffffffaab16768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 17 
Complete report #3496
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03cf3be
Build id: 3.10.0-862.2.3.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03b171e
Build id: 3.10.0-862.2.3.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03ea2e0
Build id: 3.10.0-862.2.3.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa2a9aff0
Build id: 3.10.0-862.2.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa2aa455c
Build id: 3.10.0-862.2.3.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffa2918a79
Build id: 3.10.0-862.2.3.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffa2918b3a
Build id: 3.10.0-862.2.3.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffa291c2f8
Build id: 3.10.0-862.2.3.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffa291c764
Build id: 3.10.0-862.2.3.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffa27a11e1
Build id: 3.10.0-862.2.3.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffa27a1881
Build id: 3.10.0-862.2.3.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffa2796205
Build id: 3.10.0-862.2.3.el7.x86_64
0x105
13
? down_read
vmlinux 0xffffffffa2d12092
Build id: 3.10.0-862.2.3.el7.x86_64
0x12
14
xfs_filemap_fault
xfs 0xffffffffc060b85f
Build id: 3.10.0-862.2.3.el7.x86_64
0x5f
15
__do_fault.isra.58
vmlinux 0xffffffffa27c059a
Build id: 3.10.0-862.2.3.el7.x86_64
0x8a
16
do_read_fault.isra.60
vmlinux 0xffffffffa27c0afc
Build id: 3.10.0-862.2.3.el7.x86_64
0x4c
17
handle_pte_fault
vmlinux 0xffffffffa27c527c
Build id: 3.10.0-862.2.3.el7.x86_64
0x2dc
18
handle_mm_fault
vmlinux 0xffffffffa27c741d
Build id: 3.10.0-862.2.3.el7.x86_64
0x39d
19
__do_page_fault
vmlinux 0xffffffffa2d1a557
Build id: 3.10.0-862.2.3.el7.x86_64
0x197
20
do_page_fault
vmlinux 0xffffffffa2d1a8e5
Build id: 3.10.0-862.2.3.el7.x86_64
0x35
21
? error_swapgs
vmlinux 0xffffffffa2d16aa6
Build id: 3.10.0-862.2.3.el7.x86_64
0xa7
22
page_fault
vmlinux 0xffffffffa2d16758
Build id: 3.10.0-862.2.3.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [vmtoolsd:747]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr nfit joydev vmw_balloon sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih libata e1000 crct10dif_pclmul crct10dif_common mptbase crc32c_intel i2c_core serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 747 Comm: vmtoolsd Kdump: loaded Tainted: G             L ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8b12f3fe6eb0 ti: ffff8b11fb488000 task.ti: ffff8b11fb488000
RIP: 0010:[<ffffffffc03cf3be>]  [<ffffffffc03cf3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8b11fb48b8e0  EFLAGS: 00000246
RAX: ffffac7000e80000 RBX: ffff8b11d183f860 RCX: 0000000000000049
RDX: ffff8b117543c200 RSI: ffff8b12ffdaf000 RDI: 000000000000000e
RBP: ffff8b11fb48b8f0 R08: 0000000000000003 R09: 000000007507ffff
R10: ffff8b1175f830d8 R11: ffff8b11ae9a4700 R12: 0000000100000004
R13: ffff8b118b252480 R14: ffff8b11fb48b854 R15: ffffffffa292126b
FS:  00007f511ff10880(0000) GS:ffff8b12ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f511fa6d0cc CR3: 00000000baa0c000 CR4: 00000000000607f0
Call Trace:
[<ffffffffc03b171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03ea2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa2a9aff0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa2aa455c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa2918a79>] __blk_run_queue+0x39/0x50
[<ffffffffa2918b3a>] queue_unplugged+0x2a/0xa0
[<ffffffffa291c2f8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa291c764>] blk_finish_plug+0x14/0x40
[<ffffffffa27a11e1>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffa27a1881>] ra_submit+0x21/0x30
[<ffffffffa2796205>] filemap_fault+0x105/0x490
[<ffffffffa2d12092>] ? down_read+0x12/0x40
[<ffffffffc060b85f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffffa27c059a>] __do_fault.isra.58+0x8a/0x100
[<ffffffffa27c0afc>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffffa27c527c>] handle_pte_fault+0x2dc/0xc30
[<ffffffffa27c741d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa2d1a557>] __do_page_fault+0x197/0x4f0
[<ffffffffa2d1a8e5>] do_page_fault+0x35/0x90
[<ffffffffa2d16aa6>] ? error_swapgs+0xa7/0xbd
[<ffffffffa2d16758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 3d 
Complete report #1418
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03393be
Build id: 3.10.0-957.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc035b71e
Build id: 3.10.0-957.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03542e0
Build id: 3.10.0-957.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb52d0960
Build id: 3.10.0-957.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb52d9efc
Build id: 3.10.0-957.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffb5142ff9
Build id: 3.10.0-957.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb51430ba
Build id: 3.10.0-957.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb51468e8
Build id: 3.10.0-957.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb5146d54
Build id: 3.10.0-957.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffb4fc3601
Build id: 3.10.0-957.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffb4fc3c51
Build id: 3.10.0-957.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffb4fb8ba5
Build id: 3.10.0-957.el7.x86_64
0x105
13
? down_read
vmlinux 0xffffffffb5566e72
Build id: 3.10.0-957.el7.x86_64
0x12
14
__xfs_filemap_fault
xfs 0xffffffffc0510d0e
Build id: 3.10.0-957.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc0510f0c
Build id: 3.10.0-957.el7.x86_64
0x2c
16
__do_fault.isra.59
vmlinux 0xffffffffb4fe41da
Build id: 3.10.0-957.el7.x86_64
0x8a
17
do_read_fault.isra.61
vmlinux 0xffffffffb4fe478c
Build id: 3.10.0-957.el7.x86_64
0x4c
18
handle_pte_fault
vmlinux 0xffffffffb4fe9134
Build id: 3.10.0-957.el7.x86_64
0x2f4
19
handle_mm_fault
vmlinux 0xffffffffb4febc6d
Build id: 3.10.0-957.el7.x86_64
0x39d
20
__do_page_fault
vmlinux 0xffffffffb556f5e3
Build id: 3.10.0-957.el7.x86_64
0x203
21
do_page_fault
vmlinux 0xffffffffb556f915
Build id: 3.10.0-957.el7.x86_64
0x35
22
page_fault
vmlinux 0xffffffffb556b758
Build id: 3.10.0-957.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [gnome-shell:8287]
Modules linked in: tcp_lp rfcomm fuse tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock bnep sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper snd_seq_midi cryptd snd_seq_midi_event ppdev vmw_balloon joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device btusb snd_pcm btrtl btbcm btintel bluetooth snd_timer snd soundcore rfkill sg parport_pc parport vmw_vmci i2c_piix4 xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi mptscsih nfit mptbase e1000 drm_panel_orientation_quirks libnvdimm ata_generic pata_acpi ata_piix
libata dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 8287 Comm: gnome-shell Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff9bb8f4eae180 ti: ffff9bb8dc4c0000 task.ti: ffff9bb8dc4c0000
RIP: 0010:[<ffffffffc03393be>]  [<ffffffffc03393be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9bb8dc4c38a0  EFLAGS: 00000246
RAX: ffffa75f00200000 RBX: ffff9bb8f8dde800 RCX: 000000000000000b
RDX: ffff9bb8f5b02400 RSI: ffff9bb8f5b30000 RDI: 000000000000000e
RBP: ffff9bb8dc4c38b0 R08: 0000000000000003 R09: 0000000019acefff
R10: ffff9bb8f4ebd0d8 R11: ffff9bb8f4eb0c40 R12: 0000000100000041
R13: ffff9bb8e865b180 R14: ffff9bb8dc4c3814 R15: ffffffffb514b83b
FS:  00007f070f95ba80(0000) GS:ffff9bb8fc640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f070510e150 CR3: 000000000018c000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc035b71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03542e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb52d0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb52d9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffb5142ff9>] __blk_run_queue+0x39/0x50
[<ffffffffb51430ba>] queue_unplugged+0x2a/0xa0
[<ffffffffb51468e8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb5146d54>] blk_finish_plug+0x14/0x40
[<ffffffffb4fc3601>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffb4fc3c51>] ra_submit+0x21/0x30
[<ffffffffb4fb8ba5>] filemap_fault+0x105/0x490
[<ffffffffb5566e72>] ? down_read+0x12/0x40
[<ffffffffc0510d0e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc0510f0c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffb4fe41da>] __do_fault.isra.59+0x8a/0x100
[<ffffffffb4fe478c>] do_read_fault.isra.61+0x4c/0x1b0
[<ffffffffb4fe9134>] handle_pte_fault+0x2f4/0xd10
[<ffffffffb4febc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb556f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb556f915>] do_page_fault+0x35/0x90
[<ffffffffb556b758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 73 34