Info

Permalink
Function
mpt_put_msg_frame
First occurrence
2020-10-13
Last occurrence
2024-11-24
State
NEW
Components
kernel
Quality
-41

Statistics

Operating system Count
CentOS 7 51
Architecture Count
x86_64 51
Related packages Count
kernel 51
 0:3.10.0-1160.el7 11
 0:3.10.0-1160.11.1.el7 7
 0:3.10.0-862.el7 6
 0:3.10.0-862.3.3.el7 5
 0:3.10.0-957.el7 5
 0:3.10.0-1062.el7 3
 0:3.10.0-957.10.1.el7 2
 0:3.10.0-957.1.3.el7 2
 0:3.10.0-1127.el7 2
 0:3.10.0-1127.19.1.el7 1
 0:3.10.0-957.5.1.el7 1
 0:3.10.0-862.3.2.el7 1
 0:3.10.0-1127.8.2.el7 1
 0:3.10.0-1160.2.2.el7 1
 0:3.10.0-1160.49.1.el7 1
 0:3.10.0-693.el7 1
 0:3.10.0-1160.31.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #289746
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02213be
Build id: 3.10.0-957.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc029071e
Build id: 3.10.0-957.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc021a2e0
Build id: 3.10.0-957.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb78d0960
Build id: 3.10.0-957.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb78d9efc
Build id: 3.10.0-957.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffb7742ff9
Build id: 3.10.0-957.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb77430ba
Build id: 3.10.0-957.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb77468e8
Build id: 3.10.0-957.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb7746d54
Build id: 3.10.0-957.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffb75c3601
Build id: 3.10.0-957.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffb75c3c51
Build id: 3.10.0-957.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffb75b8ba5
Build id: 3.10.0-957.el7.x86_64
0x105
13
? down_read
vmlinux 0xffffffffb7b66e72
Build id: 3.10.0-957.el7.x86_64
0x12
14
__xfs_filemap_fault
xfs 0xffffffffc0473d0e
Build id: 3.10.0-957.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc0473f0c
Build id: 3.10.0-957.el7.x86_64
0x2c
16
__do_fault.isra.59
vmlinux 0xffffffffb75e41da
Build id: 3.10.0-957.el7.x86_64
0x8a
17
do_read_fault.isra.61
vmlinux 0xffffffffb75e478c
Build id: 3.10.0-957.el7.x86_64
0x4c
18
handle_pte_fault
vmlinux 0xffffffffb75e9134
Build id: 3.10.0-957.el7.x86_64
0x2f4
19
? ep_send_events_proc
vmlinux 0xffffffffb768e100
Build id: 3.10.0-957.el7.x86_64
0x160
20
? schedule_hrtimeout_range_clock
vmlinux 0xffffffffb7b66dfd
Build id: 3.10.0-957.el7.x86_64
0x12d
21
handle_mm_fault
vmlinux 0xffffffffb75ebc6d
Build id: 3.10.0-957.el7.x86_64
0x39d
22
__do_page_fault
vmlinux 0xffffffffb7b6f5e3
Build id: 3.10.0-957.el7.x86_64
0x203
23
do_page_fault
vmlinux 0xffffffffb7b6f915
Build id: 3.10.0-957.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffffb7b6b758
Build id: 3.10.0-957.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 65s! [systemd:1]
Modules linked in: veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) overlay(T) sunrpc snd_seq_midi snd_seq_midi_event coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_ens1371 lrw snd_rawmidi gf128mul glue_helper snd_ac97_codec ablk_helper cryptd ac97_bus snd_seq snd_seq_device ppdev snd_pcm snd_timer vmw_balloon snd soundcore pcspkr sg joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul
crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx e1000 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih libata mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 1 Comm: systemd Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffffa0d3bd170000 ti: ffffa0d3bd178000 task.ti: ffffa0d3bd178000
RIP: 0010:[<ffffffffc02213be>]  [<ffffffffc02213be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffffa0d3bd17b8a0  EFLAGS: 00000246
RAX: ffffb40500cc0000 RBX: ffffa0d2d15d4f00 RCX: 000000000000001d
RDX: ffffa0d275ec3000 RSI: ffffa0d27548c000 RDI: 000000000000000e
RBP: ffffa0d3bd17b8b0 R08: 0000000000000002 R09: 0000000097983fff
R10: ffffa0d40e7a30d8 R11: ffffa0d2cdb90fc0 R12: 0000000100000001
R13: ffffa0d2624c8f00 R14: ffffa0d3bd17b814 R15: ffffffffb774b83b
FS:  00007fc98a8e1940(0000) GS:ffffa0d412680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055fb73410270 CR3: 00000001d2c7c000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc029071e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc021a2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb78d0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb78d9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffb7742ff9>] __blk_run_queue+0x39/0x50
[<ffffffffb77430ba>] queue_unplugged+0x2a/0xa0
[<ffffffffb77468e8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb7746d54>] blk_finish_plug+0x14/0x40
[<ffffffffb75c3601>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffb75c3c51>] ra_submit+0x21/0x30
[<ffffffffb75b8ba5>] filemap_fault+0x105/0x490
[<ffffffffb7b66e72>] ? down_read+0x12/0x40
[<ffffffffc0473d0e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc0473f0c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffb75e41da>] __do_fault.isra.59+0x8a/0x100
[<ffffffffb75e478c>] do_read_fault.isra.61+0x4c/0x1b0
[<ffffffffb75e9134>] handle_pte_fault+0x2f4/0xd10
[<ffffffffb768e100>] ? ep_send_events_proc+0x160/0x1b0
[<ffffffffb7b66dfd>] ? schedule_hrtimeout_range_clock+0x12d/0x150
[<ffffffffb75ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb7b6f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb7b6f915>] do_page_fault+0x35/0x90
[<ffffffffb7b6b758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 f3 22 
Complete report #248630
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc027a3be
Build id: 3.10.0-1062.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02f771e
Build id: 3.10.0-1062.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03732e0
Build id: 3.10.0-1062.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff82edc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff82ee5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff82d4cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff82d4d01a
Build id: 3.10.0-1062.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff82d50858
Build id: 3.10.0-1062.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff82d50cc4
Build id: 3.10.0-1062.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff82bc91b1
Build id: 3.10.0-1062.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff82bc9801
Build id: 3.10.0-1062.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff82bbe6a5
Build id: 3.10.0-1062.el7.x86_64
0x105
13
? task_rq_unlock
vmlinux 0xffffffff82ad20a0
Build id: 3.10.0-1062.el7.x86_64
0x20
14
? down_read
vmlinux 0xffffffff8317e342
Build id: 3.10.0-1062.el7.x86_64
0x12
15
__xfs_filemap_fault
xfs 0xffffffffc04f355e
Build id: 3.10.0-1062.el7.x86_64
0x7e
16
xfs_filemap_fault
xfs 0xffffffffc04f375c
Build id: 3.10.0-1062.el7.x86_64
0x2c
17
__do_fault.isra.61
vmlinux 0xffffffff82bea29a
Build id: 3.10.0-1062.el7.x86_64
0x8a
18
do_read_fault.isra.63
vmlinux 0xffffffff82bea84c
Build id: 3.10.0-1062.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffff82bef2fa
Build id: 3.10.0-1062.el7.x86_64
0x22a
20
? ep_send_events_proc
vmlinux 0xffffffff82c96370
Build id: 3.10.0-1062.el7.x86_64
0x160
21
? schedule_hrtimeout_range_clock
vmlinux 0xffffffff8317e2cd
Build id: 3.10.0-1062.el7.x86_64
0x12d
22
handle_mm_fault
vmlinux 0xffffffff82bf200d
Build id: 3.10.0-1062.el7.x86_64
0x39d
23
__do_page_fault
vmlinux 0xffffffff83186633
Build id: 3.10.0-1062.el7.x86_64
0x213
24
do_page_fault
vmlinux 0xffffffff83186955
Build id: 3.10.0-1062.el7.x86_64
0x35
25
page_fault
vmlinux 0xffffffff83182768
Build id: 3.10.0-1062.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 30s! [systemd-journal:421]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun bridge stp llc devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event btusb btrtl btbcm btintel snd_ens1371 snd_rawmidi bluetooth snd_ac97_codec ac97_bus rfkill snd_seq ppdev snd_seq_device snd_pcm snd_timer snd vmw_balloon
soundcore iosf_mbi joydev crc32_pclmul sg pcspkr ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul vmwgfx crct10dif_common crc32c_intel drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi e1000 scsi_transport_spi mptscsih ata_piix drm nfit libata mptbase libnvdimm drm_panel_orientation_quirks
CPU: 0 PID: 421 Comm: systemd-journal Tainted: G             L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9cd9f5f85230 ti: ffff9cd9fb648000 task.ti: ffff9cd9fb648000
RIP: 0010:[<ffffffffc027a3be>]  [<ffffffffc027a3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9cd9fb64b890  EFLAGS: 00000246
RAX: ffffaa94c02c0000 RBX: ffff9cd9fa34a780 RCX: 0000000000000012
RDX: ffff9cd9f532be00 RSI: ffff9cd9f5252000 RDI: 000000000000000e
RBP: ffff9cd9fb64b8a0 R08: 0000000000000003 R09: 0000000015e49fff
R10: ffff9cd9f52500d8 R11: ffff9cd9f457ad80 R12: 0000000100000005
R13: ffff9cd9c390fc00 R14: ffff9cd9fb64b804 R15: ffffffff82d557ab
FS:  00007fe3e3516880(0000) GS:ffff9cd9fc600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055f23228b7c0 CR3: 000000003aab4000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc02f771e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03732e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff82edc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff82ee5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff82d4cf59>] __blk_run_queue+0x39/0x50
[<ffffffff82d4d01a>] queue_unplugged+0x2a/0xa0
[<ffffffff82d50858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff82d50cc4>] blk_finish_plug+0x14/0x40
[<ffffffff82bc91b1>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff82bc9801>] ra_submit+0x21/0x30
[<ffffffff82bbe6a5>] filemap_fault+0x105/0x490
[<ffffffff82ad20a0>] ? task_rq_unlock+0x20/0x20
[<ffffffff8317e342>] ? down_read+0x12/0x40
[<ffffffffc04f355e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc04f375c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff82bea29a>] __do_fault.isra.61+0x8a/0x100
[<ffffffff82bea84c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffff82bef2fa>] handle_pte_fault+0x22a/0xe20
[<ffffffff82c96370>] ? ep_send_events_proc+0x160/0x1b0
[<ffffffff8317e2cd>] ? schedule_hrtimeout_range_clock+0x12d/0x150
[<ffffffff82bf200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff83186633>] __do_page_fault+0x213/0x500
[<ffffffff83186955>] do_page_fault+0x35/0x90
[<ffffffff83182768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 83 28 
Complete report #166695
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03a83be
Build id: 3.10.0-862.3.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc041171e
Build id: 3.10.0-862.3.2.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03c32e0
Build id: 3.10.0-862.3.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa909b4f0
Build id: 3.10.0-862.3.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa90a4a5c
Build id: 3.10.0-862.3.2.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffa8f18f19
Build id: 3.10.0-862.3.2.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffa8f18fda
Build id: 3.10.0-862.3.2.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffa8f1c798
Build id: 3.10.0-862.3.2.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffa8f1cc04
Build id: 3.10.0-862.3.2.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffa8da1581
Build id: 3.10.0-862.3.2.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffa8da1c21
Build id: 3.10.0-862.3.2.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffa8d965a5
Build id: 3.10.0-862.3.2.el7.x86_64
0x105
13
? account_entity_dequeue
vmlinux 0xffffffffa8cd4e0e
Build id: 3.10.0-862.3.2.el7.x86_64
0xae
14
? down_read
vmlinux 0xffffffffa9313092
Build id: 3.10.0-862.3.2.el7.x86_64
0x12
15
xfs_filemap_fault
xfs 0xffffffffc060a85f
Build id: 3.10.0-862.3.2.el7.x86_64
0x5f
16
__do_fault.isra.58
vmlinux 0xffffffffa8dc093a
Build id: 3.10.0-862.3.2.el7.x86_64
0x8a
17
? __alloc_pages_nodemask
vmlinux 0xffffffffa8d9e497
Build id: 3.10.0-862.3.2.el7.x86_64
0x197
18
do_read_fault.isra.60
vmlinux 0xffffffffa8dc0e9c
Build id: 3.10.0-862.3.2.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffffa8dc561c
Build id: 3.10.0-862.3.2.el7.x86_64
0x2dc
20
? ep_send_events_proc
vmlinux 0xffffffffa8e671a7
Build id: 3.10.0-862.3.2.el7.x86_64
0xb7
21
handle_mm_fault
vmlinux 0xffffffffa8dc77bd
Build id: 3.10.0-862.3.2.el7.x86_64
0x39d
22
__do_page_fault
vmlinux 0xffffffffa931b5c7
Build id: 3.10.0-862.3.2.el7.x86_64
0x197
23
do_page_fault
vmlinux 0xffffffffa931b955
Build id: 3.10.0-862.3.2.el7.x86_64
0x35
24
? error_swapgs
vmlinux 0xffffffffa9317af0
Build id: 3.10.0-862.3.2.el7.x86_64
0xc1
25
page_fault
vmlinux 0xffffffffa9317788
Build id: 3.10.0-862.3.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 244s! [systemd:1]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul ac97_bus snd_seq ghash_clmulni_intel cryptd snd_seq_device snd_pcm ppdev snd_timer snd vmw_balloon soundcore pcspkr nfit joydev sg libnvdimm
parport_pc parport vmw_vmci shpchp i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel drm serio_raw ata_piix mptspi scsi_transport_spi libata mptscsih e1000 i2c_core mptbase dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1 Comm: systemd Kdump: loaded Tainted: G             L ------------   3.10.0-862.3.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8e71064e8000 ti: ffff8e71064f0000 task.ti: ffff8e71064f0000
RIP: 0010:[<ffffffffc03a83be>]  [<ffffffffc03a83be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8e71064f38e0  EFLAGS: 00000246
RAX: ffffb48a406c0000 RBX: ffff8e712d3c4f00 RCX: 000000000000000e
RDX: ffff8e70b509d800 RSI: ffff8e70b5c83000 RDI: 000000000000000e
RBP: ffff8e71064f38f0 R08: 0000000000000002 R09: 000000008317ffff
R10: ffff8e70b5c850d8 R11: ffff8e7123683640 R12: 0000000100000001
R13: ffff8e711f3d8c00 R14: ffff8e71064f3854 R15: ffffffffa8f2170b
FS:  00007f6febc2a940(0000) GS:ffff8e713a440000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff2601009c8 CR3: 0000000035fd6000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc041171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03c32e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa909b4f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa90a4a5c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa8f18f19>] __blk_run_queue+0x39/0x50
[<ffffffffa8f18fda>] queue_unplugged+0x2a/0xa0
[<ffffffffa8f1c798>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa8f1cc04>] blk_finish_plug+0x14/0x40
[<ffffffffa8da1581>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffa8da1c21>] ra_submit+0x21/0x30
[<ffffffffa8d965a5>] filemap_fault+0x105/0x490
[<ffffffffa8cd4e0e>] ? account_entity_dequeue+0xae/0xd0
[<ffffffffa9313092>] ? down_read+0x12/0x40
[<ffffffffc060a85f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffffa8dc093a>] __do_fault.isra.58+0x8a/0x100
[<ffffffffa8d9e497>] ? __alloc_pages_nodemask+0x197/0xbb0
[<ffffffffa8dc0e9c>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffffa8dc561c>] handle_pte_fault+0x2dc/0xc30
[<ffffffffa8e671a7>] ? ep_send_events_proc+0xb7/0x1d0
[<ffffffffa8dc77bd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa931b5c7>] __do_page_fault+0x197/0x4f0
[<ffffffffa931b955>] do_page_fault+0x35/0x90
[<ffffffffa9317af0>] ? error_swapgs+0xc1/0xd7
[<ffffffffa9317788>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 3b 
Complete report #39692
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffbef6a175
Build id: 3.10.0-957.5.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc013c622
Build id: 3.10.0-957.5.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffbecd9510
Build id: 3.10.0-957.5.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffbecd0fe0
Build id: 3.10.0-957.5.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffbecda57c
Build id: 3.10.0-957.5.1.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffbeb435d9
Build id: 3.10.0-957.5.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffbeb4369a
Build id: 3.10.0-957.5.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffbeb46ec8
Build id: 3.10.0-957.5.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffbeb47334
Build id: 3.10.0-957.5.1.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffbe9c38b1
Build id: 3.10.0-957.5.1.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffbe9c3f01
Build id: 3.10.0-957.5.1.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffbe9b8e55
Build id: 3.10.0-957.5.1.el7.x86_64
0x105
13
? down_read
vmlinux 0xffffffffbef66ef2
Build id: 3.10.0-957.5.1.el7.x86_64
0x12
14
__xfs_filemap_fault
xfs 0xffffffffc0317d0e
Build id: 3.10.0-957.5.1.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc0317f0c
Build id: 3.10.0-957.5.1.el7.x86_64
0x2c
16
__do_fault.isra.59
vmlinux 0xffffffffbe9e458a
Build id: 3.10.0-957.5.1.el7.x86_64
0x8a
17
do_read_fault.isra.61
vmlinux 0xffffffffbe9e4b3c
Build id: 3.10.0-957.5.1.el7.x86_64
0x4c
18
handle_pte_fault
vmlinux 0xffffffffbe9e94e4
Build id: 3.10.0-957.5.1.el7.x86_64
0x2f4
19
? ep_send_events_proc
vmlinux 0xffffffffbea8e6b0
Build id: 3.10.0-957.5.1.el7.x86_64
0x160
20
handle_mm_fault
vmlinux 0xffffffffbe9ec01d
Build id: 3.10.0-957.5.1.el7.x86_64
0x39d
21
__do_page_fault
vmlinux 0xffffffffbef6f5e3
Build id: 3.10.0-957.5.1.el7.x86_64
0x203
22
do_page_fault
vmlinux 0xffffffffbef6f915
Build id: 3.10.0-957.5.1.el7.x86_64
0x35
23
page_fault
vmlinux 0xffffffffbef6b758
Build id: 3.10.0-957.5.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 225s! [systemd-machine:5657]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc ppdev sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm nfit ata_piix vmw_pvscsi libata libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 5657 Comm: systemd-machine Kdump: loaded Tainted: G             L ------------   3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9d1034abd140 ti: ffff9d10347f0000 task.ti: ffff9d10347f0000
RIP: 0010:[<ffffffffbef6a175>]  [<ffffffffbef6a175>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff9d10347f3908  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff9d10347f38c8 RCX: ffffffffbf4343c0
RDX: ffffae5e80f90000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9d10347f3908 R08: 0000000000000001 R09: 0000000000000000
R10: ffff9d10354e20d8 R11: ffff9d11b3d41800 R12: 0000000000000020
R13: ffff9d1035efc300 R14: ffffffffbecd8269 R15: ffff9d10347f3880
FS:  00007f0475ac8880(0000) GS:ffff9d11bfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005586c2058310 CR3: 00000000347e6000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc013c622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffbecd9510>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffbecd0fe0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbecda57c>] scsi_request_fn+0x4cc/0x680
[<ffffffffbeb435d9>] __blk_run_queue+0x39/0x50
[<ffffffffbeb4369a>] queue_unplugged+0x2a/0xa0
[<ffffffffbeb46ec8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffbeb47334>] blk_finish_plug+0x14/0x40
[<ffffffffbe9c38b1>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffbe9c3f01>] ra_submit+0x21/0x30
[<ffffffffbe9b8e55>] filemap_fault+0x105/0x490
[<ffffffffbef66ef2>] ? down_read+0x12/0x40
[<ffffffffc0317d0e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc0317f0c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffbe9e458a>] __do_fault.isra.59+0x8a/0x100
[<ffffffffbe9e4b3c>] do_read_fault.isra.61+0x4c/0x1b0
[<ffffffffbe9e94e4>] handle_pte_fault+0x2f4/0xd10
[<ffffffffbea8e6b0>] ? ep_send_events_proc+0x160/0x1b0
[<ffffffffbe9ec01d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffbef6f5e3>] __do_page_fault+0x203/0x500
[<ffffffffbef6f915>] do_page_fault+0x35/0x90
[<ffffffffbef6b758>] page_fault+0x28/0x30
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #992476
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb818b805
Build id: 3.10.0-1160.31.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc008f632
Build id: 3.10.0-1160.31.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb7eec830
Build id: 3.10.0-1160.31.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb7ee4120
Build id: 3.10.0-1160.31.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb7eed84c
Build id: 3.10.0-1160.31.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffb7d539f9
Build id: 3.10.0-1160.31.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb7d53aba
Build id: 3.10.0-1160.31.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb7d57398
Build id: 3.10.0-1160.31.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb7d57804
Build id: 3.10.0-1160.31.1.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffb7bcb931
Build id: 3.10.0-1160.31.1.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffb7bcbf51
Build id: 3.10.0-1160.31.1.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffb7bc0935
Build id: 3.10.0-1160.31.1.el7.x86_64
0x105
13
__xfs_filemap_fault
xfs 0xffffffffc030891e
Build id: 3.10.0-1160.31.1.el7.x86_64
0x7e
14
? put_prev_entity
vmlinux 0xffffffffb7ae6021
Build id: 3.10.0-1160.31.1.el7.x86_64
0x31
15
xfs_filemap_fault
xfs 0xffffffffc0308b1c
Build id: 3.10.0-1160.31.1.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffffb7bee2aa
Build id: 3.10.0-1160.31.1.el7.x86_64
0x8a
17
do_read_fault.isra.63
vmlinux 0xffffffffb7bee85c
Build id: 3.10.0-1160.31.1.el7.x86_64
0x4c
18
? schedule_hrtimeout_range_clock
vmlinux 0xffffffffb818862d
Build id: 3.10.0-1160.31.1.el7.x86_64
0x12d
19
handle_mm_fault
vmlinux 0xffffffffb7bf60a0
Build id: 3.10.0-1160.31.1.el7.x86_64
0xa20
20
? ep_scan_ready_list.isra.7
vmlinux 0xffffffffb7c9cfc9
Build id: 3.10.0-1160.31.1.el7.x86_64
0x1b9
21
__do_page_fault
vmlinux 0xffffffffb8190653
Build id: 3.10.0-1160.31.1.el7.x86_64
0x213
22
do_page_fault
vmlinux 0xffffffffb8190975
Build id: 3.10.0-1160.31.1.el7.x86_64
0x35
23
page_fault
vmlinux 0xffffffffb818c778
Build id: 3.10.0-1160.31.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 73s! [systemd-journal:15192]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ppdev joydev vmw_balloon sg vmxnet3 parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect crc_t10dif sysimgblt crct10dif_generic fb_sys_fops ttm ahci drm crct10dif_pclmul ata_piix crct10dif_common libahci
crc32c_intel libata serio_raw e1000 vmw_pvscsi drm_panel_orientation_quirks fuse
CPU: 5 PID: 15192 Comm: systemd-journal Kdump: loaded Tainted: G             L ------------   3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff884745ef4200 ti: ffff883e78d74000 task.ti: ffff883e78d74000
RIP: 0010:[<ffffffffb818b805>]  [<ffffffffb818b805>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff883e78d77980  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffb7db2a33 RCX: ffffffffb8634f40
RDX: ffffa4d8862b0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff883e78d77980 R08: 0000000000000001 R09: 0000000000000000
R10: ffff884db8ca5400 R11: ffff884db8bbd800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff884bb49f8300 R15: ffffffffb7eeb4e9
FS:  00007f2c14c04880(0000) GS:ffff884dbd740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558cb72a3820 CR3: 000000007abfe000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc008f632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffb7eec830>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffb7ee4120>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb7eed84c>] scsi_request_fn+0x4ac/0x680
[<ffffffffb7d539f9>] __blk_run_queue+0x39/0x50
[<ffffffffb7d53aba>] queue_unplugged+0x2a/0xa0
[<ffffffffb7d57398>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb7d57804>] blk_finish_plug+0x14/0x40
[<ffffffffb7bcb931>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffb7bcbf51>] ra_submit+0x21/0x30
[<ffffffffb7bc0935>] filemap_fault+0x105/0x420
[<ffffffffc030891e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffb7ae6021>] ? put_prev_entity+0x31/0x400
[<ffffffffc0308b1c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffb7bee2aa>] __do_fault.isra.61+0x8a/0x100
[<ffffffffb7bee85c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffffb818862d>] ? schedule_hrtimeout_range_clock+0x12d/0x150
[<ffffffffb7bf60a0>] handle_mm_fault+0xa20/0xfb0
[<ffffffffb7c9cfc9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[<ffffffffb8190653>] __do_page_fault+0x213/0x500
[<ffffffffb8190975>] do_page_fault+0x35/0x90
[<ffffffffb818c778>] page_fault+0x28/0x30
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #1292492
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc020c3be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc025471e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02692e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff81e9b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff81ea45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff81d18ad9
Build id: 3.10.0-862.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff81d18b9a
Build id: 3.10.0-862.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff81d1c358
Build id: 3.10.0-862.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff81d1c7c4
Build id: 3.10.0-862.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff81ba1241
Build id: 3.10.0-862.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff81ba18e1
Build id: 3.10.0-862.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff81b96265
Build id: 3.10.0-862.el7.x86_64
0x105
13
? sock_def_readable
vmlinux 0xffffffff81fd005a
Build id: 3.10.0-862.el7.x86_64
0x3a
14
? down_read
vmlinux 0xffffffff82112062
Build id: 3.10.0-862.el7.x86_64
0x12
15
xfs_filemap_fault
xfs 0xffffffffc045985f
Build id: 3.10.0-862.el7.x86_64
0x5f
16
__do_fault.isra.58
vmlinux 0xffffffff81bc05fa
Build id: 3.10.0-862.el7.x86_64
0x8a
17
? poll_select_copy_remaining
vmlinux 0xffffffff81c30830
Build id: 3.10.0-862.el7.x86_64
0x150
18
do_read_fault.isra.60
vmlinux 0xffffffff81bc0b5c
Build id: 3.10.0-862.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffff81bc52dc
Build id: 3.10.0-862.el7.x86_64
0x2dc
20
handle_mm_fault
vmlinux 0xffffffff81bc747d
Build id: 3.10.0-862.el7.x86_64
0x39d
21
__do_page_fault
vmlinux 0xffffffff8211a597
Build id: 3.10.0-862.el7.x86_64
0x197
22
do_page_fault
vmlinux 0xffffffff8211a925
Build id: 3.10.0-862.el7.x86_64
0x35
23
page_fault
vmlinux 0xffffffff82116768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [rtkit-daemon:761]
Modules linked in: tcp_lp rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmhgfs(OE) vmw_vsock_vmci_transport vsock bnep sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd btusb btrtl vmw_balloon btbcm btintel bluetooth pcspkr joydev rfkill sg nfit vmw_vmci shpchp i2c_piix4 libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul ata_piix
crct10dif_common libata crc32c_intel drm mptspi scsi_transport_spi mptscsih e1000 serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 761 Comm: rtkit-daemon Tainted: G           OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8afcf18c8fd0 ti: ffff8afcf567c000 task.ti: ffff8afcf567c000
RIP: 0010:[<ffffffffc020c3be>]  [<ffffffffc020c3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8afcf567f8e0  EFLAGS: 00000246
RAX: ffffabb700700000 RBX: ffff8afcf12bfd20 RCX: 0000000000000012
RDX: ffff8afbf5f96e00 RSI: ffff8afbf541b000 RDI: 000000000000000e
RBP: ffff8afcf567f8f0 R08: 0000000000000003 R09: 0000000019801fff
R10: ffff8afbf541a0d8 R11: ffff8afbf48756c0 R12: 0000000100000002
R13: ffff8afc79eaf600 R14: ffff8afcf567f854 R15: ffffffff81d212cb
FS:  00007ff87815d700(0000) GS:ffff8afcf9600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055cd9bb08310 CR3: 00000000b7af0000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc025471e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02692e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff81e9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff81ea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff81d18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff81d18b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff81d1c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff81d1c7c4>] blk_finish_plug+0x14/0x40
[<ffffffff81ba1241>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff81ba18e1>] ra_submit+0x21/0x30
[<ffffffff81b96265>] filemap_fault+0x105/0x490
[<ffffffff81fd005a>] ? sock_def_readable+0x3a/0x70
[<ffffffff82112062>] ? down_read+0x12/0x40
[<ffffffffc045985f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffff81bc05fa>] __do_fault.isra.58+0x8a/0x100
[<ffffffff81c30830>] ? poll_select_copy_remaining+0x150/0x150
[<ffffffff81bc0b5c>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffff81bc52dc>] handle_pte_fault+0x2dc/0xc30
[<ffffffff81bc747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8211a597>] __do_page_fault+0x197/0x4f0
[<ffffffff8211a925>] do_page_fault+0x35/0x90
[<ffffffff82116768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 a3 21 
Complete report #892261
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00003be
Build id: 3.10.0-693.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc0076729
Build id: 3.10.0-693.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffc00812b0
Build id: 3.10.0-693.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff8118f3ee
Build id: 3.10.0-693.el7.x86_64
0x1de
11
ra_submit
vmlinux 0xffffffff8118fa71
Build id: 3.10.0-693.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff81184e45
Build id: 3.10.0-693.el7.x86_64
0x105
13
? check_preempt_wakeup
vmlinux 0xffffffff810ce8d8
Build id: 3.10.0-693.el7.x86_64
0x148
14
xfs_filemap_fault
xfs 0xffffffffc025390f
Build id: 3.10.0-693.el7.x86_64
0x5f
15
__do_fault
vmlinux 0xffffffff811ad162
Build id: 3.10.0-693.el7.x86_64
0x52
16
do_read_fault.isra.44
vmlinux 0xffffffff811ad60b
Build id: 3.10.0-693.el7.x86_64
0x4b
17
handle_mm_fault
vmlinux 0xffffffff811b1f11
Build id: 3.10.0-693.el7.x86_64
0x691
18
__do_page_fault
vmlinux 0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
0x154
19
do_page_fault
vmlinux 0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
0x35
20
page_fault
vmlinux 0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [containerd:1141]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi iosf_mbi snd_ac97_codec crc32_pclmul ghash_clmulni_intel aesni_intel ac97_bus snd_seq lrw snd_seq_device gf128mul snd_pcm glue_helper ablk_helper cryptd snd_timer snd soundcore ppdev sg vmw_balloon pcspkr nfit joydev libnvdimm parport_pc parport vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace
sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi crct10dif_pclmul crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops ttm drm serio_raw e1000 i2c_core mptspi scsi_transport_spi ata_piix mptscsih libata mptbase
CPU: 0 PID: 1141 Comm: containerd Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8800afc69fa0 ti: ffff8800b2b0c000 task.ti: ffff8800b2b0c000
RIP: 0010:[<ffffffffc00003be>]  [<ffffffffc00003be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8800b2b0f9c8  EFLAGS: 00000246
RAX: ffffc900006c0000 RBX: ffff8800844aee10 RCX: 000000000000000d
RDX: ffff8801321f8a00 RSI: ffff880036218000 RDI: 000000000000000e
RBP: ffff8800b2b0f9d8 R08: 0000000000000003 R09: ffff880132b930d8
R10: ffff8801360539c0 R11: ffff8801360539c0 R12: ffff880035dd8a80
R13: ffff8800b2b0f93c R14: ffffffff812ff13b R15: ffff8800b2b0f980
FS:  00007f9aeceb4700(0000) GS:ffff880139600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000556b5f93f392 CR3: 00000001362f5000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880036218d30 0000000000000048 ffff8800b2b0fa80 ffffffffc0076729
ffff880036218008 00000000b2b0fa40 000000000000000d ffff8801360539c0
0000000000000034 ffff880036218188 000000600000000d ffff880035442d10
Call Trace:
[<ffffffffc0076729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00812b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffff8118f3ee>] __do_page_cache_readahead+0x1de/0x250
[<ffffffff8118fa71>] ra_submit+0x21/0x30
[<ffffffff81184e45>] filemap_fault+0x105/0x410
[<ffffffff810ce8d8>] ? check_preempt_wakeup+0x148/0x250
[<ffffffffc025390f>] xfs_filemap_fault+0x5f/0xf0 [xfs]
[<ffffffff811ad162>] __do_fault+0x52/0xe0
[<ffffffff811ad60b>] do_read_fault.isra.44+0x4b/0x130
[<ffffffff811b1f11>] handle_mm_fault+0x691/0xfa0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 00 
Complete report #663297
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03953be
Build id: 3.10.0-1160.49.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03b771e
Build id: 3.10.0-1160.49.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03b02e0
Build id: 3.10.0-1160.49.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffaaae4860
Build id: 3.10.0-1160.49.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffaaaedf8c
Build id: 3.10.0-1160.49.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffaa953f39
Build id: 3.10.0-1160.49.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffaa953ffa
Build id: 3.10.0-1160.49.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffaa9578d8
Build id: 3.10.0-1160.49.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffaa957d44
Build id: 3.10.0-1160.49.1.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffaa7cbe31
Build id: 3.10.0-1160.49.1.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffaa7cc451
Build id: 3.10.0-1160.49.1.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffaa7c0e35
Build id: 3.10.0-1160.49.1.el7.x86_64
0x105
13
? account_entity_dequeue
vmlinux 0xffffffffaa6e0f5e
Build id: 3.10.0-1160.49.1.el7.x86_64
0xae
14
__xfs_filemap_fault
xfs 0xffffffffc05dd91e
Build id: 3.10.0-1160.49.1.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc05ddb1c
Build id: 3.10.0-1160.49.1.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffffaa7ee7aa
Build id: 3.10.0-1160.49.1.el7.x86_64
0x8a
17
do_read_fault.isra.63
vmlinux 0xffffffffaa7eed5c
Build id: 3.10.0-1160.49.1.el7.x86_64
0x4c
18
? schedule_hrtimeout_range_clock
vmlinux 0xffffffffaad885bd
Build id: 3.10.0-1160.49.1.el7.x86_64
0x12d
19
handle_mm_fault
vmlinux 0xffffffffaa7f65a0
Build id: 3.10.0-1160.49.1.el7.x86_64
0xa20
20
? ep_scan_ready_list.isra.7
vmlinux 0xffffffffaa89d509
Build id: 3.10.0-1160.49.1.el7.x86_64
0x1b9
21
__do_page_fault
vmlinux 0xffffffffaad90653
Build id: 3.10.0-1160.49.1.el7.x86_64
0x213
22
do_page_fault
vmlinux 0xffffffffaad90975
Build id: 3.10.0-1160.49.1.el7.x86_64
0x35
23
? error_swapgs
vmlinux 0xffffffffaad8cac9
Build id: 3.10.0-1160.49.1.el7.x86_64
0xaa
24
page_fault
vmlinux 0xffffffffaad8c778
Build id: 3.10.0-1160.49.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 121s! [systemd-journal:25978]
Modules linked in: i2c_dev rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd grace fscache cmac arc4 md4 tcp_lp nls_utf8 cifs ccm dns_resolver xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_generic
cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crc32c_intel mptspi serio_raw scsi_transport_spi vmxnet3 mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 25978 Comm: systemd-journal Kdump: loaded Tainted: G             L ------------   3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88f8c4f48000 ti: ffff88f8f2478000 task.ti: ffff88f8f2478000
RIP: 0010:[<ffffffffc03953be>]  [<ffffffffc03953be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88f8f247b918  EFLAGS: 00000246
RAX: ffffa5b601bc0000 RBX: ffff88fac6150c40 RCX: 0000000000000017
RDX: ffff88f9a6a45400 RSI: ffff88faee956000 RDI: 000000000000000e
RBP: ffff88f8f247b928 R08: 0000000000000003 R09: 0000000326144fff
R10: ffff88f9a6b2d800 R11: ffff88f8e0a88a80 R12: ffff88f8f2986d80
R13: ffff88f8f247b88c R14: ffffffffaa95c8fb R15: ffff88f8f247b8d0
FS:  00007fd049012880(0000) GS:ffff88faffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055abab5af820 CR3: 000000013cb3c000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc03b771e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03b02e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffaaae4860>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffaaaedf8c>] scsi_request_fn+0x4ac/0x680
[<ffffffffaa953f39>] __blk_run_queue+0x39/0x50
[<ffffffffaa953ffa>] queue_unplugged+0x2a/0xa0
[<ffffffffaa9578d8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffaa957d44>] blk_finish_plug+0x14/0x40
[<ffffffffaa7cbe31>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffaa7cc451>] ra_submit+0x21/0x30
[<ffffffffaa7c0e35>] filemap_fault+0x105/0x420
[<ffffffffaa6e0f5e>] ? account_entity_dequeue+0xae/0xd0
[<ffffffffc05dd91e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc05ddb1c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffaa7ee7aa>] __do_fault.isra.61+0x8a/0x100
[<ffffffffaa7eed5c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffffaad885bd>] ? schedule_hrtimeout_range_clock+0x12d/0x150
[<ffffffffaa7f65a0>] handle_mm_fault+0xa20/0xfb0
[<ffffffffaa89d509>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[<ffffffffaad90653>] __do_page_fault+0x213/0x500
[<ffffffffaad90975>] do_page_fault+0x35/0x90
[<ffffffffaad8cac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffaad8c778>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 33 3a 
Complete report #375704
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff93f883c5
Build id: 3.10.0-1127.8.2.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0248632
Build id: 3.10.0-1127.8.2.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff93ceaff0
Build id: 3.10.0-1127.8.2.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff93ce28e0
Build id: 3.10.0-1127.8.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff93cec00c
Build id: 3.10.0-1127.8.2.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff93b52749
Build id: 3.10.0-1127.8.2.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff93b5280a
Build id: 3.10.0-1127.8.2.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff93b56048
Build id: 3.10.0-1127.8.2.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff93b564b4
Build id: 3.10.0-1127.8.2.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff939cb691
Build id: 3.10.0-1127.8.2.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff939cbce1
Build id: 3.10.0-1127.8.2.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff939c0e25
Build id: 3.10.0-1127.8.2.el7.x86_64
0x105
13
? dequeue_entity
vmlinux 0xffffffff938e539c
Build id: 3.10.0-1127.8.2.el7.x86_64
0x11c
14
__xfs_filemap_fault
xfs 0xffffffffc04a2a4e
Build id: 3.10.0-1127.8.2.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc04a2c4c
Build id: 3.10.0-1127.8.2.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffff939edf4a
Build id: 3.10.0-1127.8.2.el7.x86_64
0x8a
17
? __schedule
vmlinux 0xffffffff93f85942
Build id: 3.10.0-1127.8.2.el7.x86_64
0x402
18
do_read_fault.isra.63
vmlinux 0xffffffff939ee4fc
Build id: 3.10.0-1127.8.2.el7.x86_64
0x4c
19
? schedule_hrtimeout_range_clock
vmlinux 0xffffffff93f8502d
Build id: 3.10.0-1127.8.2.el7.x86_64
0x12d
20
handle_mm_fault
vmlinux 0xffffffff939f5d60
Build id: 3.10.0-1127.8.2.el7.x86_64
0xa20
21
? ep_scan_ready_list.isra.7
vmlinux 0xffffffff93a9c059
Build id: 3.10.0-1127.8.2.el7.x86_64
0x1b9
22
__do_page_fault
vmlinux 0xffffffff93f8d653
Build id: 3.10.0-1127.8.2.el7.x86_64
0x213
23
do_page_fault
vmlinux 0xffffffff93f8d975
Build id: 3.10.0-1127.8.2.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffff93f89778
Build id: 3.10.0-1127.8.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 32s! [systemd:1]
Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc fscache tcp_diag inet_diag nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack vmw_vsock_vmci_transport vsock nf_conntrack iptable_filter sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd pcspkr joydev vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci sd_mod crc_t10dif ata_piix libahci drm crct10dif_generic nfit libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel vmw_pvscsi serio_raw vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1 Comm: systemd Tainted: G             L ------------   3.10.0-1127.8.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/11/2017
task: ffff9282fce68000 ti: ffff9282fcee0000 task.ti: ffff9282fcee0000
RIP: 0010:[<ffffffff93f883c5>]  [<ffffffff93f883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff9282fcee3980  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff93bb1433 RCX: ffff9282fcaf1060
RDX: ffffac32c0e80000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9282fcee3980 R08: 0000000000000000 R09: ffff9283b4d18400
R10: ffff9283b3feb400 R11: ffff9281b674d000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff92837115a000 R15: ffffffff93ce9ca9
FS:  00007f470d095940(0000) GS:ffff9283bfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000557a24dde650 CR3: 0000000236e90000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0248632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff93ceaff0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff93ce28e0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff93cec00c>] scsi_request_fn+0x4ac/0x680
[<ffffffff93b52749>] __blk_run_queue+0x39/0x50
[<ffffffff93b5280a>] queue_unplugged+0x2a/0xa0
[<ffffffff93b56048>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff93b564b4>] blk_finish_plug+0x14/0x40
[<ffffffff939cb691>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff939cbce1>] ra_submit+0x21/0x30
[<ffffffff939c0e25>] filemap_fault+0x105/0x420
[<ffffffff938e539c>] ? dequeue_entity+0x11c/0x5c0
[<ffffffffc04a2a4e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc04a2c4c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff939edf4a>] __do_fault.isra.61+0x8a/0x100
[<ffffffff93f85942>] ? __schedule+0x402/0x840
[<ffffffff939ee4fc>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffff93f8502d>] ? schedule_hrtimeout_range_clock+0x12d/0x150
[<ffffffff939f5d60>] handle_mm_fault+0xa20/0xfb0
[<ffffffff93a9c059>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[<ffffffff93f8d653>] __do_page_fault+0x213/0x500
[<ffffffff93f8d975>] do_page_fault+0x35/0x90
[<ffffffff93f89778>] page_fault+0x28/0x30
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #372638
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02163be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc025471e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc020d2e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff87e9b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff87ea45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff87d18ad9
Build id: 3.10.0-862.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff87d18b9a
Build id: 3.10.0-862.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff87d1c358
Build id: 3.10.0-862.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff87d1c7c4
Build id: 3.10.0-862.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff87ba1241
Build id: 3.10.0-862.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff87ba18e1
Build id: 3.10.0-862.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff87b96265
Build id: 3.10.0-862.el7.x86_64
0x105
13
? sock_def_readable
vmlinux 0xffffffff87fd005a
Build id: 3.10.0-862.el7.x86_64
0x3a
14
? down_read
vmlinux 0xffffffff88112062
Build id: 3.10.0-862.el7.x86_64
0x12
15
xfs_filemap_fault
xfs 0xffffffffc046085f
Build id: 3.10.0-862.el7.x86_64
0x5f
16
__do_fault.isra.58
vmlinux 0xffffffff87bc05fa
Build id: 3.10.0-862.el7.x86_64
0x8a
17
? poll_select_copy_remaining
vmlinux 0xffffffff87c30830
Build id: 3.10.0-862.el7.x86_64
0x150
18
do_read_fault.isra.60
vmlinux 0xffffffff87bc0b5c
Build id: 3.10.0-862.el7.x86_64
0x4c
19
handle_pte_fault
vmlinux 0xffffffff87bc52dc
Build id: 3.10.0-862.el7.x86_64
0x2dc
20
handle_mm_fault
vmlinux 0xffffffff87bc747d
Build id: 3.10.0-862.el7.x86_64
0x39d
21
__do_page_fault
vmlinux 0xffffffff8811a597
Build id: 3.10.0-862.el7.x86_64
0x197
22
do_page_fault
vmlinux 0xffffffff8811a925
Build id: 3.10.0-862.el7.x86_64
0x35
23
page_fault
vmlinux 0xffffffff88116768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 36s! [rtkit-daemon:957]
Modules linked in: tcp_lp fuse vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper snd_seq ablk_helper cryptd snd_seq_device snd_pcm snd_timer snd vmw_balloon soundcore nfit ppdev joydev pcspkr sg libnvdimm parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm ata_piix mptspi crc32c_intel libata scsi_transport_spi e1000 mptscsih serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 957 Comm: rtkit-daemon Kdump: loaded Tainted: G        W    L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9917f3548fd0 ti: ffff9917f6c94000 task.ti: ffff9917f6c94000
RIP: 0010:[<ffffffffc02163be>]  [<ffffffffc02163be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9917f6c978e0  EFLAGS: 00000246
RAX: ffffb38500440000 RBX: ffff99178fb2bee0 RCX: 0000000000000009
RDX: ffff9917f4dd3200 RSI: ffff9917b5928000 RDI: 000000000000000e
RBP: ffff9917f6c978f0 R08: 0000000000000003 R09: 00000000053abfff
R10: ffff9917f9a450d8 R11: ffff9917b5508e00 R12: 0000000100000008
R13: ffff9917dd7e6280 R14: ffff9917f6c97854 R15: ffffffff87d212cb
FS:  00007f49b0da0700(0000) GS:ffff9917fb680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000560a71c49310 CR3: 0000000073752000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffc025471e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc020d2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff87e9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff87ea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff87d18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff87d18b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff87d1c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff87d1c7c4>] blk_finish_plug+0x14/0x40
[<ffffffff87ba1241>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff87ba18e1>] ra_submit+0x21/0x30
[<ffffffff87b96265>] filemap_fault+0x105/0x490
[<ffffffff87fd005a>] ? sock_def_readable+0x3a/0x70
[<ffffffff88112062>] ? down_read+0x12/0x40
[<ffffffffc046085f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffff87bc05fa>] __do_fault.isra.58+0x8a/0x100
[<ffffffff87c30830>] ? poll_select_copy_remaining+0x150/0x150
[<ffffffff87bc0b5c>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffff87bc52dc>] handle_pte_fault+0x2dc/0xc30
[<ffffffff87bc747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8811a597>] __do_page_fault+0x197/0x4f0
[<ffffffff8811a925>] do_page_fault+0x35/0x90
[<ffffffff88116768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 43 22 
Complete report #314939
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04593be
Build id: 3.10.0-1160.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc047b71e
Build id: 3.10.0-1160.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04742e0
Build id: 3.10.0-1160.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8fce3980
Build id: 3.10.0-1160.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8fced0ac
Build id: 3.10.0-1160.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff8fb534d9
Build id: 3.10.0-1160.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8fb5359a
Build id: 3.10.0-1160.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8fb56dd8
Build id: 3.10.0-1160.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8fb57244
Build id: 3.10.0-1160.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffff8f9cb261
Build id: 3.10.0-1160.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffff8f9cb881
Build id: 3.10.0-1160.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffff8f9c0975
Build id: 3.10.0-1160.el7.x86_64
0x105
13
? dequeue_entity
vmlinux 0xffffffff8f8e48bc
Build id: 3.10.0-1160.el7.x86_64
0x11c
14
__xfs_filemap_fault
xfs 0xffffffffc070391e
Build id: 3.10.0-1160.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc0703b1c
Build id: 3.10.0-1160.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffff8f9ede3a
Build id: 3.10.0-1160.el7.x86_64
0x8a
17
? __schedule
vmlinux 0xffffffff8ff86c8f
Build id: 3.10.0-1160.el7.x86_64
0x3af
18
do_read_fault.isra.63
vmlinux 0xffffffff8f9ee3ec
Build id: 3.10.0-1160.el7.x86_64
0x4c
19
? schedule_hrtimeout_range_clock
vmlinux 0xffffffff8ff863cd
Build id: 3.10.0-1160.el7.x86_64
0x12d
20
handle_mm_fault
vmlinux 0xffffffff8f9f5c30
Build id: 3.10.0-1160.el7.x86_64
0xa20
21
? ep_scan_ready_list.isra.7
vmlinux 0xffffffff8fa9cb59
Build id: 3.10.0-1160.el7.x86_64
0x1b9
22
__do_page_fault
vmlinux 0xffffffff8ff8e653
Build id: 3.10.0-1160.el7.x86_64
0x213
23
do_page_fault
vmlinux 0xffffffff8ff8e975
Build id: 3.10.0-1160.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffff8ff8a778
Build id: 3.10.0-1160.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 49s! [systemd-journal:66649]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel ac97_bus aesni_intel snd_seq snd_seq_device snd_pcm lrw gf128mul glue_helper ablk_helper cryptd pcspkr snd_timer vmw_balloon snd soundcore joydev sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm nfit libnvdimm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata mptspi serio_raw scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 66649 Comm: systemd-journal Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8bf58f8f2100 ti: ffff8bf5a5034000 task.ti: ffff8bf5a5034000
RIP: 0010:[<ffffffffc04593be>]  [<ffffffffc04593be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8bf5a5037918  EFLAGS: 00000246
RAX: ffffa6e500d40000 RBX: ffff8bf5a1966040 RCX: 000000000000001f
RDX: ffff8bf5b5e03a00 RSI: ffff8bf5b5df2000 RDI: 000000000000000e
RBP: ffff8bf5a5037928 R08: 0000000000000003 R09: 00000001418effff
R10: ffff8bf7b1c61000 R11: ffff8bf63b0ece00 R12: ffff8bf69446ee80
R13: ffff8bf5a503788c R14: ffffffff8fb5bdfb R15: ffff8bf5a50378d0
FS:  00007f7656114880(0000) GS:ffff8bf7b5600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056187e7a4820 CR3: 000000007b8ba000 CR4: 00000000007607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffc047b71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04742e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8fce3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8fced0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffff8fb534d9>] __blk_run_queue+0x39/0x50
[<ffffffff8fb5359a>] queue_unplugged+0x2a/0xa0
[<ffffffff8fb56dd8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8fb57244>] blk_finish_plug+0x14/0x40
[<ffffffff8f9cb261>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffff8f9cb881>] ra_submit+0x21/0x30
[<ffffffff8f9c0975>] filemap_fault+0x105/0x420
[<ffffffff8f8e48bc>] ? dequeue_entity+0x11c/0x5c0
[<ffffffffc070391e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc0703b1c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff8f9ede3a>] __do_fault.isra.61+0x8a/0x100
[<ffffffff8ff86c8f>] ? __schedule+0x3af/0x860
[<ffffffff8f9ee3ec>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffff8ff863cd>] ? schedule_hrtimeout_range_clock+0x12d/0x150
[<ffffffff8f9f5c30>] handle_mm_fault+0xa20/0xfb0
[<ffffffff8fa9cb59>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[<ffffffff8ff8e653>] __do_page_fault+0x213/0x500
[<ffffffff8ff8e975>] do_page_fault+0x35/0x90
[<ffffffff8ff8a778>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 73 46 
Complete report #143034
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02613be
Build id: 3.10.0-1127.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc028371e
Build id: 3.10.0-1127.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc029b2e0
Build id: 3.10.0-1127.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffafae2850
Build id: 3.10.0-1127.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffafaebefc
Build id: 3.10.0-1127.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffaf9526f9
Build id: 3.10.0-1127.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffaf9527ba
Build id: 3.10.0-1127.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffaf955ff8
Build id: 3.10.0-1127.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffaf956464
Build id: 3.10.0-1127.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffaf7cb631
Build id: 3.10.0-1127.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffaf7cbc81
Build id: 3.10.0-1127.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffaf7c0dc5
Build id: 3.10.0-1127.el7.x86_64
0x105
13
? _cond_resched
vmlinux 0xffffffffafd8602a
Build id: 3.10.0-1127.el7.x86_64
0x3a
14
__xfs_filemap_fault
xfs 0xffffffffc050aa4e
Build id: 3.10.0-1127.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc050ac4c
Build id: 3.10.0-1127.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffffaf7edeea
Build id: 3.10.0-1127.el7.x86_64
0x8a
17
do_read_fault.isra.63
vmlinux 0xffffffffaf7ee49c
Build id: 3.10.0-1127.el7.x86_64
0x4c
18
handle_mm_fault
vmlinux 0xffffffffaf7f5d00
Build id: 3.10.0-1127.el7.x86_64
0xa20
19
? ep_scan_ready_list.isra.7
vmlinux 0xffffffffaf89bff9
Build id: 3.10.0-1127.el7.x86_64
0x1b9
20
__do_page_fault
vmlinux 0xffffffffafd8d653
Build id: 3.10.0-1127.el7.x86_64
0x213
21
do_page_fault
vmlinux 0xffffffffafd8d975
Build id: 3.10.0-1127.el7.x86_64
0x35
22
? error_swapgs
vmlinux 0xffffffffafd89ac9
Build id: 3.10.0-1127.el7.x86_64
0xaa
23
page_fault
vmlinux 0xffffffffafd89778
Build id: 3.10.0-1127.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 30s! [systemd-journal:515]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables vmw_vsock_vmci_transport vsock ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sunrpc snd_seq_midi snd_seq_midi_event coretemp iosf_mbi ppdev crc32_pclmul snd_ens1371 ghash_clmulni_intel aesni_intel snd_rawmidi snd_ac97_codec lrw gf128mul ac97_bus glue_helper snd_seq pcspkr ablk_helper snd_seq_device snd_pcm snd_timer snd cryptd soundcore joydev sg vmw_balloon parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper
crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix nfit libnvdimm drm libata serio_raw e1000 mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 515 Comm: systemd-journal Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8da3367662a0 ti: ffff8da33ba3c000 task.ti: ffff8da33ba3c000
RIP: 0010:[<ffffffffc02613be>]  [<ffffffffc02613be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8da33ba3f918  EFLAGS: 00000246
RAX: ffffb21ec06c0000 RBX: ffff8da2d1bf34d0 RCX: 0000000000000007
RDX: ffff8da2b54fd200 RSI: ffff8da2b59a8000 RDI: 000000000000000e
RBP: ffff8da33ba3f928 R08: 0000000000000003 R09: 000000006d9bbfff
R10: ffff8da33beac400 R11: ffff8da296392d80 R12: ffff8da3136a1e00
R13: ffff8da33ba3f88c R14: ffffffffaf95b01b R15: ffff8da33ba3f8d0
FS:  00007f8c1a354880(0000) GS:ffff8da33a480000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f87d1cf8000 CR3: 000000003598c000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc028371e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc029b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffafae2850>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffafaebefc>] scsi_request_fn+0x4ac/0x660
[<ffffffffaf9526f9>] __blk_run_queue+0x39/0x50
[<ffffffffaf9527ba>] queue_unplugged+0x2a/0xa0
[<ffffffffaf955ff8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffaf956464>] blk_finish_plug+0x14/0x40
[<ffffffffaf7cb631>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffaf7cbc81>] ra_submit+0x21/0x30
[<ffffffffaf7c0dc5>] filemap_fault+0x105/0x420
[<ffffffffafd8602a>] ? _cond_resched+0x3a/0x50
[<ffffffffc050aa4e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc050ac4c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffaf7edeea>] __do_fault.isra.61+0x8a/0x100
[<ffffffffaf7ee49c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffffaf7f5d00>] handle_mm_fault+0xa20/0xfb0
[<ffffffffaf89bff9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[<ffffffffafd8d653>] __do_page_fault+0x213/0x500
[<ffffffffafd8d975>] do_page_fault+0x35/0x90
[<ffffffffafd89ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffafd89778>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 f3 26 
Complete report #10617
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02023be
Build id: 3.10.0-862.3.3.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc021d71e
Build id: 3.10.0-862.3.3.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc026c2e0
Build id: 3.10.0-862.3.3.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa689b4f0
Build id: 3.10.0-862.3.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa68a4a5c
Build id: 3.10.0-862.3.3.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffa6718f19
Build id: 3.10.0-862.3.3.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffa6718fda
Build id: 3.10.0-862.3.3.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffa671c798
Build id: 3.10.0-862.3.3.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffa671cc04
Build id: 3.10.0-862.3.3.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffa65a1581
Build id: 3.10.0-862.3.3.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffa65a1c21
Build id: 3.10.0-862.3.3.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffa65965a5
Build id: 3.10.0-862.3.3.el7.x86_64
0x105
13
? sock_def_readable
vmlinux 0xffffffffa69d105a
Build id: 3.10.0-862.3.3.el7.x86_64
0x3a
14
? down_read
vmlinux 0xffffffffa6b13092
Build id: 3.10.0-862.3.3.el7.x86_64
0x12
15
xfs_filemap_fault
xfs 0xffffffffc02fc85f
Build id: 3.10.0-862.3.3.el7.x86_64
0x5f
16
__do_fault.isra.58
vmlinux 0xffffffffa65c093a
Build id: 3.10.0-862.3.3.el7.x86_64
0x8a
17
? poll_select_copy_remaining
vmlinux 0xffffffffa6630b70
Build id: 3.10.0-862.3.3.el7.x86_64
0x150
18
do_read_fault.isra.60
vmlinux 0xffffffffa65c0e9c
Build id: 3.10.0-862.3.3.el7.x86_64
0x4c
19
? path_lookupat
vmlinux 0xffffffffa6629e72
Build id: 3.10.0-862.3.3.el7.x86_64
0x122
20
handle_pte_fault
vmlinux 0xffffffffa65c561c
Build id: 3.10.0-862.3.3.el7.x86_64
0x2dc
21
handle_mm_fault
vmlinux 0xffffffffa65c77bd
Build id: 3.10.0-862.3.3.el7.x86_64
0x39d
22
__do_page_fault
vmlinux 0xffffffffa6b1b5c7
Build id: 3.10.0-862.3.3.el7.x86_64
0x197
23
do_page_fault
vmlinux 0xffffffffa6b1b955
Build id: 3.10.0-862.3.3.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffffa6b17788
Build id: 3.10.0-862.3.3.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 76s! [rtkit-daemon:5444]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr sg nfit parport_pc parport libnvdimm vmw_vmci i2c_piix4 shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod
crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptspi e1000 scsi_transport_spi ahci mptscsih libahci mptbase vmwgfx ata_generic pata_acpi drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 5444 Comm: rtkit-daemon Kdump: loaded Tainted: G             L ------------   3.10.0-862.3.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
task: ffff8a8e76b30fd0 ti: ffff8a8e76b3c000 task.ti: ffff8a8e76b3c000
RIP: 0010:[<ffffffffc02023be>]  [<ffffffffc02023be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8a8e76b3f8e0  EFLAGS: 00000246
RAX: ffffb17680400000 RBX: ffff8a8e565bdf20 RCX: 0000000000000018
RDX: ffff8a8e35c1ea00 RSI: ffff8a8e35f6c000 RDI: 000000000000000e
RBP: ffff8a8e76b3f8f0 R08: 0000000000000002 R09: 0000000069f49fff
R10: ffff8a8e35f6d0d8 R11: ffff8a8e7a2768c0 R12: 0000000100000002
R13: ffff8a8e773a3d80 R14: ffff8a8e76b3f854 R15: ffffffffa672170b
FS:  00007f0c0ddc2700(0000) GS:ffff8a8e7fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055f79f4ce310 CR3: 000000007a3cc000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc021d71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc026c2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa689b4f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa68a4a5c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa6718f19>] __blk_run_queue+0x39/0x50
[<ffffffffa6718fda>] queue_unplugged+0x2a/0xa0
[<ffffffffa671c798>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa671cc04>] blk_finish_plug+0x14/0x40
[<ffffffffa65a1581>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffa65a1c21>] ra_submit+0x21/0x30
[<ffffffffa65965a5>] filemap_fault+0x105/0x490
[<ffffffffa69d105a>] ? sock_def_readable+0x3a/0x70
[<ffffffffa6b13092>] ? down_read+0x12/0x40
[<ffffffffc02fc85f>] xfs_filemap_fault+0x5f/0xe0 [xfs]
[<ffffffffa65c093a>] __do_fault.isra.58+0x8a/0x100
[<ffffffffa6630b70>] ? poll_select_copy_remaining+0x150/0x150
[<ffffffffa65c0e9c>] do_read_fault.isra.60+0x4c/0x1a0
[<ffffffffa6629e72>] ? path_lookupat+0x122/0x8b0
[<ffffffffa65c561c>] handle_pte_fault+0x2dc/0xc30
[<ffffffffa65c77bd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa6b1b5c7>] __do_page_fault+0x197/0x4f0
[<ffffffffa6b1b955>] do_page_fault+0x35/0x90
[<ffffffffa6b17788>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 21 
Complete report #8158
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc026f3be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc029171e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc032f2e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa26e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa26ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffa25527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffa255286a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffa25560a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffa2556514
Build id: 3.10.0-1127.19.1.el7.x86_64
0x14
10
__do_page_cache_readahead
vmlinux 0xffffffffa23cb6e1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1e1
11
ra_submit
vmlinux 0xffffffffa23cbd01
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
12
filemap_fault
vmlinux 0xffffffffa23c0e75
Build id: 3.10.0-1127.19.1.el7.x86_64
0x105
13
? dequeue_entity
vmlinux 0xffffffffa22e539c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x11c
14
__xfs_filemap_fault
xfs 0xffffffffc052098e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7e
15
xfs_filemap_fault
xfs 0xffffffffc0520b8c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2c
16
__do_fault.isra.61
vmlinux 0xffffffffa23edf6a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x8a
17
? __schedule
vmlinux 0xffffffffa2985942
Build id: 3.10.0-1127.19.1.el7.x86_64
0x402
18
do_read_fault.isra.63
vmlinux 0xffffffffa23ee51c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4c
19
? schedule_hrtimeout_range_clock
vmlinux 0xffffffffa298502d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x12d
20
handle_mm_fault
vmlinux 0xffffffffa23f5d80
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa20
21
? ep_scan_ready_list.isra.7
vmlinux 0xffffffffa249c029
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1b9
22
__do_page_fault
vmlinux 0xffffffffa298d653
Build id: 3.10.0-1127.19.1.el7.x86_64
0x213
23
do_page_fault
vmlinux 0xffffffffa298d975
Build id: 3.10.0-1127.19.1.el7.x86_64
0x35
24
page_fault
vmlinux 0xffffffffa2989778
Build id: 3.10.0-1127.19.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 45s! [systemd-journal:4053]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_ens1371 snd_rawmidi lrw snd_ac97_codec gf128mul glue_helper ablk_helper cryptd ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd ppdev soundcore btusb btrtl btbcm btintel bluetooth vmw_balloon rfkill sg pcspkr i2c_piix4 vmw_vmci joydev parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic
pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm nfit ata_piix libnvdimm mptspi libata serio_raw scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 4053 Comm: systemd-journal Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff94276336d230 ti: ffff94277957c000 task.ti: ffff94277957c000
RIP: 0010:[<ffffffffc026f3be>]  [<ffffffffc026f3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff94277957f918  EFLAGS: 00000246
RAX: ffffb46e40200000 RBX: ffff9427577ee000 RCX: 0000000000000012
RDX: ffff94277559ee00 RSI: ffff942775530000 RDI: 000000000000000e
RBP: ffff94277957f928 R08: 0000000000000002 R09: 0000000031aedfff
R10: ffff942775542c00 R11: ffff942777b072c0 R12: ffff942778452880
R13: ffff94277957f88c R14: ffffffffa255b0cb R15: ffff94277957f8d0
FS:  00007f5dece06880(0000) GS:ffff94277c600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555ca506f820 CR3: 0000000022a98000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc029171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc032f2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa26e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa26ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffffa25527a9>] __blk_run_queue+0x39/0x50
[<ffffffffa255286a>] queue_unplugged+0x2a/0xa0
[<ffffffffa25560a8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa2556514>] blk_finish_plug+0x14/0x40
[<ffffffffa23cb6e1>] __do_page_cache_readahead+0x1e1/0x260
[<ffffffffa23cbd01>] ra_submit+0x21/0x30
[<ffffffffa23c0e75>] filemap_fault+0x105/0x420
[<ffffffffa22e539c>] ? dequeue_entity+0x11c/0x5c0
[<ffffffffc052098e>] __xfs_filemap_fault+0x7e/0x1d0 [xfs]
[<ffffffffc0520b8c>] xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffffa23edf6a>] __do_fault.isra.61+0x8a/0x100
[<ffffffffa2985942>] ? __schedule+0x402/0x840
[<ffffffffa23ee51c>] do_read_fault.isra.63+0x4c/0x1b0
[<ffffffffa298502d>] ? schedule_hrtimeout_range_clock+0x12d/0x150
[<ffffffffa23f5d80>] handle_mm_fault+0xa20/0xfb0
[<ffffffffa249c029>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[<ffffffffa298d653>] __do_page_fault+0x213/0x500
[<ffffffffa298d975>] do_page_fault+0x35/0x90
[<ffffffffa2989778>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 27