Info

Permalink
Function
mpt_put_msg_frame
First occurrence
2020-09-23
Last occurrence
2025-09-25
State
NEW
Components
kernel
Quality
-51

Statistics

Operating system Count
CentOS 7 2,925
Architecture Count
x86_64 2,925
Related packages Count
kernel 2,925
 0:3.10.0-1160.15.2.el7 851
 0:3.10.0-693.5.2.el7 781
 0:3.10.0-1160.53.1.el7 400
 0:3.10.0-862.el7 183
 0:3.10.0-1127.19.1.el7 140
 0:3.10.0-957.el7 66
 0:3.10.0-1160.25.1.el7 49
 0:3.10.0-1160.108.1.el7 45
 0:3.10.0-862.2.3.el7 44
 0:3.10.0-1160.118.1.el7 43
 0:3.10.0-1160.6.1.el7 37
 0:3.10.0-693.el7 37
 0:3.10.0-1160.49.1.el7 32
 0:3.10.0-862.6.3.el7 29
 0:3.10.0-957.27.2.el7 18
 0:3.10.0-862.3.3.el7 16
 0:3.10.0-1160.41.1.el7 14
 0:3.10.0-1160.24.1.el7 13
 0:3.10.0-1160.el7 11
 0:3.10.0-957.12.2.el7 11
 0:3.10.0-862.14.4.el7 11
 0:3.10.0-1062.18.1.el7 9
 0:3.10.0-1160.21.1.el7 8
 0:3.10.0-957.21.3.el7 7
 0:3.10.0-1160.36.2.el7 6
 0:3.10.0-693.21.1.el7 6
 0:3.10.0-1062.el7 6
 0:3.10.0-1062.12.1.el7 5
 0:3.10.0-957.5.1.el7 5
 0:3.10.0-1127.el7 5
 0:3.10.0-1160.83.1.el7 5
 0:3.10.0-957.1.3.el7 4
 0:3.10.0-693.2.2.el7 4
 0:3.10.0-862.3.2.el7 3
 0:3.10.0-862.9.1.el7 3
 0:3.10.0-1127.18.2.el7 2
 0:3.10.0-1160.76.1.el7 2
 0:3.10.0-957.12.1.el7 2
 0:3.10.0-1160.45.1.el7 2
 0:3.10.0-1160.66.1.el7 1
 0:3.10.0-1160.102.1.el7 1
 0:3.10.0-957.10.1.el7 1
 0:3.10.0-693.17.1.el7 1
 0:3.10.0-1160.62.1.el7 1
 0:3.10.0-1160.119.1.el7 1
 0:3.10.0-1160.80.1.el7 1
 0:3.10.0-1160.92.1.el7 1
 0:3.10.0-862.11.6.el7 1
 0:3.10.0-1062.1.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1322161
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc026244e
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc024574e
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02172f0
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9c103030
Build id: 3.10.0-1160.118.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff9c10cbdc
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9bf8e21e
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff9bf687d9
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff9bf6c703
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff9bf6a537
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x157
10
submit_bio
vmlinux 0xffffffff9bf6a7e0
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc04aa341
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc04aa68f
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffff9bdd6b01
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff9be8cd40
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff9be8d90c
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x1cc
16
__writeback_inodes_wb
vmlinux 0xffffffff9be8dc17
Build id: 3.10.0-1160.118.1.el7.x86_64 		0xa7
17
wb_writeback
vmlinux 0xffffffff9be8e0f3
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9be8ebfc
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x1cc
19
process_one_work
vmlinux 0xffffffff9bcc32ef
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff9bcc4436
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff9bcc4310
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x2b0
22
kthread
vmlinux 0xffffffff9bccb621
Build id: 3.10.0-1160.118.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9bccb550
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff9c3c51f7
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffff9bccb550
Build id: 3.10.0-1160.118.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 25s! [kworker/2:0:22028]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [kworker/u8:0:17737]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_18108(E) falcon_lsm_pinned_18003(E) cts 8021q garp mrp ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache falcon_lsm_pinned_17804(E) nf_nat_tftp nf_conntrack_tftp nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter
overlay(T) tun vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi nfit libata scsi_transport_spi mptscsih libnvdimm mptbase crct10dif_pclmul crct10dif_common vmxnet3 crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 0 PID: 17737 Comm: kworker/u8:0 Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:3)
task: ffff90dcf4683180 ti: ffff90dc82084000 task.ti: ffff90dc82084000
RIP: 0010:[<ffffffffc026244e>]  [<ffffffffc026244e>] mpt_put_msg_frame+0x5e/0x90 [mptbase]
RSP: 0018:ffff90dc82087888  EFLAGS: 00000246
RAX: ffffa871c0e80000 RBX: ffff90db7b131000 RCX: 000000000000003f
RDX: ffff90dcf2785a00 RSI: ffff90dcf2743000 RDI: 000000000000000e
RBP: ffff90dc82087898 R08: 0000000000000002 R09: 000000010d6f7fff
R10: ffff90daf5cddc00 R11: ffff90dcf459d180 R12: ffff90dc7ac9d500
R13: ffff90dc820877fc R14: ffffffff9bf7158b R15: ffff90dc82087840
FS:  0000000000000000(0000) GS:ffff90dcffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000012989fc CR3: 0000000213d04000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc024574e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02172f0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9c103030>] scsi_dispatch_cmd+0xb0/0x250
[<ffffffff9c10cbdc>] scsi_request_fn+0x4ac/0x680
[<ffffffff9bf8e21e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9bf687d9>] __blk_run_queue+0x39/0x50
[<ffffffff9bf6c703>] blk_queue_bio+0x3b3/0x400
[<ffffffff9bf6a537>] generic_make_request+0x157/0x390
[<ffffffff9bf6a7e0>] submit_bio+0x70/0x150
[<ffffffffc04aa341>] xfs_submit_ioend.isra.12+0x61/0xf0 [xfs]
[<ffffffffc04aa68f>] xfs_vm_writepages+0x7f/0xb0 [xfs]
[<ffffffff9bdd6b01>] do_writepages+0x21/0x50
[<ffffffff9be8cd40>] __writeback_single_inode+0x40/0x260
[<ffffffff9be8d90c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff9be8dc17>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff9be8e0f3>] wb_writeback+0x263/0x2f0
[<ffffffff9be8ebfc>] bdi_writeback_workfn+0x1cc/0x470
[<ffffffff9bcc32ef>] process_one_work+0x17f/0x440
[<ffffffff9bcc4436>] worker_thread+0x126/0x3c0
[<ffffffff9bcc4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff9bccb621>] kthread+0xd1/0xe0
[<ffffffff9bccb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff9c3c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9bccb550>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 14 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 cc cc cc cc 48 8d 76 08 0f b7 c8 44 89 e2 48 c7
Complete report #1070228
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03513be
Build id: 3.10.0-1160.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02ce71e
Build id: 3.10.0-1160.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03ed2e0
Build id: 3.10.0-1160.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9d2e3980
Build id: 3.10.0-1160.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff9d2ed0ac
Build id: 3.10.0-1160.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9d177f8e
Build id: 3.10.0-1160.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff9d1534d9
Build id: 3.10.0-1160.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff9d1571e3
Build id: 3.10.0-1160.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff9d155167
Build id: 3.10.0-1160.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff9d155410
Build id: 3.10.0-1160.el7.x86_64 		0x70
11
? __switch_to
vmlinux 0xffffffff9ce2b59e
Build id: 3.10.0-1160.el7.x86_64 		0xce
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc0544f31
Build id: 3.10.0-1160.el7.x86_64 		0x61
13
xfs_vm_writepages
xfs 0xffffffffc054526f
Build id: 3.10.0-1160.el7.x86_64 		0x7f
14
do_writepages
vmlinux 0xffffffff9cfcade1
Build id: 3.10.0-1160.el7.x86_64 		0x21
15
__writeback_single_inode
vmlinux 0xffffffff9d07d6f0
Build id: 3.10.0-1160.el7.x86_64 		0x40
16
writeback_sb_inodes
vmlinux 0xffffffff9d07e274
Build id: 3.10.0-1160.el7.x86_64 		0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff9d07e57f
Build id: 3.10.0-1160.el7.x86_64 		0x9f
18
wb_writeback
vmlinux 0xffffffff9d07ea53
Build id: 3.10.0-1160.el7.x86_64 		0x263
19
bdi_writeback_workfn
vmlinux 0xffffffff9d07f64b
Build id: 3.10.0-1160.el7.x86_64 		0x2cb
20
process_one_work
vmlinux 0xffffffff9cebdc4f
Build id: 3.10.0-1160.el7.x86_64 		0x17f
21
worker_thread
vmlinux 0xffffffff9cebed66
Build id: 3.10.0-1160.el7.x86_64 		0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff9cebec40
Build id: 3.10.0-1160.el7.x86_64 		0x2a0
23
kthread
vmlinux 0xffffffff9cec5c21
Build id: 3.10.0-1160.el7.x86_64 		0xd1
24
? insert_kthread_work
vmlinux 0xffffffff9cec5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff9d593df7
Build id: 3.10.0-1160.el7.x86_64 		0x21
26
? insert_kthread_work
vmlinux 0xffffffff9cec5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u256:1:46261]
Modules linked in: tcp_lp rfcomm xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock bnep sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul snd_ens1371 snd_rawmidi ghash_clmulni_intel snd_ac97_codec ppdev ac97_bus snd_seq aesni_intel btusb lrw btrtl gf128mul btbcm glue_helper ablk_helper btintel snd_seq_device vmw_balloon cryptd bluetooth snd_pcm joydev pcspkr rfkill snd_timer snd sg vmw_vmci soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod
cdrom ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops pata_acpi ttm mptspi crct10dif_pclmul scsi_transport_spi crct10dif_common mptscsih ata_piix nfit e1000 crc32c_intel mptbase serio_raw drm libnvdimm libata drm_panel_orientation_quirks fuse
CPU: 2 PID: 46261 Comm: kworker/u256:1 Kdump: loaded Tainted: G        W      ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff935c37fad280 ti: ffff935d0aed4000 task.ti: ffff935d0aed4000
RIP: 0010:[<ffffffffc03513be>]  [<ffffffffc03513be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff935d0aed7888  EFLAGS: 00010246
RAX: ffffa73800980000 RBX: ffff935cb2ef2000 RCX: 000000000000001d
RDX: ffff935d2c7b6a00 RSI: ffff935d3678e000 RDI: 000000000000000e
RBP: ffff935d0aed7898 R08: 0000000000000002 R09: 0000000037720fff
R10: ffff935c35e43400 R11: ffff935c35707b80 R12: ffff935c02a30c00
R13: ffff935d0aed77fc R14: ffffffff9d15bdfb R15: ffff935d0aed7840
FS:  0000000000000000(0000) GS:ffff935d39680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4f17a7f200 CR3: 00000000a07f8000 CR4: 0000000000360fe0
Call Trace:
[<ffffffffc02ce71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03ed2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9d2e3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9d2ed0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffff9d177f8e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9d1534d9>] __blk_run_queue+0x39/0x50
[<ffffffff9d1571e3>] blk_queue_bio+0x3b3/0x400
[<ffffffff9d155167>] generic_make_request+0x147/0x380
[<ffffffff9d155410>] submit_bio+0x70/0x150
[<ffffffff9ce2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc0544f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc054526f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9cfcade1>] do_writepages+0x21/0x50
[<ffffffff9d07d6f0>] __writeback_single_inode+0x40/0x260
[<ffffffff9d07e274>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9d07e57f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9d07ea53>] wb_writeback+0x263/0x2f0
[<ffffffff9d07f64b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9cebdc4f>] process_one_work+0x17f/0x440
[<ffffffff9cebed66>] worker_thread+0x126/0x3c0
[<ffffffff9cebec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9cec5c21>] kthread+0xd1/0xe0
[<ffffffff9cec5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff9d593df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9cec5b50>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 f3 35
Complete report #1017572
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01363be
Build id: 3.10.0-1160.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc012a71e
Build id: 3.10.0-1160.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc015b2e0
Build id: 3.10.0-1160.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c8e3980
Build id: 3.10.0-1160.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c8ed0ac
Build id: 3.10.0-1160.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8c777f8e
Build id: 3.10.0-1160.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff8c7534d9
Build id: 3.10.0-1160.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff8c7571e3
Build id: 3.10.0-1160.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff8c755167
Build id: 3.10.0-1160.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff8c755410
Build id: 3.10.0-1160.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0334f31
Build id: 3.10.0-1160.el7.x86_64 		0x61
12
xfs_do_writepage
xfs 0xffffffffc0335724
Build id: 3.10.0-1160.el7.x86_64 		0x494
13
write_cache_pages
vmlinux 0xffffffff8c5c9d8c
Build id: 3.10.0-1160.el7.x86_64 		0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc0335290
Build id: 3.10.0-1160.el7.x86_64 		0xa0
15
? set_next_entity
vmlinux 0xffffffff8c4e055c
Build id: 3.10.0-1160.el7.x86_64 		0x3c
16
? __switch_to
vmlinux 0xffffffff8c42b59e
Build id: 3.10.0-1160.el7.x86_64 		0xce
17
xfs_vm_writepages
xfs 0xffffffffc033525b
Build id: 3.10.0-1160.el7.x86_64 		0x6b
18
do_writepages
vmlinux 0xffffffff8c5cade1
Build id: 3.10.0-1160.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffff8c67d6f0
Build id: 3.10.0-1160.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffff8c67e274
Build id: 3.10.0-1160.el7.x86_64 		0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffff8c67e57f
Build id: 3.10.0-1160.el7.x86_64 		0x9f
22
wb_writeback
vmlinux 0xffffffff8c67ea53
Build id: 3.10.0-1160.el7.x86_64 		0x263
23
bdi_writeback_workfn
vmlinux 0xffffffff8c67f54c
Build id: 3.10.0-1160.el7.x86_64 		0x1cc
24
process_one_work
vmlinux 0xffffffff8c4bdc4f
Build id: 3.10.0-1160.el7.x86_64 		0x17f
25
worker_thread
vmlinux 0xffffffff8c4bed66
Build id: 3.10.0-1160.el7.x86_64 		0x126
26
? manage_workers.isra.26
vmlinux 0xffffffff8c4bec40
Build id: 3.10.0-1160.el7.x86_64 		0x2a0
27
kthread
vmlinux 0xffffffff8c4c5c21
Build id: 3.10.0-1160.el7.x86_64 		0xd1
28
? insert_kthread_work
vmlinux 0xffffffff8c4c5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff8cb93df7
Build id: 3.10.0-1160.el7.x86_64 		0x21
30
? insert_kthread_work
vmlinux 0xffffffff8c4c5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/u256:0:100620]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm crc32_pclmul snd_timer snd ghash_clmulni_intel ppdev aesni_intel soundcore lrw vmw_balloon gf128mul glue_helper pcspkr ablk_helper sg cryptd joydev parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm crct10dif_pclmul crct10dif_common crc32c_intel e1000 mptspi ata_piix scsi_transport_spi mptscsih mptbase libata nfit serio_raw libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 100620 Comm: kworker/u256:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9f727dea2100 ti: ffff9f71b8d58000 task.ti: ffff9f71b8d58000
RIP: 0010:[<ffffffffc01363be>]  [<ffffffffc01363be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9f71b8d5b708  EFLAGS: 00000246
RAX: ffffc12943480000 RBX: ffff9f70b4d1d000 RCX: 000000000000000f
RDX: ffff9f70b5d9be00 RSI: ffff9f752d52f000 RDI: 000000000000000e
RBP: ffff9f71b8d5b718 R08: 0000000000000002 R09: 000000032258afff
R10: ffff9f70b5c93800 R11: ffff9f70b5fbed80 R12: ffff9f71f47f4300
R13: ffff9f71b8d5b67c R14: ffffffff8c75bdfb R15: ffff9f71b8d5b6c0
FS:  0000000000000000(0000) GS:ffff9f752b640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc3aec04f00 CR3: 000000017e3ba000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc012a71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc015b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8c8e3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c8ed0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffff8c777f8e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8c7534d9>] __blk_run_queue+0x39/0x50
[<ffffffff8c7571e3>] blk_queue_bio+0x3b3/0x400
[<ffffffff8c755167>] generic_make_request+0x147/0x380
[<ffffffff8c755410>] submit_bio+0x70/0x150
[<ffffffffc0334f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc0335724>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff8c5c9d8c>] write_cache_pages+0x21c/0x470
[<ffffffffc0335290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff8c4e055c>] ? set_next_entity+0x3c/0xe0
[<ffffffff8c42b59e>] ? __switch_to+0xce/0x580
[<ffffffffc033525b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff8c5cade1>] do_writepages+0x21/0x50
[<ffffffff8c67d6f0>] __writeback_single_inode+0x40/0x260
[<ffffffff8c67e274>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8c67e57f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8c67ea53>] wb_writeback+0x263/0x2f0
[<ffffffff8c67f54c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff8c4bdc4f>] process_one_work+0x17f/0x440
[<ffffffff8c4bed66>] worker_thread+0x126/0x3c0
[<ffffffff8c4bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8c4c5c21>] kthread+0xd1/0xe0
[<ffffffff8c4c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8cb93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8c4c5b50>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 43 14
Complete report #957430
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03383be
Build id: 3.10.0-1127.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc035a71e
Build id: 3.10.0-1127.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03722e0
Build id: 3.10.0-1127.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa36e2850
Build id: 3.10.0-1127.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffa36ebefc
Build id: 3.10.0-1127.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffffa357709e
Build id: 3.10.0-1127.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffa35526f9
Build id: 3.10.0-1127.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffa3556403
Build id: 3.10.0-1127.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffa3554387
Build id: 3.10.0-1127.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffa3554630
Build id: 3.10.0-1127.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc05c1071
Build id: 3.10.0-1127.el7.x86_64 		0x61
12
xfs_do_writepage
xfs 0xffffffffc05c1864
Build id: 3.10.0-1127.el7.x86_64 		0x494
13
write_cache_pages
vmlinux 0xffffffffa33ca15c
Build id: 3.10.0-1127.el7.x86_64 		0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc05c13d0
Build id: 3.10.0-1127.el7.x86_64 		0xa0
15
? pick_next_entity
vmlinux 0xffffffffa32e2e89
Build id: 3.10.0-1127.el7.x86_64 		0xa9
16
? set_next_entity
vmlinux 0xffffffffa32e103c
Build id: 3.10.0-1127.el7.x86_64 		0x3c
17
? __switch_to
vmlinux 0xffffffffa322b59e
Build id: 3.10.0-1127.el7.x86_64 		0xce
18
xfs_vm_writepages
xfs 0xffffffffc05c139b
Build id: 3.10.0-1127.el7.x86_64 		0x6b
19
do_writepages
vmlinux 0xffffffffa33cb1b1
Build id: 3.10.0-1127.el7.x86_64 		0x21
20
__writeback_single_inode
vmlinux 0xffffffffa347cbd0
Build id: 3.10.0-1127.el7.x86_64 		0x40
21
writeback_sb_inodes
vmlinux 0xffffffffa347d764
Build id: 3.10.0-1127.el7.x86_64 		0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffffa347da6f
Build id: 3.10.0-1127.el7.x86_64 		0x9f
23
wb_writeback
vmlinux 0xffffffffa347df53
Build id: 3.10.0-1127.el7.x86_64 		0x263
24
bdi_writeback_workfn
vmlinux 0xffffffffa347eb4b
Build id: 3.10.0-1127.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffffa32be6bf
Build id: 3.10.0-1127.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffffa32bf7d6
Build id: 3.10.0-1127.el7.x86_64 		0x126
27
? manage_workers.isra.26
vmlinux 0xffffffffa32bf6b0
Build id: 3.10.0-1127.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffffa32c6691
Build id: 3.10.0-1127.el7.x86_64 		0xd1
29
? insert_kthread_work
vmlinux 0xffffffffa32c65c0
Build id: 3.10.0-1127.el7.x86_64 		0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffffa3992d1d
Build id: 3.10.0-1127.el7.x86_64 		0x7
31
? insert_kthread_work
vmlinux 0xffffffffa32c65c0
Build id: 3.10.0-1127.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 107s! [kworker/u256:1:49047]
Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm dns_resolver xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables tun xt_conntrack iptable_filter vmw_vsock_vmci_transport vsock xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_timer vmw_balloon snd sg crc32_pclmul soundcore ghash_clmulni_intel aesni_intel pcspkr lrw gf128mul glue_helper ablk_helper cryptd joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx ata_piix drm_kms_helper syscopyarea
sysfillrect sysimgblt libata fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm nfit e1000 serio_raw libnvdimm mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 49047 Comm: kworker/u256:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff911fba8bd230 ti: ffff911f85268000 task.ti: ffff911f85268000
RIP: 0010:[<ffffffffc03383be>]  [<ffffffffc03383be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff911f8526b708  EFLAGS: 00000246
RAX: ffffae0a83200000 RBX: ffff911b35cec000 RCX: 0000000000000000
RDX: ffff911b35db9200 RSI: ffff91202380a000 RDI: 000000000000000e
RBP: ffff911f8526b718 R08: 0000000000000002 R09: 0000000073306fff
R10: ffff911c772f6400 R11: ffff91202310fd40 R12: ffff911e6ba2ea00
R13: ffff911f8526b67c R14: ffffffffa355b01b R15: ffff911f8526b6c0
FS:  0000000000000000(0000) GS:ffff912029600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe966e3c000 CR3: 0000000510bfe000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc035a71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03722e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa36e2850>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa36ebefc>] scsi_request_fn+0x4ac/0x660
[<ffffffffa357709e>] ? deadline_add_request+0x6e/0x90
[<ffffffffa35526f9>] __blk_run_queue+0x39/0x50
[<ffffffffa3556403>] blk_queue_bio+0x3b3/0x400
[<ffffffffa3554387>] generic_make_request+0x147/0x380
[<ffffffffa3554630>] submit_bio+0x70/0x150
[<ffffffffc05c1071>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc05c1864>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffa33ca15c>] write_cache_pages+0x21c/0x470
[<ffffffffc05c13d0>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffa32e2e89>] ? pick_next_entity+0xa9/0x190
[<ffffffffa32e103c>] ? set_next_entity+0x3c/0xe0
[<ffffffffa322b59e>] ? __switch_to+0xce/0x580
[<ffffffffc05c139b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffa33cb1b1>] do_writepages+0x21/0x50
[<ffffffffa347cbd0>] __writeback_single_inode+0x40/0x260
[<ffffffffa347d764>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa347da6f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa347df53>] wb_writeback+0x263/0x2f0
[<ffffffffa347eb4b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa32be6bf>] process_one_work+0x17f/0x440
[<ffffffffa32bf7d6>] worker_thread+0x126/0x3c0
[<ffffffffa32bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa32c6691>] kthread+0xd1/0xe0
[<ffffffffa32c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa3992d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa32c65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 34
Complete report #945018
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff91dba385
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc01846a2
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff91b0b630
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff91b02ae0
Build id: 3.10.0-1160.83.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff91b0c68c
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9198db9e
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff91968159
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff9196c083
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff91969eb7
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x157
10
submit_bio
vmlinux 0xffffffff9196a160
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc03cf341
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x61
12
xfs_do_writepage
xfs 0xffffffffc03cfb54
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x494
13
write_cache_pages
vmlinux 0xffffffff917d587c
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc03cf6c0
Build id: 3.10.0-1160.83.1.el7.x86_64 		0xb0
15
? __enqueue_entity
vmlinux 0xffffffff916e5bf8
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x78
16
? put_prev_entity
vmlinux 0xffffffff916ea09b
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x8b
17
xfs_vm_writepages
xfs 0xffffffffc03cf67b
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x6b
18
do_writepages
vmlinux 0xffffffff917d6921
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffff9188c790
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffff9188d35c
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x1cc
21
__writeback_inodes_wb
vmlinux 0xffffffff9188d667
Build id: 3.10.0-1160.83.1.el7.x86_64 		0xa7
22
wb_writeback
vmlinux 0xffffffff9188db43
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x263
23
bdi_writeback_workfn
vmlinux 0xffffffff9188e74f
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x2cf
24
process_one_work
vmlinux 0xffffffff916c31df
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x17f
25
worker_thread
vmlinux 0xffffffff916c4326
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x126
26
? manage_workers.isra.26
vmlinux 0xffffffff916c4200
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x2b0
27
kthread
vmlinux 0xffffffff916cb511
Build id: 3.10.0-1160.83.1.el7.x86_64 		0xd1
28
? insert_kthread_work
vmlinux 0xffffffff916cb440
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff91dc51dd
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x7
30
? insert_kthread_work
vmlinux 0xffffffff916cb440
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/u4:2:6402]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_16108(E) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon pcspkr joydev sg vmw_vmci parport_pc parport i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel drm ata_piix serio_raw nfit libnvdimm libata vmxnet3 vmw_pvscsi
drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 6402 Comm: kworker/u4:2 Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.83.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff905120dd2100 ti: ffff9051584cc000 task.ti: ffff9051584cc000
RIP: 0010:[<ffffffff91dba385>]  [<ffffffff91dba385>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9051584cf770  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff919c9843 RCX: ffffffff92234f40
RDX: ffffa9eb41b80000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff9051584cf770 R08: 0000000000000001 R09: 0000000000000000
R10: ffff90526d399c00 R11: ffff90526eadf000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff904f9b0dd200 R15: ffffffff91b0a269
FS:  0000000000000000(0000) GS:ffff90527fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0a380fa9d0 CR3: 000000018365a000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc01846a2>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff91b0b630>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff91b02ae0>] scsi_dispatch_cmd+0xb0/0x250
[<ffffffff91b0c68c>] scsi_request_fn+0x4ac/0x680
[<ffffffff9198db9e>] ? deadline_add_request+0x6e/0x90
[<ffffffff91968159>] __blk_run_queue+0x39/0x50
[<ffffffff9196c083>] blk_queue_bio+0x3b3/0x400
[<ffffffff91969eb7>] generic_make_request+0x157/0x390
[<ffffffff9196a160>] submit_bio+0x70/0x150
[<ffffffffc03cf341>] xfs_submit_ioend.isra.12+0x61/0xf0 [xfs]
[<ffffffffc03cfb54>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff917d587c>] write_cache_pages+0x21c/0x480
[<ffffffffc03cf6c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffff916e5bf8>] ? __enqueue_entity+0x78/0x80
[<ffffffff916ea09b>] ? put_prev_entity+0x8b/0x400
[<ffffffffc03cf67b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffff917d6921>] do_writepages+0x21/0x50
[<ffffffff9188c790>] __writeback_single_inode+0x40/0x260
[<ffffffff9188d35c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff9188d667>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff9188db43>] wb_writeback+0x263/0x2f0
[<ffffffff9188e74f>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffff916c31df>] process_one_work+0x17f/0x440
[<ffffffff916c4326>] worker_thread+0x126/0x3c0
[<ffffffff916c4200>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff916cb511>] kthread+0xd1/0xe0
[<ffffffff916cb440>] ? insert_kthread_work+0x40/0x40
[<ffffffff91dc51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff916cb440>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 cc cc cc cc 66 66 66 66 90 55 48 89 e5 48
Complete report #781532
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc021f3be
Build id: 3.10.0-1062.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc028e71e
Build id: 3.10.0-1062.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02872e0
Build id: 3.10.0-1062.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffae0dc890
Build id: 3.10.0-1062.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffae0e5efc
Build id: 3.10.0-1062.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffffadf716de
Build id: 3.10.0-1062.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffadf4cf59
Build id: 3.10.0-1062.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffadf50c63
Build id: 3.10.0-1062.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffadf4ebf7
Build id: 3.10.0-1062.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffadf4eea0
Build id: 3.10.0-1062.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc044bcd1
Build id: 3.10.0-1062.el7.x86_64 		0x61
12
xfs_do_writepage
xfs 0xffffffffc044c4c4
Build id: 3.10.0-1062.el7.x86_64 		0x494
13
write_cache_pages
vmlinux 0xffffffffaddc7cec
Build id: 3.10.0-1062.el7.x86_64 		0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc044c030
Build id: 3.10.0-1062.el7.x86_64 		0xa0
15
? submit_bio
vmlinux 0xffffffffadf4eea0
Build id: 3.10.0-1062.el7.x86_64 		0x70
16
? __switch_to
vmlinux 0xffffffffadc2b59e
Build id: 3.10.0-1062.el7.x86_64 		0xce
17
xfs_vm_writepages
xfs 0xffffffffc044bffb
Build id: 3.10.0-1062.el7.x86_64 		0x6b
18
do_writepages
vmlinux 0xffffffffaddc8d31
Build id: 3.10.0-1062.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffffade776c0
Build id: 3.10.0-1062.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffffade78254
Build id: 3.10.0-1062.el7.x86_64 		0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffffade7855f
Build id: 3.10.0-1062.el7.x86_64 		0x9f
22
wb_writeback
vmlinux 0xffffffffade78a43
Build id: 3.10.0-1062.el7.x86_64 		0x263
23
bdi_writeback_workfn
vmlinux 0xffffffffade7963b
Build id: 3.10.0-1062.el7.x86_64 		0x2cb
24
process_one_work
vmlinux 0xffffffffadcbd0ff
Build id: 3.10.0-1062.el7.x86_64 		0x17f
25
worker_thread
vmlinux 0xffffffffadcbe216
Build id: 3.10.0-1062.el7.x86_64 		0x126
26
? manage_workers.isra.26
vmlinux 0xffffffffadcbe0f0
Build id: 3.10.0-1062.el7.x86_64 		0x2a0
27
kthread
vmlinux 0xffffffffadcc50d1
Build id: 3.10.0-1062.el7.x86_64 		0xd1
28
? insert_kthread_work
vmlinux 0xffffffffadcc5000
Build id: 3.10.0-1062.el7.x86_64 		0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffffae38bd1d
Build id: 3.10.0-1062.el7.x86_64 		0x7
30
? insert_kthread_work
vmlinux 0xffffffffadcc5000
Build id: 3.10.0-1062.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u256:2:27071]
Modules linked in: xt_nat veth binfmt_misc tcp_lp nls_utf8 isofs rfcomm fuse nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev vmw_balloon crc32_pclmul ghash_clmulni_intel btusb snd_ens1371 btrtl snd_rawmidi btbcm snd_ac97_codec btintel bluetooth ac97_bus aesni_intel snd_seq snd_seq_device lrw gf128mul snd_pcm glue_helper ablk_helper cryptd snd_timer joydev rfkill snd pcspkr sg vmw_vmci soundcore i2c_piix4 parport_pc parport
ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix crct10dif_pclmul crct10dif_common nfit scsi_transport_spi crc32c_intel mptscsih libata mptbase libnvdimm serio_raw e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 27071 Comm: kworker/u256:2 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffffa110f1fb62a0 ti: ffffa11120e60000 task.ti: ffffa11120e60000
RIP: 0010:[<ffffffffc021f3be>]  [<ffffffffc021f3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffffa11120e63708  EFLAGS: 00000246
RAX: ffffbb9c81980000 RBX: ffffa10f94e93288 RCX: 000000000000000a
RDX: ffffa1112b6b0200 RSI: ffffa1112d7e1000 RDI: 000000000000000e
RBP: ffffa11120e63718 R08: 0000000000000003 R09: 000000032912a3ff
R10: ffffa1112d2250d8 R11: ffffa11129908700 R12: ffffa1111d698d80
R13: ffffa11120e6367c R14: ffffffffadf557ab R15: ffffa11120e636c0
FS:  0000000000000000(0000) GS:ffffa11131600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff2ac045480 CR3: 0000000322b70000 CR4: 00000000007607f0
PKRU: 00000000
Call Trace:
[<ffffffffc028e71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02872e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffae0dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffae0e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffadf716de>] ? deadline_add_request+0x6e/0x90
[<ffffffffadf4cf59>] __blk_run_queue+0x39/0x50
[<ffffffffadf50c63>] blk_queue_bio+0x3b3/0x400
[<ffffffffadf4ebf7>] generic_make_request+0x147/0x380
[<ffffffffadf4eea0>] submit_bio+0x70/0x150
[<ffffffffc044bcd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc044c4c4>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffaddc7cec>] write_cache_pages+0x21c/0x470
[<ffffffffc044c030>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffadf4eea0>] ? submit_bio+0x70/0x150
[<ffffffffadc2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc044bffb>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffaddc8d31>] do_writepages+0x21/0x50
[<ffffffffade776c0>] __writeback_single_inode+0x40/0x260
[<ffffffffade78254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffade7855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffade78a43>] wb_writeback+0x263/0x2f0
[<ffffffffade7963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffadcbd0ff>] process_one_work+0x17f/0x440
[<ffffffffadcbe216>] worker_thread+0x126/0x3c0
[<ffffffffadcbe0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffadcc50d1>] kthread+0xd1/0xe0
[<ffffffffadcc5000>] ? insert_kthread_work+0x40/0x40
[<ffffffffae38bd1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffadcc5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 22
Complete report #738582
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01223be
Build id: 3.10.0-1062.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc01c771e
Build id: 3.10.0-1062.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00e92e0
Build id: 3.10.0-1062.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff93edc890
Build id: 3.10.0-1062.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff93ee5efc
Build id: 3.10.0-1062.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffff93d716de
Build id: 3.10.0-1062.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff93d4cf59
Build id: 3.10.0-1062.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff93d50c63
Build id: 3.10.0-1062.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff93d4ebf7
Build id: 3.10.0-1062.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff93d4eea0
Build id: 3.10.0-1062.el7.x86_64 		0x70
11
? __schedule
vmlinux 0xffffffff9417eb31
Build id: 3.10.0-1062.el7.x86_64 		0x351
12
? __switch_to
vmlinux 0xffffffff93a2b59e
Build id: 3.10.0-1062.el7.x86_64 		0xce
13
xfs_submit_ioend.isra.12
xfs 0xffffffffc034ccd1
Build id: 3.10.0-1062.el7.x86_64 		0x61
14
xfs_vm_writepages
xfs 0xffffffffc034d00f
Build id: 3.10.0-1062.el7.x86_64 		0x7f
15
do_writepages
vmlinux 0xffffffff93bc8d31
Build id: 3.10.0-1062.el7.x86_64 		0x21
16
__writeback_single_inode
vmlinux 0xffffffff93c776c0
Build id: 3.10.0-1062.el7.x86_64 		0x40
17
writeback_sb_inodes
vmlinux 0xffffffff93c78254
Build id: 3.10.0-1062.el7.x86_64 		0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffff93c7855f
Build id: 3.10.0-1062.el7.x86_64 		0x9f
19
wb_writeback
vmlinux 0xffffffff93c78a43
Build id: 3.10.0-1062.el7.x86_64 		0x263
20
bdi_writeback_workfn
vmlinux 0xffffffff93c7963b
Build id: 3.10.0-1062.el7.x86_64 		0x2cb
21
process_one_work
vmlinux 0xffffffff93abd0ff
Build id: 3.10.0-1062.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffff93abe216
Build id: 3.10.0-1062.el7.x86_64 		0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff93abe0f0
Build id: 3.10.0-1062.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffff93ac50d1
Build id: 3.10.0-1062.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffff93ac5000
Build id: 3.10.0-1062.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff9418bd1d
Build id: 3.10.0-1062.el7.x86_64 		0x7
27
? insert_kthread_work
vmlinux 0xffffffff93ac5000
Build id: 3.10.0-1062.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 35s! [kworker/u256:1:110864]
Modules linked in: nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_vmci ac97_bus aesni_intel ppdev snd_seq lrw gf128mul glue_helper snd_seq_device vmw_balloon snd_pcm ablk_helper pcspkr sg cryptd snd_timer joydev i2c_piix4 snd soundcore parport_pc parport pcc_cpufreq ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih libata mptbase e1000 nfit libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 110864 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8e5f78f61070 ti: ffff8e5962ba4000 task.ti: ffff8e5962ba4000
RIP: 0010:[<ffffffffc01223be>]  [<ffffffffc01223be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8e5962ba7888  EFLAGS: 00000246
RAX: ffffb1ab062c0000 RBX: ffff8e639f7ee000 RCX: 0000000000000028
RDX: ffff8e63b80c7400 RSI: ffff8e63bf5a9000 RDI: 000000000000000e
RBP: ffff8e5962ba7898 R08: 0000000000000002 R09: 0000000f9b51dfff
R10: ffff8e63bf5a80d8 R11: ffff8e63b6762840 R12: ffff8e634bbc7c00
R13: ffff8e5962ba77fc R14: ffffffff93d557ab R15: ffff8e5962ba7840
FS:  0000000000000000(0000) GS:ffff8e63bd680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002bdc61b48000 CR3: 0000000fe1620000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffc01c771e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00e92e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff93edc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff93ee5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff93d716de>] ? deadline_add_request+0x6e/0x90
[<ffffffff93d4cf59>] __blk_run_queue+0x39/0x50
[<ffffffff93d50c63>] blk_queue_bio+0x3b3/0x400
[<ffffffff93d4ebf7>] generic_make_request+0x147/0x380
[<ffffffff93d4eea0>] submit_bio+0x70/0x150
[<ffffffff9417eb31>] ? __schedule+0x351/0x9c0
[<ffffffff93a2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc034ccd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc034d00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff93bc8d31>] do_writepages+0x21/0x50
[<ffffffff93c776c0>] __writeback_single_inode+0x40/0x260
[<ffffffff93c78254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff93c7855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff93c78a43>] wb_writeback+0x263/0x2f0
[<ffffffff93c7963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff93abd0ff>] process_one_work+0x17f/0x440
[<ffffffff93abe216>] worker_thread+0x126/0x3c0
[<ffffffff93abe0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff93ac50d1>] kthread+0xd1/0xe0
[<ffffffff93ac5000>] ? insert_kthread_work+0x40/0x40
[<ffffffff9418bd1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff93ac5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 13
Complete report #651670
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03003be
Build id: 3.10.0-862.9.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02de71e
Build id: 3.10.0-862.9.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02b92e0
Build id: 3.10.0-862.9.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffbd09bb60
Build id: 3.10.0-862.9.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffbd0a50cc
Build id: 3.10.0-862.9.1.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffffbcf3d28e
Build id: 3.10.0-862.9.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffbcf19519
Build id: 3.10.0-862.9.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffbcf1d1a3
Build id: 3.10.0-862.9.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffbcf1b0fb
Build id: 3.10.0-862.9.1.el7.x86_64 		0x10b
10
submit_bio
vmlinux 0xffffffffbcf1b380
Build id: 3.10.0-862.9.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc05254b1
Build id: 3.10.0-862.9.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc0525671
Build id: 3.10.0-862.9.1.el7.x86_64 		0xd1
13
do_writepages
vmlinux 0xffffffffbcda15e1
Build id: 3.10.0-862.9.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffffbce49260
Build id: 3.10.0-862.9.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffffbce49cf4
Build id: 3.10.0-862.9.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffbce4a05f
Build id: 3.10.0-862.9.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffffbce4a893
Build id: 3.10.0-862.9.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffbce4b21b
Build id: 3.10.0-862.9.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffffbccb35ef
Build id: 3.10.0-862.9.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffffbccb4686
Build id: 3.10.0-862.9.1.el7.x86_64 		0x126
21
? manage_workers.isra.24
vmlinux 0xffffffffbccb4560
Build id: 3.10.0-862.9.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffffbccbb621
Build id: 3.10.0-862.9.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffffbccbb550
Build id: 3.10.0-862.9.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffbd3205f7
Build id: 3.10.0-862.9.1.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffffbccbb550
Build id: 3.10.0-862.9.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u256:1:80388]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel snd_ens1371 lrw snd_rawmidi gf128mul glue_helper ablk_helper cryptd snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm joydev pcspkr sg snd_timer nfit snd soundcore i2c_piix4 libnvdimm vmw_vmci shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom
crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops ttm ata_piix serio_raw drm libata mptspi scsi_transport_spi e1000 mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 80388 Comm: kworker/u256:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-862.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff8e433be50fd0 ti: ffff8e41b87ac000 task.ti: ffff8e41b87ac000
RIP: 0010:[<ffffffffc03003be>]  [<ffffffffc03003be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8e41b87af888  EFLAGS: 00000246
RAX: ffffae9e43200000 RBX: ffff8e4712f88000 RCX: 0000000000000014
RDX: ffff8e41b5cc5400 RSI: ffff8e48eef0a000 RDI: 000000000000000e
RBP: ffff8e41b87af898 R08: 0000000000000002 R09: 000000063339dfff
R10: ffff8e48f4f280d8 R11: ffff8e45a569fb80 R12: ffff8e465b880c00
R13: ffff8e41b87af7fc R14: ffffffffbcf21d0b R15: ffff8e41b87af840
FS:  0000000000000000(0000) GS:ffff8e48f3000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000cf5d360 CR3: 0000000771924000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc02de71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02b92e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffbd09bb60>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbd0a50cc>] scsi_request_fn+0x4cc/0x680
[<ffffffffbcf3d28e>] ? deadline_add_request+0x6e/0x90
[<ffffffffbcf19519>] __blk_run_queue+0x39/0x50
[<ffffffffbcf1d1a3>] blk_queue_bio+0x3b3/0x400
[<ffffffffbcf1b0fb>] generic_make_request+0x10b/0x320
[<ffffffffbcf1b380>] submit_bio+0x70/0x150
[<ffffffffc05254b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0525671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffffbcda15e1>] do_writepages+0x21/0x50
[<ffffffffbce49260>] __writeback_single_inode+0x40/0x260
[<ffffffffbce49cf4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbce4a05f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbce4a893>] wb_writeback+0x263/0x2f0
[<ffffffffbce4b21b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbccb35ef>] process_one_work+0x17f/0x440
[<ffffffffbccb4686>] worker_thread+0x126/0x3c0
[<ffffffffbccb4560>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffbccbb621>] kthread+0xd1/0xe0
[<ffffffffbccbb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd3205f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbccbb550>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 30
Complete report #533271
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9ed8b795
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0098632
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff9eaecf50
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff9eae4840
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff9eaedf6c
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9e978bce
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff9e953f19
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff9e957cc3
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff9e955ba7
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff9e955e50
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x70
11
? pick_next_entity
vmlinux 0xffffffff9e6e22d9
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xa9
12
? set_next_entity
vmlinux 0xffffffff9e6e26ec
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x3c
13
xfs_submit_ioend.isra.12
xfs 0xffffffffc02ebf31
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x61
14
xfs_vm_writepages
xfs 0xffffffffc02ec26f
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x7f
15
do_writepages
vmlinux 0xffffffff9e7cb9b1
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x21
16
__writeback_single_inode
vmlinux 0xffffffff9e87def0
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40
17
writeback_sb_inodes
vmlinux 0xffffffff9e87ea74
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x1c4
18
wb_writeback
vmlinux 0xffffffff9e87f0ef
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xff
19
bdi_writeback_workfn
vmlinux 0xffffffff9e87fc95
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x115
20
process_one_work
vmlinux 0xffffffff9e6bde8f
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x17f
21
worker_thread
vmlinux 0xffffffff9e6befa6
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff9e6bee80
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x2a0
23
kthread
vmlinux 0xffffffff9e6c5e61
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xd1
24
? insert_kthread_work
vmlinux 0xffffffff9e6c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff9ed95df7
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x21
26
? insert_kthread_work
vmlinux 0xffffffff9e6c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [kworker/u20:0:22646]
Modules linked in: binfmt_misc xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg vmw_balloon pcspkr joydev vmw_vmci parport_pc parport i2c_piix4
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops nfit ttm sd_mod libata libnvdimm crc_t10dif crct10dif_generic drm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 7 PID: 22646 Comm: kworker/u20:0 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8c948feb8000 ti: ffff8c9f260bc000 task.ti: ffff8c9f260bc000
RIP: 0010:[<ffffffff9ed8b795>]  [<ffffffff9ed8b795>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8c9f260bf940  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff9e9b2f63 RCX: ffffffff9f234f40
RDX: ffffa257cce40000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8c9f260bf940 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8ca31a31cc00 R11: ffff8ca319d82000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8c9952b06180 R15: ffffffff9eaebc09
FS:  0000000000000000(0000) GS:ffff8ca31f380000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fff1a662000 CR3: 0000001718c68000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0098632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff9eaecf50>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff9eae4840>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9eaedf6c>] scsi_request_fn+0x4ac/0x680
[<ffffffff9e978bce>] ? deadline_add_request+0x6e/0x90
[<ffffffff9e953f19>] __blk_run_queue+0x39/0x50
[<ffffffff9e957cc3>] blk_queue_bio+0x3b3/0x400
[<ffffffff9e955ba7>] generic_make_request+0x147/0x380
[<ffffffff9e955e50>] submit_bio+0x70/0x150
[<ffffffff9e6e22d9>] ? pick_next_entity+0xa9/0x190
[<ffffffff9e6e26ec>] ? set_next_entity+0x3c/0xe0
[<ffffffffc02ebf31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc02ec26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9e7cb9b1>] do_writepages+0x21/0x50
[<ffffffff9e87def0>] __writeback_single_inode+0x40/0x260
[<ffffffff9e87ea74>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9e87f0ef>] wb_writeback+0xff/0x2f0
[<ffffffff9e87fc95>] bdi_writeback_workfn+0x115/0x460
[<ffffffff9e6bde8f>] process_one_work+0x17f/0x440
[<ffffffff9e6befa6>] worker_thread+0x126/0x3c0
[<ffffffff9e6bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9e6c5e61>] kthread+0xd1/0xe0
[<ffffffff9e6c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffff9ed95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9e6c5d90>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #532558
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa6d8b795
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc023e632
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffa6aecf70
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffa6ae4860
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffa6aedf8c
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffffa6978bee
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffa6953f39
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffa6957ce3
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffa6955bc7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffa6955e70
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc04b6f31
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc04b726f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffffa67cb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffffa687df10
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffffa687ea94
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffa687ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffffa687f273
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffa687fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffffa66bde8f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffffa66befa6
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffffa66bee80
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffffa66c5e61
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffffa66c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa6d95df7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffffa66c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [kworker/u8:1:9584]
Modules linked in: eset_rtp(OE) rpcsec_gss_krb5 vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_vmci pcspkr joydev sg vmw_balloon i2c_piix4 parport_pc parport dm_multipath nfsd nfs_acl lockd binfmt_misc grace auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif ahci crct10dif_generic drm_kms_helper ata_piix libahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata crct10dif_pclmul crct10dif_common nfit drm crc32c_intel libnvdimm serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: eset_rtp]
CPU: 3 PID: 9584 Comm: kworker/u8:1 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff89dc4992b180 ti: ffff89dcf865c000 task.ti: ffff89dcf865c000
RIP: 0010:[<ffffffffa6d8b795>]  [<ffffffffa6d8b795>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff89dcf865f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffa69b2f83 RCX: ffff89ddbab21060
RDX: ffffad2780e90000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff89dcf865f8f0 R08: 0000000000000000 R09: ffff89dc4514a400
R10: ffff89de73ac3c00 R11: ffff89de72850800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff89de754ac180 R15: ffffffffa6aebc29
FS:  0000000000000000(0000) GS:ffff89de7fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe798436000 CR3: 000000007b210000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc023e632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffa6aecf70>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffa6ae4860>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa6aedf8c>] scsi_request_fn+0x4ac/0x680
[<ffffffffa6978bee>] ? deadline_add_request+0x6e/0x90
[<ffffffffa6953f39>] __blk_run_queue+0x39/0x50
[<ffffffffa6957ce3>] blk_queue_bio+0x3b3/0x400
[<ffffffffa6955bc7>] generic_make_request+0x147/0x380
[<ffffffffa6955e70>] submit_bio+0x70/0x150
[<ffffffffc04b6f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04b726f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffa67cb9b1>] do_writepages+0x21/0x50
[<ffffffffa687df10>] __writeback_single_inode+0x40/0x260
[<ffffffffa687ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa687ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa687f273>] wb_writeback+0x263/0x2f0
[<ffffffffa687fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa66bde8f>] process_one_work+0x17f/0x440
[<ffffffffa66befa6>] worker_thread+0x126/0x3c0
[<ffffffffa66bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa66c5e61>] kthread+0xd1/0xe0
[<ffffffffa66c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffa6d95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa66c5d90>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #527816
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc037a3be
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc046571e
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc047e2e0
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff97adc890
Build id: 3.10.0-1062.1.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff97ae5efc
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffff979716de
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff9794cf59
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff97950c63
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff9794ebf7
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff9794eea0
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc060ccd1
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc060d00f
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffff977c8d31
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff978776c0
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff97878254
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff9787855f
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff97878a43
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9787963b
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff976bd0ff
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff976be216
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff976be0f0
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff976c50d1
Build id: 3.10.0-1062.1.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff976c5000
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff97d8cd1d
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x7
25
? insert_kthread_work
vmlinux 0xffffffff976c5000
Build id: 3.10.0-1062.1.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [kworker/u256:2:4140]
Modules linked in: resguard_linux(OE) secmodel_linux(OE) syshook_linux(OE) secmodel2(OE) tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr sg i2c_piix4 vmw_vmci parport_pc parport
ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul mptspi crct10dif_common ata_piix crc32c_intel scsi_transport_spi mptscsih libata nfit e1000 libnvdimm serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 4140 Comm: kworker/u256:2 Tainted: G        W  OEL ------------   3.10.0-1062.1.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff94be05e320e0 ti: ffff94be277a8000 task.ti: ffff94be277a8000
RIP: 0010:[<ffffffffc037a3be>]  [<ffffffffc037a3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff94be277ab888  EFLAGS: 00000246
RAX: ffffadaa803c0000 RBX: ffff94be36114000 RCX: 0000000000000005
RDX: ffff94be362b2c00 RSI: ffff94be35dda000 RDI: 000000000000000e
RBP: ffff94be277ab898 R08: 0000000000000002 R09: 0000000049773fff
R10: ffff94be35d5e0d8 R11: ffff94be595d0540 R12: ffff94be0c9e0000
R13: ffff94be277ab7fc R14: ffffffff979557ab R15: ffff94be277ab840
FS:  0000000000000000(0000) GS:ffff94be7b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006373d0 CR3: 0000000057de0000 CR4: 00000000007607f0
PKRU: 00000000
Call Trace:
[<ffffffffc046571e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc047e2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff97adc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff97ae5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff979716de>] ? deadline_add_request+0x6e/0x90
[<ffffffff9794cf59>] __blk_run_queue+0x39/0x50
[<ffffffff97950c63>] blk_queue_bio+0x3b3/0x400
[<ffffffff9794ebf7>] generic_make_request+0x147/0x380
[<ffffffff9794eea0>] submit_bio+0x70/0x150
[<ffffffffc060ccd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc060d00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff977c8d31>] do_writepages+0x21/0x50
[<ffffffff978776c0>] __writeback_single_inode+0x40/0x260
[<ffffffff97878254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9787855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff97878a43>] wb_writeback+0x263/0x2f0
[<ffffffff9787963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff976bd0ff>] process_one_work+0x17f/0x440
[<ffffffff976be216>] worker_thread+0x126/0x3c0
[<ffffffff976be0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff976c50d1>] kthread+0xd1/0xe0
[<ffffffff976c5000>] ? insert_kthread_work+0x40/0x40
[<ffffffff97d8cd1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff976c5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 83 38
Complete report #465935
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffad51b2c5
Build id: 3.10.0-862.11.6.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0125622
Build id: 3.10.0-862.11.6.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffad2a8500
Build id: 3.10.0-862.11.6.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffad2a0010
Build id: 3.10.0-862.11.6.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffad2a957c
Build id: 3.10.0-862.11.6.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffffad14122e
Build id: 3.10.0-862.11.6.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffad11d4e9
Build id: 3.10.0-862.11.6.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffad121173
Build id: 3.10.0-862.11.6.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffad11f0cb
Build id: 3.10.0-862.11.6.el7.x86_64 		0x10b
10
submit_bio
vmlinux 0xffffffffad11f350
Build id: 3.10.0-862.11.6.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc03634b1
Build id: 3.10.0-862.11.6.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc0363671
Build id: 3.10.0-862.11.6.el7.x86_64 		0xd1
13
do_writepages
vmlinux 0xffffffffacfa3ac1
Build id: 3.10.0-862.11.6.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffffad04cf00
Build id: 3.10.0-862.11.6.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffffad04d994
Build id: 3.10.0-862.11.6.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffad04dcff
Build id: 3.10.0-862.11.6.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffffad04e533
Build id: 3.10.0-862.11.6.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffad04edbc
Build id: 3.10.0-862.11.6.el7.x86_64 		0x1cc
19
process_one_work
vmlinux 0xffffffffaceb613f
Build id: 3.10.0-862.11.6.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffffaceb71d6
Build id: 3.10.0-862.11.6.el7.x86_64 		0x126
21
? manage_workers.isra.24
vmlinux 0xffffffffaceb70b0
Build id: 3.10.0-862.11.6.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffffacebdf21
Build id: 3.10.0-862.11.6.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffffacebde50
Build id: 3.10.0-862.11.6.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffad5255dd
Build id: 3.10.0-862.11.6.el7.x86_64 		0x7
25
? insert_kthread_work
vmlinux 0xffffffffacebde50
Build id: 3.10.0-862.11.6.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [kworker/u21:0:26266]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_2_4_933667(OE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon ppdev
pcspkr nfit sg joydev libnvdimm parport_pc parport vmw_vmci shpchp i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul libata crct10dif_common crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 26266 Comm: kworker/u21:0 Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9c42b6f48000 ti: ffff9c3d53bdc000 task.ti: ffff9c3d53bdc000
RIP: 0010:[<ffffffffad51b2c5>]  [<ffffffffad51b2c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9c3d53bdf8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffad1795b3 RCX: ffffffffada30880
RDX: ffffab220ce20000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9c3d53bdf8f0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff9c537ff008d8 R11: ffff9c46cb5bc800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9c44f9649c80 R15: ffffffffad2a7259
FS:  0000000000000000(0000) GS:ffff9c46cda00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001c77a50 CR3: 000000190637c000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc0125622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffad2a8500>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffad2a0010>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffad2a957c>] scsi_request_fn+0x4cc/0x680
[<ffffffffad14122e>] ? deadline_add_request+0x6e/0x90
[<ffffffffad11d4e9>] __blk_run_queue+0x39/0x50
[<ffffffffad121173>] blk_queue_bio+0x3b3/0x400
[<ffffffffad11f0cb>] generic_make_request+0x10b/0x320
[<ffffffffad11f350>] submit_bio+0x70/0x150
[<ffffffffc03634b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0363671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffffacfa3ac1>] do_writepages+0x21/0x50
[<ffffffffad04cf00>] __writeback_single_inode+0x40/0x260
[<ffffffffad04d994>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffad04dcff>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffad04e533>] wb_writeback+0x263/0x2f0
[<ffffffffad04edbc>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffaceb613f>] process_one_work+0x17f/0x440
[<ffffffffaceb71d6>] worker_thread+0x126/0x3c0
[<ffffffffaceb70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffacebdf21>] kthread+0xd1/0xe0
[<ffffffffacebde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffad5255dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffacebde50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #453083
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02af3be
Build id: 3.10.0-1160.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02d171e
Build id: 3.10.0-1160.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02ca2e0
Build id: 3.10.0-1160.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb8ce3980
Build id: 3.10.0-1160.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffb8ced0ac
Build id: 3.10.0-1160.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb8b77f8e
Build id: 3.10.0-1160.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffb8b534d9
Build id: 3.10.0-1160.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffb8b571e3
Build id: 3.10.0-1160.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffb8b55167
Build id: 3.10.0-1160.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffb8b55410
Build id: 3.10.0-1160.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc054df31
Build id: 3.10.0-1160.el7.x86_64 		0x61
12
xfs_do_writepage
xfs 0xffffffffc054e724
Build id: 3.10.0-1160.el7.x86_64 		0x494
13
write_cache_pages
vmlinux 0xffffffffb89c9d8c
Build id: 3.10.0-1160.el7.x86_64 		0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc054e290
Build id: 3.10.0-1160.el7.x86_64 		0xa0
15
? __enqueue_entity
vmlinux 0xffffffffb88dfb48
Build id: 3.10.0-1160.el7.x86_64 		0x78
16
? put_prev_entity
vmlinux 0xffffffffb88e5f1b
Build id: 3.10.0-1160.el7.x86_64 		0x8b
17
? __switch_to
vmlinux 0xffffffffb882b59e
Build id: 3.10.0-1160.el7.x86_64 		0xce
18
xfs_vm_writepages
xfs 0xffffffffc054e25b
Build id: 3.10.0-1160.el7.x86_64 		0x6b
19
do_writepages
vmlinux 0xffffffffb89cade1
Build id: 3.10.0-1160.el7.x86_64 		0x21
20
__writeback_single_inode
vmlinux 0xffffffffb8a7d6f0
Build id: 3.10.0-1160.el7.x86_64 		0x40
21
writeback_sb_inodes
vmlinux 0xffffffffb8a7e274
Build id: 3.10.0-1160.el7.x86_64 		0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffffb8a7e57f
Build id: 3.10.0-1160.el7.x86_64 		0x9f
23
wb_writeback
vmlinux 0xffffffffb8a7ea53
Build id: 3.10.0-1160.el7.x86_64 		0x263
24
bdi_writeback_workfn
vmlinux 0xffffffffb8a7f64b
Build id: 3.10.0-1160.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffffb88bdc4f
Build id: 3.10.0-1160.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffffb88bed66
Build id: 3.10.0-1160.el7.x86_64 		0x126
27
? manage_workers.isra.26
vmlinux 0xffffffffb88bec40
Build id: 3.10.0-1160.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffffb88c5c21
Build id: 3.10.0-1160.el7.x86_64 		0xd1
29
? insert_kthread_work
vmlinux 0xffffffffb88c5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffffb8f93df7
Build id: 3.10.0-1160.el7.x86_64 		0x21
31
? insert_kthread_work
vmlinux 0xffffffffb88c5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 27s! [kworker/u256:0:5275]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi coretemp snd_ac97_codec ac97_bus snd_seq btusb btrtl snd_seq_device iosf_mbi btbcm btintel crc32_pclmul bluetooth ghash_clmulni_intel snd_pcm aesni_intel lrw ppdev gf128mul snd_timer snd glue_helper soundcore ablk_helper rfkill cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix serio_raw drm libata nfit e1000 mptspi scsi_transport_spi libnvdimm mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 5275 Comm: kworker/u256:0 Kdump: loaded Not tainted 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9f78ddb28000 ti: ffff9f78b4e08000 task.ti: ffff9f78b4e08000
RIP: 0010:[<ffffffffc02af3be>]  [<ffffffffc02af3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9f78b4e0b708  EFLAGS: 00000246
RAX: ffffb47340440000 RBX: ffff9f78f34bc000 RCX: 0000000000000005
RDX: ffff9f78b54fc000 RSI: ffff9f78b5af3000 RDI: 000000000000000e
RBP: ffff9f78b4e0b718 R08: 0000000000000002 R09: 000000000da43fff
R10: ffff9f78b4d6d000 R11: ffff9f78cf952d80 R12: ffff9f788e97a700
R13: ffff9f78b4e0b67c R14: ffffffffb8b5bdfb R15: ffff9f78b4e0b6c0
FS:  0000000000000000(0000) GS:ffff9f78fb640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f23d7f40000 CR3: 000000004df2e000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc02d171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02ca2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb8ce3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb8ced0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffffb8b77f8e>] ? deadline_add_request+0x6e/0x90
[<ffffffffb8b534d9>] __blk_run_queue+0x39/0x50
[<ffffffffb8b571e3>] blk_queue_bio+0x3b3/0x400
[<ffffffffb8b55167>] generic_make_request+0x147/0x380
[<ffffffffb8b55410>] submit_bio+0x70/0x150
[<ffffffffc054df31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc054e724>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffb89c9d8c>] write_cache_pages+0x21c/0x470
[<ffffffffc054e290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffb88dfb48>] ? __enqueue_entity+0x78/0x80
[<ffffffffb88e5f1b>] ? put_prev_entity+0x8b/0x400
[<ffffffffb882b59e>] ? __switch_to+0xce/0x580
[<ffffffffc054e25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb89cade1>] do_writepages+0x21/0x50
[<ffffffffb8a7d6f0>] __writeback_single_inode+0x40/0x260
[<ffffffffb8a7e274>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb8a7e57f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb8a7ea53>] wb_writeback+0x263/0x2f0
[<ffffffffb8a7f64b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb88bdc4f>] process_one_work+0x17f/0x440
[<ffffffffb88bed66>] worker_thread+0x126/0x3c0
[<ffffffffb88bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb88c5c21>] kthread+0xd1/0xe0
[<ffffffffb88c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb8f93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb88c5b50>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 2b
Complete report #409469
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffbd36a0f5
Build id: 3.10.0-957.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc03ba622
Build id: 3.10.0-957.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffbd0d8e90
Build id: 3.10.0-957.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffbd0d0960
Build id: 3.10.0-957.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffbd0d9efc
Build id: 3.10.0-957.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffffbcf675fe
Build id: 3.10.0-957.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffbcf42ff9
Build id: 3.10.0-957.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffbcf46cf3
Build id: 3.10.0-957.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffbcf44c87
Build id: 3.10.0-957.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffbcf44f30
Build id: 3.10.0-957.el7.x86_64 		0x70
11
? __switch_to
vmlinux 0xffffffffbcc2a59e
Build id: 3.10.0-957.el7.x86_64 		0xce
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc04e5251
Build id: 3.10.0-957.el7.x86_64 		0x61
13
xfs_vm_writepages
xfs 0xffffffffc04e53bf
Build id: 3.10.0-957.el7.x86_64 		0x7f
14
do_writepages
vmlinux 0xffffffffbcdc3181
Build id: 3.10.0-957.el7.x86_64 		0x21
15
__writeback_single_inode
vmlinux 0xffffffffbce6f930
Build id: 3.10.0-957.el7.x86_64 		0x40
16
writeback_sb_inodes
vmlinux 0xffffffffbce703c4
Build id: 3.10.0-957.el7.x86_64 		0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffbce7072f
Build id: 3.10.0-957.el7.x86_64 		0x9f
18
wb_writeback
vmlinux 0xffffffffbce70f63
Build id: 3.10.0-957.el7.x86_64 		0x263
19
bdi_writeback_workfn
vmlinux 0xffffffffbce718eb
Build id: 3.10.0-957.el7.x86_64 		0x2cb
20
process_one_work
vmlinux 0xffffffffbccb9d4f
Build id: 3.10.0-957.el7.x86_64 		0x17f
21
worker_thread
vmlinux 0xffffffffbccbade6
Build id: 3.10.0-957.el7.x86_64 		0x126
22
? manage_workers.isra.25
vmlinux 0xffffffffbccbacc0
Build id: 3.10.0-957.el7.x86_64 		0x2a0
23
kthread
vmlinux 0xffffffffbccc1c31
Build id: 3.10.0-957.el7.x86_64 		0xd1
24
? insert_kthread_work
vmlinux 0xffffffffbccc1b60
Build id: 3.10.0-957.el7.x86_64 		0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffffbd374c1d
Build id: 3.10.0-957.el7.x86_64 		0x7
26
? insert_kthread_work
vmlinux 0xffffffffbccc1b60
Build id: 3.10.0-957.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [kworker/u16:2:17936]
Modules linked in: tcp_diag inet_diag rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_set nf_tables(T) ip6t_MASQUERADE nf_nat_masquerade_ipv6 xt_comment xt_mark ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack_netlink xt_addrtype xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 nf_log_ipv4 ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common xt_LOG xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables veth vxlan ip6_udp_tunnel udp_tunnel
ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport dummy br_netfilter tun devlink binfmt_misc overlay(T) ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr joydev vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper vmxnet3 ahci syscopyarea sysfillrect sysimgblt fb_sys_fops vmw_pvscsi libahci ttm nfit libnvdimm ata_piix drm libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 3 PID: 17936 Comm: kworker/u16:2 Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff900484040000 ti: ffff900485de4000 task.ti: ffff900485de4000
RIP: 0010:[<ffffffffbd36a0f5>]  [<ffffffffbd36a0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff900485de78f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffbcfa10f3 RCX: ffff90063f763060
RDX: ffffb1a780ea0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff900485de78f0 R08: 0000000000000000 R09: ffff9005810cc700
R10: ffff90063fd6c8d8 R11: ffff900435cea800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9004b72c5200 R15: ffffffffbd0d7be9
FS:  0000000000000000(0000) GS:ffff90063fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007faf75f8b950 CR3: 000000014d7b8000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc03ba622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffbd0d8e90>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffbd0d0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbd0d9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffbcf675fe>] ? deadline_add_request+0x6e/0x90
[<ffffffffbcf42ff9>] __blk_run_queue+0x39/0x50
[<ffffffffbcf46cf3>] blk_queue_bio+0x3b3/0x400
[<ffffffffbcf44c87>] generic_make_request+0x147/0x380
[<ffffffffbcf44f30>] submit_bio+0x70/0x150
[<ffffffffbcc2a59e>] ? __switch_to+0xce/0x580
[<ffffffffc04e5251>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04e53bf>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffbcdc3181>] do_writepages+0x21/0x50
[<ffffffffbce6f930>] __writeback_single_inode+0x40/0x260
[<ffffffffbce703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbce7072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbce70f63>] wb_writeback+0x263/0x2f0
[<ffffffffbce718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbccb9d4f>] process_one_work+0x17f/0x440
[<ffffffffbccbade6>] worker_thread+0x126/0x3c0
[<ffffffffbccbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffbccc1c31>] kthread+0xd1/0xe0
[<ffffffffbccc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd374c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffbccc1b60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #372660
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00823be
Build id: 3.10.0-693.21.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc005e72e
Build id: 3.10.0-693.21.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00692b0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146d8f0
Build id: 3.10.0-693.21.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff8147678f
Build id: 3.10.0-693.21.1.el7.x86_64 		0x4df
6
__blk_run_queue
vmlinux 0xffffffff812fd1d9
Build id: 3.10.0-693.21.1.el7.x86_64 		0x39
7
blk_queue_bio
vmlinux 0xffffffff81300b33
Build id: 3.10.0-693.21.1.el7.x86_64 		0x363
8
generic_make_request
vmlinux 0xffffffff812feb7b
Build id: 3.10.0-693.21.1.el7.x86_64 		0x10b
9
submit_bio
vmlinux 0xffffffff812fee00
Build id: 3.10.0-693.21.1.el7.x86_64 		0x70
10
? __switch_to
vmlinux 0xffffffff8102954d
Build id: 3.10.0-693.21.1.el7.x86_64 		0xcd
11
xfs_submit_ioend.isra.16
xfs 0xffffffffc0283071
Build id: 3.10.0-693.21.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc0283231
Build id: 3.10.0-693.21.1.el7.x86_64 		0xd1
13
do_writepages
vmlinux 0xffffffff81193531
Build id: 3.10.0-693.21.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff812326d0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff81233344
Build id: 3.10.0-693.21.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff812336af
Build id: 3.10.0-693.21.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff81233ee3
Build id: 3.10.0-693.21.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8123438b
Build id: 3.10.0-693.21.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff810abe2f
Build id: 3.10.0-693.21.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff810acaf6
Build id: 3.10.0-693.21.1.el7.x86_64 		0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff810ac9d0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff810b4031
Build id: 3.10.0-693.21.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64 		0x40
24
ret_from_fork
vmlinux 0xffffffff816c0577
Build id: 3.10.0-693.21.1.el7.x86_64 		0x77
25
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/u256:2:84064]
Modules linked in: nls_utf8 isofs fuse vmw_vsock_vmci_transport vsock ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter snd_seq_midi snd_seq_midi_event sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper snd_ens1371 snd_rawmidi snd_ac97_codec ablk_helper cryptd ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore ppdev vmw_balloon joydev sg pcspkr i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata e1000 mptspi scsi_transport_spi serio_raw mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 84064 Comm: kworker/u256:2 Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff8803e4791fa0 ti: ffff8800a3b54000 task.ti: ffff8800a3b54000
RIP: 0010:[<ffffffffc00823be>]  [<ffffffffc00823be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8800a3b57890  EFLAGS: 00000246
RAX: ffffc90001940000 RBX: ffff88041bbe2f00 RCX: 0000000000000002
RDX: ffff8804270b7e00 RSI: ffff88042f56e000 RDI: 000000000000000e
RBP: ffff8800a3b578a0 R08: 0000000000000002 R09: ffff88042e27f0d8
R10: ffff8804276d8a80 R11: ffff8804276d8a80 R12: 0000000100000001
R13: ffff8800a3a0ad00 R14: ffff8800a3b57804 R15: ffffffff813051fb
FS:  0000000000000000(0000) GS:ffff88042d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2dce056480 CR3: 000000041f3a4000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc005e72e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00692b0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8146d8f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8147678f>] scsi_request_fn+0x4df/0x680
[<ffffffff812fd1d9>] __blk_run_queue+0x39/0x50
[<ffffffff81300b33>] blk_queue_bio+0x363/0x3a0
[<ffffffff812feb7b>] generic_make_request+0x10b/0x320
[<ffffffff812fee00>] submit_bio+0x70/0x150
[<ffffffff8102954d>] ? __switch_to+0xcd/0x500
[<ffffffffc0283071>] xfs_submit_ioend.isra.16+0x61/0xe0 [xfs]
[<ffffffffc0283231>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff81193531>] do_writepages+0x21/0x50
[<ffffffff812326d0>] __writeback_single_inode+0x40/0x250
[<ffffffff81233344>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff812336af>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff81233ee3>] wb_writeback+0x263/0x2f0
[<ffffffff8123438b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810abe2f>] process_one_work+0x17f/0x440
[<ffffffff810acaf6>] worker_thread+0x126/0x3c0
[<ffffffff810ac9d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b4031>] kthread+0xd1/0xe0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 09
Complete report #328157
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01753be
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc01ce71e
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02202e0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffac4e4860
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffac4edf8c
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffffac378bee
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffac353f39
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffac357ce3
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffac355bc7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffac355e70
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x70
11
? __enqueue_entity
vmlinux 0xffffffffac0dfbd8
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x78
12
? put_prev_entity
vmlinux 0xffffffffac0e637b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x8b
13
xfs_submit_ioend.isra.12
xfs 0xffffffffc0410f31
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x61
14
xfs_vm_writepages
xfs 0xffffffffc041126f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x7f
15
do_writepages
vmlinux 0xffffffffac1cb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
16
__writeback_single_inode
vmlinux 0xffffffffac27df10
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
17
writeback_sb_inodes
vmlinux 0xffffffffac27ea94
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffffac27ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x9f
19
wb_writeback
vmlinux 0xffffffffac27f273
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x263
20
bdi_writeback_workfn
vmlinux 0xffffffffac27fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2cb
21
process_one_work
vmlinux 0xffffffffac0bde8f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffffac0befa6
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffac0bee80
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffffac0c5e61
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffffac0c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffac795df7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
27
? insert_kthread_work
vmlinux 0xffffffffac0c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [kworker/u256:2:19367]
Modules linked in: rfcomm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat talpa_vfshook(OE) iptable_mangle talpa_pedconnector(OE) iptable_security talpa_pedevice(OE) iptable_raw talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) nf_conntrack talpa_syscallhook(OE) ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi snd_ens1371 crc32_pclmul ghash_clmulni_intel aesni_intel snd_rawmidi snd_ac97_codec lrw ac97_bus gf128mul btusb snd_seq btrtl snd_seq_device
btbcm btintel glue_helper snd_pcm bluetooth ablk_helper cryptd rfkill snd_timer snd soundcore joydev ppdev sg pcspkr vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi nfit ata_piix drm libnvdimm crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel libata mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 19367 Comm: kworker/u256:2 Kdump: loaded Tainted: P           OEL ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff948cfc032100 ti: ffff948d7aa7c000 task.ti: ffff948d7aa7c000
RIP: 0010:[<ffffffffc01753be>]  [<ffffffffc01753be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff948d7aa7f888  EFLAGS: 00000246
RAX: ffffaf34c1940000 RBX: ffff948ceacac000 RCX: 0000000000000013
RDX: ffff948e0cb80e00 RSI: ffff948e0d3d7000 RDI: 000000000000000e
RBP: ffff948d7aa7f898 R08: 0000000000000002 R09: 00000001d9547fff
R10: ffff948e0bc6d400 R11: ffff948d2a7101c0 R12: ffff948ba2285c80
R13: ffff948d7aa7f7fc R14: ffffffffac35c8fb R15: ffff948d7aa7f840
FS:  0000000000000000(0000) GS:ffff948e11b00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa9a01ce000 CR3: 00000002d096c000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc01ce71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02202e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffac4e4860>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffac4edf8c>] scsi_request_fn+0x4ac/0x680
[<ffffffffac378bee>] ? deadline_add_request+0x6e/0x90
[<ffffffffac353f39>] __blk_run_queue+0x39/0x50
[<ffffffffac357ce3>] blk_queue_bio+0x3b3/0x400
[<ffffffffac355bc7>] generic_make_request+0x147/0x380
[<ffffffffac355e70>] submit_bio+0x70/0x150
[<ffffffffac0dfbd8>] ? __enqueue_entity+0x78/0x80
[<ffffffffac0e637b>] ? put_prev_entity+0x8b/0x400
[<ffffffffc0410f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc041126f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffac1cb9b1>] do_writepages+0x21/0x50
[<ffffffffac27df10>] __writeback_single_inode+0x40/0x260
[<ffffffffac27ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffac27ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffac27f273>] wb_writeback+0x263/0x2f0
[<ffffffffac27fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffac0bde8f>] process_one_work+0x17f/0x440
[<ffffffffac0befa6>] worker_thread+0x126/0x3c0
[<ffffffffac0bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffac0c5e61>] kthread+0xd1/0xe0
[<ffffffffac0c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffac795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffac0c5d90>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 33 18
Complete report #264372
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8d98b4d5
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc03e5632
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8d6ec6b0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8d6e3fa0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff8d6ed6cc
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8d57853e
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff8d553939
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff8d557743
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff8d5555c7
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff8d555870
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0626f31
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc062726f
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffff8d3cb001
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff8d47d9a0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8d47e524
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8d47e82f
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff8d47ed03
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8d47f8fb
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff8d2bddcf
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff8d2beee6
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff8d2bedc0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff8d2c5da1
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8d2c5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8d995df7
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffff8d2c5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 30s! [kworker/u4:30:390]
Modules linked in: talpa_vfshook(OE) talpa_pedconnector(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) tcp_diag inet_diag talpa_pedevice(OE) vmw_vsock_vmci_transport vsock nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack xt_multiport iptable_filter sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper sg ablk_helper vmw_balloon cryptd pcspkr joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx ahci libahci ata_piix crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel serio_raw syscopyarea sysfillrect libata sysimgblt fb_sys_fops ttm vmxnet3 drm vmw_pvscsi nfit drm_panel_orientation_quirks libnvdimm
dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: talpa_syscallhook]
CPU: 1 PID: 390 Comm: kworker/u4:30 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8afcf5801080 ti: ffff8afcf580c000 task.ti: ffff8afcf580c000
RIP: 0010:[<ffffffff8d98b4d5>]  [<ffffffff8d98b4d5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8afcf580f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8d5b28d3 RCX: ffff8afeffcb9060
RDX: fffface680f40000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8afcf580f8f0 R08: 0000000000000000 R09: ffff8afccf7fbb00
R10: ffff8afef5dab000 R11: ffff8afcf660e800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8afcf59a2600 R15: ffffffff8d6eb369
FS:  0000000000000000(0000) GS:ffff8afeffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb8fb775000 CR3: 000000017fef6000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc03e5632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8d6ec6b0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8d6e3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8d6ed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffff8d57853e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8d553939>] __blk_run_queue+0x39/0x50
[<ffffffff8d557743>] blk_queue_bio+0x3b3/0x400
[<ffffffff8d5555c7>] generic_make_request+0x147/0x380
[<ffffffff8d555870>] submit_bio+0x70/0x150
[<ffffffffc0626f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc062726f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff8d3cb001>] do_writepages+0x21/0x50
[<ffffffff8d47d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffff8d47e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8d47e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8d47ed03>] wb_writeback+0x263/0x2f0
[<ffffffff8d47f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8d2bddcf>] process_one_work+0x17f/0x440
[<ffffffff8d2beee6>] worker_thread+0x126/0x3c0
[<ffffffff8d2bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8d2c5da1>] kthread+0xd1/0xe0
[<ffffffff8d2c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8d995df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8d2c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #229782
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04873be
Build id: 3.10.0-862.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc04d671e
Build id: 3.10.0-862.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03e72e0
Build id: 3.10.0-862.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9fe9b050
Build id: 3.10.0-862.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff9fea45bc
Build id: 3.10.0-862.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffff9fd3c84e
Build id: 3.10.0-862.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff9fd18ad9
Build id: 3.10.0-862.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff9fd1c763
Build id: 3.10.0-862.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff9fd1a6bb
Build id: 3.10.0-862.el7.x86_64 		0x10b
10
submit_bio
vmlinux 0xffffffff9fd1a940
Build id: 3.10.0-862.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc06784b1
Build id: 3.10.0-862.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc0678671
Build id: 3.10.0-862.el7.x86_64 		0xd1
13
do_writepages
vmlinux 0xffffffff9fba0e21
Build id: 3.10.0-862.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff9fc48af0
Build id: 3.10.0-862.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff9fc49584
Build id: 3.10.0-862.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff9fc498ef
Build id: 3.10.0-862.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff9fc4a123
Build id: 3.10.0-862.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9fc4a9ac
Build id: 3.10.0-862.el7.x86_64 		0x1cc
19
process_one_work
vmlinux 0xffffffff9fab2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff9fab3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff9fab39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff9fabae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa011f637
Build id: 3.10.0-862.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 66s! [kworker/u128:0:2698]
Modules linked in: esn_cfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul
syscopyarea crct10dif_common sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2698 Comm: kworker/u128:0 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff88709bea5ee0 ti: ffff887064dec000 task.ti: ffff887064dec000
RIP: 0010:[<ffffffffc04873be>]  [<ffffffffc04873be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff887064def888  EFLAGS: 00000246
RAX: ffffb42600380000 RBX: ffff8870b5afe000 RCX: 0000000000000017
RDX: ffff887075b7a200 RSI: ffff887075b88000 RDI: 000000000000000e
RBP: ffff887064def898 R08: 0000000000000002 R09: 000000001d15efff
R10: ffff887075df10d8 R11: ffff887075b7d880 R12: ffff8870614c0000
R13: ffff887064def7fc R14: ffffffff9fd212cb R15: ffff887064def840
FS:  0000000000000000(0000) GS:ffff8870bc6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b67eed090 CR3: 00000000785fa000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc04d671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03e72e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9fe9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9fea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff9fd3c84e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9fd18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff9fd1c763>] blk_queue_bio+0x3b3/0x400
[<ffffffff9fd1a6bb>] generic_make_request+0x10b/0x320
[<ffffffff9fd1a940>] submit_bio+0x70/0x150
[<ffffffffc06784b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0678671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff9fba0e21>] do_writepages+0x21/0x50
[<ffffffff9fc48af0>] __writeback_single_inode+0x40/0x260
[<ffffffff9fc49584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff9fc498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9fc4a123>] wb_writeback+0x263/0x2f0
[<ffffffff9fc4a9ac>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff9fab2dff>] process_one_work+0x17f/0x440
[<ffffffff9fab3ac6>] worker_thread+0x126/0x3c0
[<ffffffff9fab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff9fabae31>] kthread+0xd1/0xe0
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa011f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 49
Complete report #187898
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8a78b495
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0179632
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8a4ec580
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8a4e3e70
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff8a4ed59c
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8a37840e
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff8a353809
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff8a357613
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff8a355497
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff8a355740
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc03d5f31
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc03d626f
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffff8a1caef1
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff8a27d870
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8a27e3f4
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8a27e6ff
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff8a27ebd3
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8a27f7cb
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff8a0bddcf
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff8a0beee6
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff8a0bedc0
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff8a0c5da1
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8a0c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a795df7
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffff8a0c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 28s! [kworker/u25:0:14007]
Modules linked in: vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr sg joydev vmw_balloon parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect sysimgblt crc_t10dif fb_sys_fops crct10dif_generic ttm drm ata_piix crct10dif_pclmul crct10dif_common nfit libata crc32c_intel serio_raw libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 14007 Comm: kworker/u25:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8edff95b4200 ti: ffff8edff0acc000 task.ti: ffff8edff0acc000
RIP: 0010:[<ffffffff8a78b495>]  [<ffffffff8a78b495>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8edff0acf8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8a3b27a3 RCX: ffffffff8ac34f40
RDX: ffffb0a081bd0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8edff0acf8f0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8edff7ea9800 R11: ffff8edff9d3b000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8edfbfda8900 R15: ffffffff8a4eb239
FS:  0000000000000000(0000) GS:ffff8edff9d40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fce6b2ca4b1 CR3: 000000031fa10000 CR4: 00000000000407e0
Call Trace:
[<ffffffffc0179632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8a4ec580>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8a4e3e70>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8a4ed59c>] scsi_request_fn+0x4ac/0x680
[<ffffffff8a37840e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8a353809>] __blk_run_queue+0x39/0x50
[<ffffffff8a357613>] blk_queue_bio+0x3b3/0x400
[<ffffffff8a355497>] generic_make_request+0x147/0x380
[<ffffffff8a355740>] submit_bio+0x70/0x150
[<ffffffffc03d5f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc03d626f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff8a1caef1>] do_writepages+0x21/0x50
[<ffffffff8a27d870>] __writeback_single_inode+0x40/0x260
[<ffffffff8a27e3f4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8a27e6ff>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8a27ebd3>] wb_writeback+0x263/0x2f0
[<ffffffff8a27f7cb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8a0bddcf>] process_one_work+0x17f/0x440
[<ffffffff8a0beee6>] worker_thread+0x126/0x3c0
[<ffffffff8a0bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8a0c5da1>] kthread+0xd1/0xe0
[<ffffffff8a0c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8a0c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #148712
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb858b4d5
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0121632
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb82ec6b0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb82e3fa0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffb82ed6cc
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb817853e
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffb8153939
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffb8157743
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffb81555c7
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffb8155870
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0367f31
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x61
12
xfs_do_writepage
xfs 0xffffffffc0368724
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x494
13
write_cache_pages
vmlinux 0xffffffffb7fc9fac
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc0368290
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xa0
15
? pick_next_entity
vmlinux 0xffffffffb7ee2499
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xa9
16
? set_next_entity
vmlinux 0xffffffffb7ee273c
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x3c
17
? __switch_to
vmlinux 0xffffffffb7e2b59e
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xce
18
xfs_vm_writepages
xfs 0xffffffffc036825b
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x6b
19
do_writepages
vmlinux 0xffffffffb7fcb001
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x21
20
__writeback_single_inode
vmlinux 0xffffffffb807d9a0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40
21
writeback_sb_inodes
vmlinux 0xffffffffb807e524
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffffb807e82f
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x9f
23
wb_writeback
vmlinux 0xffffffffb807ed03
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x263
24
bdi_writeback_workfn
vmlinux 0xffffffffb807f8fb
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffffb7ebddcf
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffffb7ebeee6
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x126
27
? manage_workers.isra.26
vmlinux 0xffffffffb7ebedc0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffffb7ec5da1
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xd1
29
? insert_kthread_work
vmlinux 0xffffffffb7ec5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffffb8595df7
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x21
31
? insert_kthread_work
vmlinux 0xffffffffb7ec5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/u4:0:31140]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd pcspkr vmw_balloon joydev sg parport_pc i2c_piix4 parport vmw_vmci binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif vmwgfx crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel drm serio_raw
libata nfit libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 31140 Comm: kworker/u4:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9e6ab6a9c200 ti: ffff9e6c6e6ec000 task.ti: ffff9e6c6e6ec000
RIP: 0010:[<ffffffffb858b4d5>]  [<ffffffffb858b4d5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9e6c6e6ef770  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffffb81b28d3 RCX: ffff9e6ba9fae060
RDX: ffffb3cd01be0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff9e6c6e6ef770 R08: 0000000000000000 R09: ffff9e6ac76bcd00
R10: ffff9e6d716f7c00 R11: ffff9e6d71062800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9e6c7ea1c480 R15: ffffffffb82eb369
FS:  0000000000000000(0000) GS:ffff9e6d7fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ef569f369 CR3: 00000001e49a8000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc0121632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffb82ec6b0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffb82e3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb82ed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffffb817853e>] ? deadline_add_request+0x6e/0x90
[<ffffffffb8153939>] __blk_run_queue+0x39/0x50
[<ffffffffb8157743>] blk_queue_bio+0x3b3/0x400
[<ffffffffb81555c7>] generic_make_request+0x147/0x380
[<ffffffffb8155870>] submit_bio+0x70/0x150
[<ffffffffc0367f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc0368724>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffb7fc9fac>] write_cache_pages+0x21c/0x470
[<ffffffffc0368290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffb7ee2499>] ? pick_next_entity+0xa9/0x190
[<ffffffffb7ee273c>] ? set_next_entity+0x3c/0xe0
[<ffffffffb7e2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc036825b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb7fcb001>] do_writepages+0x21/0x50
[<ffffffffb807d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffffb807e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb807e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb807ed03>] wb_writeback+0x263/0x2f0
[<ffffffffb807f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb7ebddcf>] process_one_work+0x17f/0x440
[<ffffffffb7ebeee6>] worker_thread+0x126/0x3c0
[<ffffffffb7ebedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb7ec5da1>] kthread+0xd1/0xe0
[<ffffffffb7ec5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb8595df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb7ec5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #71185
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa196b455
Build id: 3.10.0-957.21.3.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0333622
Build id: 3.10.0-957.21.3.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffa16d9c30
Build id: 3.10.0-957.21.3.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffa16d1700
Build id: 3.10.0-957.21.3.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffa16dac9c
Build id: 3.10.0-957.21.3.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffffa1567e6e
Build id: 3.10.0-957.21.3.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffa1543849
Build id: 3.10.0-957.21.3.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffa1547553
Build id: 3.10.0-957.21.3.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffa15454e7
Build id: 3.10.0-957.21.3.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffa1545790
Build id: 3.10.0-957.21.3.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc056f471
Build id: 3.10.0-957.21.3.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc056f5df
Build id: 3.10.0-957.21.3.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffffa13c36d1
Build id: 3.10.0-957.21.3.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffffa14701b0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffffa1470c44
Build id: 3.10.0-957.21.3.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffa1470faf
Build id: 3.10.0-957.21.3.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffffa14717e3
Build id: 3.10.0-957.21.3.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffa147206c
Build id: 3.10.0-957.21.3.el7.x86_64 		0x1cc
19
process_one_work
vmlinux 0xffffffffa12b9ebf
Build id: 3.10.0-957.21.3.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffffa12baf56
Build id: 3.10.0-957.21.3.el7.x86_64 		0x126
21
? manage_workers.isra.25
vmlinux 0xffffffffa12bae30
Build id: 3.10.0-957.21.3.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffffa12c1da1
Build id: 3.10.0-957.21.3.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffffa12c1cd0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa1975c1d
Build id: 3.10.0-957.21.3.el7.x86_64 		0x7
25
? insert_kthread_work
vmlinux 0xffffffffa12c1cd0
Build id: 3.10.0-957.21.3.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/u4:3:30439]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_11110(E) falcon_lsm_pinned_10807(E) binfmt_misc falcon_lsm_pinned_10703(E) falcon_lsm_pinned_10404(E) falcon_lsm_pinned_9918(E) falcon_lsm_pinned_9808(E) falcon_lsm_pinned_9705(E) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock falcon_lsm_pinned_9607(E) sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables
xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper ahci syscopyarea sysfillrect sysimgblt fb_sys_fops libahci ttm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm libata vmxnet3 nfit vmw_pvscsi drm_panel_orientation_quirks libnvdimm floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_lsm_serviceable]
CPU: 1 PID: 30439 Comm: kworker/u4:3 Tainted: P            E  ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff89049de35140 ti: ffff89048a124000 task.ti: ffff89048a124000
RIP: 0010:[<ffffffffa196b455>]  [<ffffffffa196b455>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff89048a1278f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffa16d89fe RCX: 0000000000001000
RDX: ffffb456803f0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff89048a1278f0 R08: ffffffffffffffff R09: ffff8904895a6a00
R10: ffff8904b5ce40d8 R11: ffff8904b5a59800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8904a1f62600 R15: ffffffffa16d8989
FS:  0000000000000000(0000) GS:ffff8904ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5b48df90e0 CR3: 0000000008082000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc0333622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffa16d9c30>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffa16d1700>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa16dac9c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa1567e6e>] ? deadline_add_request+0x6e/0x90
[<ffffffffa1543849>] __blk_run_queue+0x39/0x50
[<ffffffffa1547553>] blk_queue_bio+0x3b3/0x400
[<ffffffffa15454e7>] generic_make_request+0x147/0x380
[<ffffffffa1545790>] submit_bio+0x70/0x150
[<ffffffffc056f471>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc056f5df>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffa13c36d1>] do_writepages+0x21/0x50
[<ffffffffa14701b0>] __writeback_single_inode+0x40/0x260
[<ffffffffa1470c44>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa1470faf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa14717e3>] wb_writeback+0x263/0x2f0
[<ffffffffa147206c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffa12b9ebf>] process_one_work+0x17f/0x440
[<ffffffffa12baf56>] worker_thread+0x126/0x3c0
[<ffffffffa12bae30>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa12c1da1>] kthread+0xd1/0xe0
[<ffffffffa12c1cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa1975c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa12c1cd0>] ? insert_kthread_work+0x40/0x40
Complete report #58249
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc040e3be
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc048f71e
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04862e0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c4e2990
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c4ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8c3771ae
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff8c3527a9
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff8c3564b3
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff8c354437
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff8c3546e0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0626fb1
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc06272ef
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffff8c1cb261
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff8c27cc40
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8c27d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8c27dacf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff8c27dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8c27eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff8c0be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff8c0bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff8c0bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff8c0c6691
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8c0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8c792d1d
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x7
25
? insert_kthread_work
vmlinux 0xffffffff8c0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/u256:0:4298]
Modules linked in: xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmhgfs(OE) vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit sg libnvdimm pcspkr joydev vmw_balloon parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm ata_piix mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi mptscsih libata mptbase crc32c_intel e1000 drm_panel_orientation_quirks serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 4298 Comm: kworker/u256:0 Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff90d130de1070 ti: ffff90d176864000 task.ti: ffff90d176864000
RIP: 0010:[<ffffffffc040e3be>]  [<ffffffffc040e3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff90d176867888  EFLAGS: 00000246
RAX: ffffb780c0700000 RBX: ffff90d176fed000 RCX: 000000000000000d
RDX: ffff90d0f58dda00 RSI: ffff90d0f5d22000 RDI: 000000000000000e
RBP: ffff90d176867898 R08: 0000000000000002 R09: 000000005880dfff
R10: ffff90d1f1b60c00 R11: ffff90d0f5d281c0 R12: ffff90d13e94d200
R13: ffff90d1768677fc R14: ffffffff8c35b0cb R15: ffff90d176867840
FS:  0000000000000000(0000) GS:ffff90d1f9600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f49e99c2118 CR3: 0000000080090000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc048f71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04862e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8c4e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c4ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff8c3771ae>] ? deadline_add_request+0x6e/0x90
[<ffffffff8c3527a9>] __blk_run_queue+0x39/0x50
[<ffffffff8c3564b3>] blk_queue_bio+0x3b3/0x400
[<ffffffff8c354437>] generic_make_request+0x147/0x380
[<ffffffff8c3546e0>] submit_bio+0x70/0x150
[<ffffffffc0626fb1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc06272ef>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff8c1cb261>] do_writepages+0x21/0x50
[<ffffffff8c27cc40>] __writeback_single_inode+0x40/0x260
[<ffffffff8c27d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8c27dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8c27dfa3>] wb_writeback+0x263/0x2f0
[<ffffffff8c27eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8c0be6bf>] process_one_work+0x17f/0x440
[<ffffffff8c0bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff8c0bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8c0c6691>] kthread+0xd1/0xe0
[<ffffffff8c0c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8c792d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff8c0c65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 c3 41
Complete report #20754
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8c916475
Build id: 3.10.0-862.3.2.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc02ab622
Build id: 3.10.0-862.3.2.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8c6a39e0
Build id: 3.10.0-862.3.2.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c69b4f0
Build id: 3.10.0-862.3.2.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c6a4a5c
Build id: 3.10.0-862.3.2.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffff8c53cc8e
Build id: 3.10.0-862.3.2.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff8c518f19
Build id: 3.10.0-862.3.2.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff8c51cba3
Build id: 3.10.0-862.3.2.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff8c51aafb
Build id: 3.10.0-862.3.2.el7.x86_64 		0x10b
10
submit_bio
vmlinux 0xffffffff8c51ad80
Build id: 3.10.0-862.3.2.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc04df4b1
Build id: 3.10.0-862.3.2.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc04df671
Build id: 3.10.0-862.3.2.el7.x86_64 		0xd1
13
do_writepages
vmlinux 0xffffffff8c3a1161
Build id: 3.10.0-862.3.2.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff8c448e30
Build id: 3.10.0-862.3.2.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8c4498c4
Build id: 3.10.0-862.3.2.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8c449c2f
Build id: 3.10.0-862.3.2.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff8c44a463
Build id: 3.10.0-862.3.2.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8c44adeb
Build id: 3.10.0-862.3.2.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff8c2b312f
Build id: 3.10.0-862.3.2.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff8c2b3df6
Build id: 3.10.0-862.3.2.el7.x86_64 		0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff8c2b3cd0
Build id: 3.10.0-862.3.2.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff8c2bb161
Build id: 3.10.0-862.3.2.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8c2bb090
Build id: 3.10.0-862.3.2.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8c920677
Build id: 3.10.0-862.3.2.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffff8c2bb090
Build id: 3.10.0-862.3.2.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 50s! [kworker/u2:0:24987]
Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache binfmt_misc vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev nfit sg libnvdimm parport_pc shpchp parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic drm crct10dif_pclmul ata_piix crct10dif_common crc32c_intel serio_raw libata vmw_pvscsi i2c_core vmxnet3 floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 24987 Comm: kworker/u2:0 Kdump: loaded Tainted: G             L ------------   3.10.0-862.3.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff94a0d1aa1fa0 ti: ffff94a107f6c000 task.ti: ffff94a107f6c000
RIP: 0010:[<ffffffff8c916475>]  [<ffffffff8c916475>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff94a107f6f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8c574e63 RCX: ffff94a0cc09a060
RDX: ffffb3d841b70000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff94a107f6f8f0 R08: 0000000000000000 R09: ffff94a1276ec700
R10: ffff94a12ba0e8d8 R11: ffff94a12ba0f000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff94a12a238480 R15: ffffffff8c6a2739
FS:  0000000000000000(0000) GS:ffff94a13fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9e2cd76270 CR3: 00000000b2efc000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc02ab622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8c6a39e0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8c69b4f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c6a4a5c>] scsi_request_fn+0x4cc/0x680
[<ffffffff8c53cc8e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8c518f19>] __blk_run_queue+0x39/0x50
[<ffffffff8c51cba3>] blk_queue_bio+0x3b3/0x400
[<ffffffff8c51aafb>] generic_make_request+0x10b/0x320
[<ffffffff8c51ad80>] submit_bio+0x70/0x150
[<ffffffffc04df4b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc04df671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff8c3a1161>] do_writepages+0x21/0x50
[<ffffffff8c448e30>] __writeback_single_inode+0x40/0x260
[<ffffffff8c4498c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8c449c2f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8c44a463>] wb_writeback+0x263/0x2f0
[<ffffffff8c44adeb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8c2b312f>] process_one_work+0x17f/0x440
[<ffffffff8c2b3df6>] worker_thread+0x126/0x3c0
[<ffffffff8c2b3cd0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff8c2bb161>] kthread+0xd1/0xe0
[<ffffffff8c2bb090>] ? insert_kthread_work+0x40/0x40
[<ffffffff8c920677>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8c2bb090>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #18678
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00743be
Build id: 3.10.0-693.21.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc00f672e
Build id: 3.10.0-693.21.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00de2b0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146d8f0
Build id: 3.10.0-693.21.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff8147678f
Build id: 3.10.0-693.21.1.el7.x86_64 		0x4df
6
__blk_run_queue
vmlinux 0xffffffff812fd1d9
Build id: 3.10.0-693.21.1.el7.x86_64 		0x39
7
blk_queue_bio
vmlinux 0xffffffff81300b33
Build id: 3.10.0-693.21.1.el7.x86_64 		0x363
8
generic_make_request
vmlinux 0xffffffff812feb7b
Build id: 3.10.0-693.21.1.el7.x86_64 		0x10b
9
submit_bio
vmlinux 0xffffffff812fee00
Build id: 3.10.0-693.21.1.el7.x86_64 		0x70
10
xfs_submit_ioend.isra.16
xfs 0xffffffffc0276071
Build id: 3.10.0-693.21.1.el7.x86_64 		0x61
11
xfs_vm_writepages
xfs 0xffffffffc0276231
Build id: 3.10.0-693.21.1.el7.x86_64 		0xd1
12
do_writepages
vmlinux 0xffffffff81193531
Build id: 3.10.0-693.21.1.el7.x86_64 		0x21
13
__writeback_single_inode
vmlinux 0xffffffff812326d0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x40
14
writeback_sb_inodes
vmlinux 0xffffffff81233344
Build id: 3.10.0-693.21.1.el7.x86_64 		0x1c4
15
__writeback_inodes_wb
vmlinux 0xffffffff812336af
Build id: 3.10.0-693.21.1.el7.x86_64 		0x9f
16
wb_writeback
vmlinux 0xffffffff81233ee3
Build id: 3.10.0-693.21.1.el7.x86_64 		0x263
17
bdi_writeback_workfn
vmlinux 0xffffffff8123438b
Build id: 3.10.0-693.21.1.el7.x86_64 		0x2cb
18
process_one_work
vmlinux 0xffffffff810abe2f
Build id: 3.10.0-693.21.1.el7.x86_64 		0x17f
19
worker_thread
vmlinux 0xffffffff810acaf6
Build id: 3.10.0-693.21.1.el7.x86_64 		0x126
20
? manage_workers.isra.24
vmlinux 0xffffffff810ac9d0
Build id: 3.10.0-693.21.1.el7.x86_64 		0x2a0
21
kthread
vmlinux 0xffffffff810b4031
Build id: 3.10.0-693.21.1.el7.x86_64 		0xd1
22
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64 		0x40
23
ret_from_fork
vmlinux 0xffffffff816c0577
Build id: 3.10.0-693.21.1.el7.x86_64 		0x77
24
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 79s! [kworker/u4:0:24852]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif
sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix scsi_transport_spi drm crct10dif_pclmul mptscsih crct10dif_common libata crc32c_intel mptbase serio_raw vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 24852 Comm: kworker/u4:0 Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 10/22/2013
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff88005b662f70 ti: ffff880049c7c000 task.ti: ffff880049c7c000
RIP: 0010:[<ffffffffc00743be>]  [<ffffffffc00743be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880049c7f890  EFLAGS: 00000246
RAX: ffffc90000440000 RBX: ffff880077ea7900 RCX: 000000000000001d
RDX: ffff880035e9fa00 RSI: ffff880035019000 RDI: 000000000000000e
RBP: ffff880049c7f8a0 R08: 0000000000000002 R09: 0000000073658fff
R10: ffff880005c9a700 R11: ffff880005c9a700 R12: 0000000100000001
R13: ffff880029c9e180 R14: ffff880049c7f804 R15: ffffffff813051fb
FS:  0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f0478635000 CR3: 000000005365e000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc00f672e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00de2b0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8146d8f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8147678f>] scsi_request_fn+0x4df/0x680
[<ffffffff812fd1d9>] __blk_run_queue+0x39/0x50
[<ffffffff81300b33>] blk_queue_bio+0x363/0x3a0
[<ffffffff812feb7b>] generic_make_request+0x10b/0x320
[<ffffffff812fee00>] submit_bio+0x70/0x150
[<ffffffffc0276071>] xfs_submit_ioend.isra.16+0x61/0xe0 [xfs]
[<ffffffffc0276231>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff81193531>] do_writepages+0x21/0x50
[<ffffffff812326d0>] __writeback_single_inode+0x40/0x250
[<ffffffff81233344>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff812336af>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff81233ee3>] wb_writeback+0x263/0x2f0
[<ffffffff8123438b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810abe2f>] process_one_work+0x17f/0x440
[<ffffffff810acaf6>] worker_thread+0x126/0x3c0
[<ffffffff810ac9d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b4031>] kthread+0xd1/0xe0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 08
Complete report #4725
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02b83be
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02ef71e
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc032b2e0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff85ee2990
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff85eec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x4ac
6
? deadline_add_request
vmlinux 0xffffffff85d771ae
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff85d527a9
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff85d564b3
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff85d54437
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff85d546e0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc052efb1
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc052f2ef
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffff85bcb261
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff85c7cc40
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff85c7d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff85c7dacf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff85c7dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff85c7eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff85abe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff85abf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff85abf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff85ac6691
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff86192d37
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [kworker/u64:0:15179]
Modules linked in: arc4 md4 nls_utf8 cifs ccm dns_resolver fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel ata_piix serio_raw drm libata mptspi scsi_transport_spi nfit libnvdimm mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmfsrtx_bridge_helper]
CPU: 0 PID: 15179 Comm: kworker/u64:0 Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff92a577b15230 ti: ffff92a55028c000 task.ti: ffff92a55028c000
RIP: 0010:[<ffffffffc02b83be>]  [<ffffffffc02b83be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff92a55028f888  EFLAGS: 00000246
RAX: ffffae5d407c0000 RBX: ffff92a5754093b0 RCX: 000000000000001d
RDX: ffff92a475499400 RSI: ffff92a573aba000 RDI: 000000000000000e
RBP: ffff92a55028f898 R08: 0000000000000003 R09: 000000008a36cfff
R10: ffff92a4754a1400 R11: ffff92a4b9822a00 R12: ffff92a4a3fe0300
R13: ffff92a55028f7fc R14: ffffffff85d5b0cb R15: ffff92a55028f840
FS:  0000000000000000(0000) GS:ffff92a57ae00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc7d39a6000 CR3: 0000000079b58000 CR4: 00000000000007f0
Call Trace:
[<ffffffffc02ef71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc032b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff85ee2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff85eec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff85d771ae>] ? deadline_add_request+0x6e/0x90
[<ffffffff85d527a9>] __blk_run_queue+0x39/0x50
[<ffffffff85d564b3>] blk_queue_bio+0x3b3/0x400
[<ffffffff85d54437>] generic_make_request+0x147/0x380
[<ffffffff85d546e0>] submit_bio+0x70/0x150
[<ffffffffc052efb1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc052f2ef>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff85bcb261>] do_writepages+0x21/0x50
[<ffffffff85c7cc40>] __writeback_single_inode+0x40/0x260
[<ffffffff85c7d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff85c7dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff85c7dfa3>] wb_writeback+0x263/0x2f0
[<ffffffff85c7eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff85abe6bf>] process_one_work+0x17f/0x440
[<ffffffff85abf7d6>] worker_thread+0x126/0x3c0
[<ffffffff85abf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff85ac6691>] kthread+0xd1/0xe0
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff86192d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 2c
Complete report #4154
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03c43be
Build id: 3.10.0-957.27.2.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03e671e
Build id: 3.10.0-957.27.2.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03b42e0
Build id: 3.10.0-957.27.2.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9fed2ee0
Build id: 3.10.0-957.27.2.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff9fedc47c
Build id: 3.10.0-957.27.2.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffff9fd694de
Build id: 3.10.0-957.27.2.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffff9fd44e59
Build id: 3.10.0-957.27.2.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffff9fd48b63
Build id: 3.10.0-957.27.2.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffff9fd46af7
Build id: 3.10.0-957.27.2.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffff9fd46da0
Build id: 3.10.0-957.27.2.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc064c471
Build id: 3.10.0-957.27.2.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc064c5df
Build id: 3.10.0-957.27.2.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffff9fbc47a1
Build id: 3.10.0-957.27.2.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff9fc71020
Build id: 3.10.0-957.27.2.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffff9fc71ab4
Build id: 3.10.0-957.27.2.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff9fc71e1f
Build id: 3.10.0-957.27.2.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffff9fc72653
Build id: 3.10.0-957.27.2.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9fc72fdb
Build id: 3.10.0-957.27.2.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffff9fabaf9f
Build id: 3.10.0-957.27.2.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffff9fabc036
Build id: 3.10.0-957.27.2.el7.x86_64 		0x126
21
? manage_workers.isra.25
vmlinux 0xffffffff9fabbf10
Build id: 3.10.0-957.27.2.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffff9fac2e81
Build id: 3.10.0-957.27.2.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9fac2db0
Build id: 3.10.0-957.27.2.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa0176c37
Build id: 3.10.0-957.27.2.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffff9fac2db0
Build id: 3.10.0-957.27.2.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 36s! [kworker/u128:2:55375]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_hda_codec_generic sb_edac snd_hda_intel coretemp snd_hda_codec iosf_mbi crc32_pclmul snd_hda_core ghash_clmulni_intel snd_hwdep snd_seq ppdev aesni_intel snd_seq_device vmw_balloon lrw gf128mul
glue_helper ablk_helper snd_pcm cryptd joydev snd_timer pcspkr sg snd soundcore vmw_vmci i2c_piix4 parport_pc parport xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul e1000 crct10dif_common libata crc32c_intel mptspi scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks floppy [last unloaded: ip_tables]
CPU: 7 PID: 55375 Comm: kworker/u128:2 Tainted: G        W    L ------------   3.10.0-957.27.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff94da2449c100 ti: ffff94da66624000 task.ti: ffff94da66624000
RIP: 0010:[<ffffffffc03c43be>]  [<ffffffffc03c43be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff94da66627888  EFLAGS: 00000246
RAX: ffffb330c0740000 RBX: ffff94da5e235020 RCX: 000000000000001c
RDX: ffff94d9f594b800 RSI: ffff94d9f5d55000 RDI: 000000000000000e
RBP: ffff94da66627898 R08: 0000000000000003 R09: 0000000062fb3fff
R10: ffff94da77e790d8 R11: ffff94da461a2300 R12: ffff94da2433ad00
R13: ffff94da666277fc R14: ffffffff9fd4d6ab R15: ffff94da66627840
FS:  0000000000000000(0000) GS:ffff94da7b5c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f31fe0a3000 CR3: 00000000ba8fa000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc03e671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03b42e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9fed2ee0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9fedc47c>] scsi_request_fn+0x4cc/0x680
[<ffffffff9fd694de>] ? deadline_add_request+0x6e/0x90
[<ffffffff9fd44e59>] __blk_run_queue+0x39/0x50
[<ffffffff9fd48b63>] blk_queue_bio+0x3b3/0x400
[<ffffffff9fd46af7>] generic_make_request+0x147/0x380
[<ffffffff9fd46da0>] submit_bio+0x70/0x150
[<ffffffffc064c471>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc064c5df>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9fbc47a1>] do_writepages+0x21/0x50
[<ffffffff9fc71020>] __writeback_single_inode+0x40/0x260
[<ffffffff9fc71ab4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff9fc71e1f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9fc72653>] wb_writeback+0x263/0x2f0
[<ffffffff9fc72fdb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9fabaf9f>] process_one_work+0x17f/0x440
[<ffffffff9fabc036>] worker_thread+0x126/0x3c0
[<ffffffff9fabbf10>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff9fac2e81>] kthread+0xd1/0xe0
[<ffffffff9fac2db0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa0176c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fac2db0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 3d
Complete report #1229
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03cf3be
Build id: 3.10.0-862.2.3.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03b171e
Build id: 3.10.0-862.2.3.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03ea2e0
Build id: 3.10.0-862.2.3.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa2a9aff0
Build id: 3.10.0-862.2.3.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffa2aa455c
Build id: 3.10.0-862.2.3.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffffa293c7ee
Build id: 3.10.0-862.2.3.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffa2918a79
Build id: 3.10.0-862.2.3.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffa291c703
Build id: 3.10.0-862.2.3.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffa291a65b
Build id: 3.10.0-862.2.3.el7.x86_64 		0x10b
10
submit_bio
vmlinux 0xffffffffa291a8e0
Build id: 3.10.0-862.2.3.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc06014b1
Build id: 3.10.0-862.2.3.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc0601671
Build id: 3.10.0-862.2.3.el7.x86_64 		0xd1
13
do_writepages
vmlinux 0xffffffffa27a0dc1
Build id: 3.10.0-862.2.3.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffffa2848a90
Build id: 3.10.0-862.2.3.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffffa2849524
Build id: 3.10.0-862.2.3.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffa284988f
Build id: 3.10.0-862.2.3.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffffa284a0c3
Build id: 3.10.0-862.2.3.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffa284a94c
Build id: 3.10.0-862.2.3.el7.x86_64 		0x1cc
19
process_one_work
vmlinux 0xffffffffa26b2dff
Build id: 3.10.0-862.2.3.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffffa26b3ac6
Build id: 3.10.0-862.2.3.el7.x86_64 		0x126
21
? manage_workers.isra.24
vmlinux 0xffffffffa26b39a0
Build id: 3.10.0-862.2.3.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffffa26bae31
Build id: 3.10.0-862.2.3.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffffa26bad60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa2d1f5f7
Build id: 3.10.0-862.2.3.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffffa26bad60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 24s! [kworker/u8:2:27541]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr nfit joydev vmw_balloon sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih libata e1000 crct10dif_pclmul crct10dif_common mptbase crc32c_intel i2c_core serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 27541 Comm: kworker/u8:2 Kdump: loaded Tainted: G             L ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8b12e96cdee0 ti: ffff8b11dc1c4000 task.ti: ffff8b11dc1c4000
RIP: 0010:[<ffffffffc03cf3be>]  [<ffffffffc03cf3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8b11dc1c7888  EFLAGS: 00000246
RAX: ffffac7000e80000 RBX: ffff8b12c12179d0 RCX: 000000000000001b
RDX: ffff8b117543c200 RSI: ffff8b12ffdaf000 RDI: 000000000000000e
RBP: ffff8b11dc1c7898 R08: 0000000000000003 R09: 00000001a97e0fff
R10: ffff8b1175f830d8 R11: ffff8b12f7cf6c40 R12: ffff8b12f4662f00
R13: ffff8b11dc1c77fc R14: ffffffffa292126b R15: ffff8b11dc1c7840
FS:  0000000000000000(0000) GS:ffff8b12ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f50aa88a980 CR3: 0000000017cb4000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc03b171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03ea2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa2a9aff0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa2aa455c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa293c7ee>] ? deadline_add_request+0x6e/0x90
[<ffffffffa2918a79>] __blk_run_queue+0x39/0x50
[<ffffffffa291c703>] blk_queue_bio+0x3b3/0x400
[<ffffffffa291a65b>] generic_make_request+0x10b/0x320
[<ffffffffa291a8e0>] submit_bio+0x70/0x150
[<ffffffffc06014b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0601671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffffa27a0dc1>] do_writepages+0x21/0x50
[<ffffffffa2848a90>] __writeback_single_inode+0x40/0x260
[<ffffffffa2849524>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa284988f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa284a0c3>] wb_writeback+0x263/0x2f0
[<ffffffffa284a94c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffa26b2dff>] process_one_work+0x17f/0x440
[<ffffffffa26b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffa26b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffa26bae31>] kthread+0xd1/0xe0
[<ffffffffa26bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa2d1f5f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa26bad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 3d
Complete report #910
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffbd56a175
Build id: 3.10.0-957.1.3.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc00b3622
Build id: 3.10.0-957.1.3.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffbd2d9100
Build id: 3.10.0-957.1.3.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffbd2d0bd0
Build id: 3.10.0-957.1.3.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffbd2da16c
Build id: 3.10.0-957.1.3.el7.x86_64 		0x4cc
6
? deadline_add_request
vmlinux 0xffffffffbd16786e
Build id: 3.10.0-957.1.3.el7.x86_64 		0x6e
7
__blk_run_queue
vmlinux 0xffffffffbd143269
Build id: 3.10.0-957.1.3.el7.x86_64 		0x39
8
blk_queue_bio
vmlinux 0xffffffffbd146f63
Build id: 3.10.0-957.1.3.el7.x86_64 		0x3b3
9
generic_make_request
vmlinux 0xffffffffbd144ef7
Build id: 3.10.0-957.1.3.el7.x86_64 		0x147
10
submit_bio
vmlinux 0xffffffffbd1451a0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0334251
Build id: 3.10.0-957.1.3.el7.x86_64 		0x61
12
xfs_vm_writepages
xfs 0xffffffffc03343bf
Build id: 3.10.0-957.1.3.el7.x86_64 		0x7f
13
do_writepages
vmlinux 0xffffffffbcfc33f1
Build id: 3.10.0-957.1.3.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffffbd06fba0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40
15
writeback_sb_inodes
vmlinux 0xffffffffbd070634
Build id: 3.10.0-957.1.3.el7.x86_64 		0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffbd07099f
Build id: 3.10.0-957.1.3.el7.x86_64 		0x9f
17
wb_writeback
vmlinux 0xffffffffbd0711d3
Build id: 3.10.0-957.1.3.el7.x86_64 		0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffbd071b5b
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2cb
19
process_one_work
vmlinux 0xffffffffbceb9d4f
Build id: 3.10.0-957.1.3.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffffbcebade6
Build id: 3.10.0-957.1.3.el7.x86_64 		0x126
21
? manage_workers.isra.25
vmlinux 0xffffffffbcebacc0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffffbcec1c31
Build id: 3.10.0-957.1.3.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffffbcec1b60
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffbd574c37
Build id: 3.10.0-957.1.3.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffffbcec1b60
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u4:0:19800]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc ppdev sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw e1000e ptp pps_core ata_generic pata_acpi vmwgfx drm_kms_helper nfit ahci syscopyarea
libnvdimm sysfillrect sysimgblt fb_sys_fops ttm libahci drm ata_piix libata vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 19800 Comm: kworker/u4:0 Kdump: loaded Not tainted 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8da59ad88000 ti: ffff8da92adfc000 task.ti: ffff8da92adfc000
RIP: 0010:[<ffffffffbd56a175>]  [<ffffffffbd56a175>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8da92adff8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffbd1a1363 RCX: ffff8da9bfcbe060
RDX: ffffba0ec1be0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8da92adff8f0 R08: 0000000000000000 R09: ffff8da5bf97eb00
R10: ffff8da9282f80d8 R11: ffff8da5b5c68000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8da915fc4a80 R15: ffffffffbd2d7e59
FS:  0000000000000000(0000) GS:ffff8da9bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f268819e6f8 CR3: 00000003bdb16000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc00b3622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffbd2d9100>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffbd2d0bd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbd2da16c>] scsi_request_fn+0x4cc/0x680
[<ffffffffbd16786e>] ? deadline_add_request+0x6e/0x90
[<ffffffffbd143269>] __blk_run_queue+0x39/0x50
[<ffffffffbd146f63>] blk_queue_bio+0x3b3/0x400
[<ffffffffbd144ef7>] generic_make_request+0x147/0x380
[<ffffffffbd1451a0>] submit_bio+0x70/0x150
[<ffffffffc0334251>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc03343bf>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffbcfc33f1>] do_writepages+0x21/0x50
[<ffffffffbd06fba0>] __writeback_single_inode+0x40/0x260
[<ffffffffbd070634>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbd07099f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbd0711d3>] wb_writeback+0x263/0x2f0
[<ffffffffbd071b5b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbceb9d4f>] process_one_work+0x17f/0x440
[<ffffffffbcebade6>] worker_thread+0x126/0x3c0
[<ffffffffbcebacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffbcec1c31>] kthread+0xd1/0xe0
[<ffffffffbcec1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd574c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbcec1b60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48