Info

Permalink
Function
mpt_put_msg_frame
First occurrence
2020-09-22
Last occurrence
2025-12-20
State
NEW
Components
kernel
Quality
-49

Statistics

Operating system Count
CentOS 7 4,634
Architecture Count
x86_64 4,634
Related packages Count
kernel 4,634
 0:3.10.0-1160.15.2.el7 912
 0:3.10.0-693.5.2.el7 692
 0:3.10.0-1160.53.1.el7 553
 0:3.10.0-1127.19.1.el7 210
 0:3.10.0-862.el7 184
 0:3.10.0-1062.12.1.el7 160
 0:3.10.0-957.el7 133
 0:3.10.0-957.12.2.el7 124
 0:3.10.0-957.27.2.el7 108
 0:3.10.0-1062.el7 102
 0:3.10.0-1127.el7 96
 0:3.10.0-1160.el7 93
 0:3.10.0-1160.108.1.el7 88
 0:3.10.0-1062.18.1.el7 72
 0:3.10.0-1160.31.1.el7 66
 0:3.10.0-1160.62.1.el7 65
 0:3.10.0-1160.25.1.el7 60
 0:3.10.0-1160.118.1.el7 54
 0:3.10.0-1160.71.1.el7 48
 0:3.10.0-957.21.3.el7 48
 0:3.10.0-1160.6.1.el7 47
 0:3.10.0-693.el7 46
 0:3.10.0-1160.49.1.el7 44
 0:3.10.0-862.2.3.el7 44
 0:3.10.0-1062.1.2.el7 42
 0:3.10.0-1160.83.1.el7 35
 0:3.10.0-1160.42.2.el7 35
 0:3.10.0-1160.21.1.el7 34
 0:3.10.0-1160.45.1.el7 31
 0:3.10.0-1160.80.1.el7 31
 0:3.10.0-862.6.3.el7 29
 0:3.10.0-1127.10.1.el7 26
 0:3.10.0-1160.119.1.el7 22
 0:3.10.0-1160.76.1.el7 20
 0:3.10.0-1160.88.1.el7 20
 0:3.10.0-1160.24.1.el7 19
 0:3.10.0-1160.41.1.el7 17
 0:3.10.0-1160.11.1.el7 16
 0:3.10.0-862.3.3.el7 16
 0:3.10.0-1127.18.2.el7 15
 0:3.10.0-1160.81.1.el7 15
 0:3.10.0-957.1.3.el7 14
 0:3.10.0-957.5.1.el7 12
 0:3.10.0-1160.66.1.el7 12
 0:3.10.0-1160.36.2.el7 11
 0:3.10.0-1160.59.1.el7 11
 0:3.10.0-862.14.4.el7 11
 0:3.10.0-1062.9.1.el7 9
 0:3.10.0-1160.114.2.el7 8
 0:3.10.0-1160.92.1.el7 7
 0:3.10.0-957.12.1.el7 7
 0:3.10.0-693.21.1.el7 7
 0:3.10.0-1127.8.2.el7 6
 0:3.10.0-1160.90.1.el7 5
 0:3.10.0-1062.4.1.el7 4
 0:3.10.0-1127.13.1.el7 4
 0:3.10.0-693.2.2.el7 4
 0:3.10.0-862.11.6.el7 4
 0:3.10.0-1062.1.1.el7 3
 0:3.10.0-957.10.1.el7 3
 0:3.10.0-957.21.2.el7 3
 0:3.10.0-1160.99.1.el7 3
 0:3.10.0-862.3.2.el7 3
 0:3.10.0-862.9.1.el7 3
 0:3.10.0-1062.4.3.el7 2
 0:3.10.0-1160.105.1.el7 2
 0:3.10.0-693.17.1.el7 2
 0:3.10.0-1160.95.1.el7 1
 0:3.10.0-1160.102.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1229
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03cf3be
Build id: 3.10.0-862.2.3.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03b171e
Build id: 3.10.0-862.2.3.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03ea2e0
Build id: 3.10.0-862.2.3.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa2a9aff0
Build id: 3.10.0-862.2.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa2aa455c
Build id: 3.10.0-862.2.3.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffa293c7ee
Build id: 3.10.0-862.2.3.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa2918a79
Build id: 3.10.0-862.2.3.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa291c703
Build id: 3.10.0-862.2.3.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa291a65b
Build id: 3.10.0-862.2.3.el7.x86_64
0x10b
10
submit_bio
vmlinux 0xffffffffa291a8e0
Build id: 3.10.0-862.2.3.el7.x86_64
0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc06014b1
Build id: 3.10.0-862.2.3.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc0601671
Build id: 3.10.0-862.2.3.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffffa27a0dc1
Build id: 3.10.0-862.2.3.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffa2848a90
Build id: 3.10.0-862.2.3.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffffa2849524
Build id: 3.10.0-862.2.3.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffa284988f
Build id: 3.10.0-862.2.3.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffffa284a0c3
Build id: 3.10.0-862.2.3.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffa284a94c
Build id: 3.10.0-862.2.3.el7.x86_64
0x1cc
19
process_one_work
vmlinux 0xffffffffa26b2dff
Build id: 3.10.0-862.2.3.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffa26b3ac6
Build id: 3.10.0-862.2.3.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffffa26b39a0
Build id: 3.10.0-862.2.3.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffa26bae31
Build id: 3.10.0-862.2.3.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffa26bad60
Build id: 3.10.0-862.2.3.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa2d1f5f7
Build id: 3.10.0-862.2.3.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffa26bad60
Build id: 3.10.0-862.2.3.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 24s! [kworker/u8:2:27541]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr nfit joydev vmw_balloon sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih libata e1000 crct10dif_pclmul crct10dif_common mptbase crc32c_intel i2c_core serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 27541 Comm: kworker/u8:2 Kdump: loaded Tainted: G             L ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8b12e96cdee0 ti: ffff8b11dc1c4000 task.ti: ffff8b11dc1c4000
RIP: 0010:[<ffffffffc03cf3be>]  [<ffffffffc03cf3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8b11dc1c7888  EFLAGS: 00000246
RAX: ffffac7000e80000 RBX: ffff8b12c12179d0 RCX: 000000000000001b
RDX: ffff8b117543c200 RSI: ffff8b12ffdaf000 RDI: 000000000000000e
RBP: ffff8b11dc1c7898 R08: 0000000000000003 R09: 00000001a97e0fff
R10: ffff8b1175f830d8 R11: ffff8b12f7cf6c40 R12: ffff8b12f4662f00
R13: ffff8b11dc1c77fc R14: ffffffffa292126b R15: ffff8b11dc1c7840
FS:  0000000000000000(0000) GS:ffff8b12ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f50aa88a980 CR3: 0000000017cb4000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc03b171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03ea2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa2a9aff0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa2aa455c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa293c7ee>] ? deadline_add_request+0x6e/0x90
[<ffffffffa2918a79>] __blk_run_queue+0x39/0x50
[<ffffffffa291c703>] blk_queue_bio+0x3b3/0x400
[<ffffffffa291a65b>] generic_make_request+0x10b/0x320
[<ffffffffa291a8e0>] submit_bio+0x70/0x150
[<ffffffffc06014b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0601671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffffa27a0dc1>] do_writepages+0x21/0x50
[<ffffffffa2848a90>] __writeback_single_inode+0x40/0x260
[<ffffffffa2849524>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa284988f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa284a0c3>] wb_writeback+0x263/0x2f0
[<ffffffffa284a94c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffa26b2dff>] process_one_work+0x17f/0x440
[<ffffffffa26b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffa26b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffa26bae31>] kthread+0xd1/0xe0
[<ffffffffa26bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa2d1f5f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa26bad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 3d 
Complete report #1322161
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc026244e
Build id: 3.10.0-1160.118.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc024574e
Build id: 3.10.0-1160.118.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02172f0
Build id: 3.10.0-1160.118.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9c103030
Build id: 3.10.0-1160.118.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9c10cbdc
Build id: 3.10.0-1160.118.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9bf8e21e
Build id: 3.10.0-1160.118.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9bf687d9
Build id: 3.10.0-1160.118.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9bf6c703
Build id: 3.10.0-1160.118.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9bf6a537
Build id: 3.10.0-1160.118.1.el7.x86_64
0x157
10
submit_bio
vmlinux 0xffffffff9bf6a7e0
Build id: 3.10.0-1160.118.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc04aa341
Build id: 3.10.0-1160.118.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc04aa68f
Build id: 3.10.0-1160.118.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff9bdd6b01
Build id: 3.10.0-1160.118.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff9be8cd40
Build id: 3.10.0-1160.118.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff9be8d90c
Build id: 3.10.0-1160.118.1.el7.x86_64
0x1cc
16
__writeback_inodes_wb
vmlinux 0xffffffff9be8dc17
Build id: 3.10.0-1160.118.1.el7.x86_64
0xa7
17
wb_writeback
vmlinux 0xffffffff9be8e0f3
Build id: 3.10.0-1160.118.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9be8ebfc
Build id: 3.10.0-1160.118.1.el7.x86_64
0x1cc
19
process_one_work
vmlinux 0xffffffff9bcc32ef
Build id: 3.10.0-1160.118.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff9bcc4436
Build id: 3.10.0-1160.118.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff9bcc4310
Build id: 3.10.0-1160.118.1.el7.x86_64
0x2b0
22
kthread
vmlinux 0xffffffff9bccb621
Build id: 3.10.0-1160.118.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9bccb550
Build id: 3.10.0-1160.118.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff9c3c51f7
Build id: 3.10.0-1160.118.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff9bccb550
Build id: 3.10.0-1160.118.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 25s! [kworker/2:0:22028]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [kworker/u8:0:17737]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_18108(E) falcon_lsm_pinned_18003(E) cts 8021q garp mrp ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache falcon_lsm_pinned_17804(E) nf_nat_tftp nf_conntrack_tftp nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter
overlay(T) tun vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi nfit libata scsi_transport_spi mptscsih libnvdimm mptbase crct10dif_pclmul crct10dif_common vmxnet3 crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 0 PID: 17737 Comm: kworker/u8:0 Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:3)
task: ffff90dcf4683180 ti: ffff90dc82084000 task.ti: ffff90dc82084000
RIP: 0010:[<ffffffffc026244e>]  [<ffffffffc026244e>] mpt_put_msg_frame+0x5e/0x90 [mptbase]
RSP: 0018:ffff90dc82087888  EFLAGS: 00000246
RAX: ffffa871c0e80000 RBX: ffff90db7b131000 RCX: 000000000000003f
RDX: ffff90dcf2785a00 RSI: ffff90dcf2743000 RDI: 000000000000000e
RBP: ffff90dc82087898 R08: 0000000000000002 R09: 000000010d6f7fff
R10: ffff90daf5cddc00 R11: ffff90dcf459d180 R12: ffff90dc7ac9d500
R13: ffff90dc820877fc R14: ffffffff9bf7158b R15: ffff90dc82087840
FS:  0000000000000000(0000) GS:ffff90dcffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000012989fc CR3: 0000000213d04000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc024574e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02172f0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9c103030>] scsi_dispatch_cmd+0xb0/0x250
[<ffffffff9c10cbdc>] scsi_request_fn+0x4ac/0x680
[<ffffffff9bf8e21e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9bf687d9>] __blk_run_queue+0x39/0x50
[<ffffffff9bf6c703>] blk_queue_bio+0x3b3/0x400
[<ffffffff9bf6a537>] generic_make_request+0x157/0x390
[<ffffffff9bf6a7e0>] submit_bio+0x70/0x150
[<ffffffffc04aa341>] xfs_submit_ioend.isra.12+0x61/0xf0 [xfs]
[<ffffffffc04aa68f>] xfs_vm_writepages+0x7f/0xb0 [xfs]
[<ffffffff9bdd6b01>] do_writepages+0x21/0x50
[<ffffffff9be8cd40>] __writeback_single_inode+0x40/0x260
[<ffffffff9be8d90c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff9be8dc17>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff9be8e0f3>] wb_writeback+0x263/0x2f0
[<ffffffff9be8ebfc>] bdi_writeback_workfn+0x1cc/0x470
[<ffffffff9bcc32ef>] process_one_work+0x17f/0x440
[<ffffffff9bcc4436>] worker_thread+0x126/0x3c0
[<ffffffff9bcc4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff9bccb621>] kthread+0xd1/0xe0
[<ffffffff9bccb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff9c3c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9bccb550>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 14 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 cc cc cc cc 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 
Complete report #1264242
Frame # Function Binary Source or offset Line
1
scsi_request_fn
vmlinux 0xffffffff95d0c315
Build id: 3.10.0-1160.83.1.el7.x86_64
0x135
2
? deadline_add_request
vmlinux 0xffffffff95b8db9e
Build id: 3.10.0-1160.83.1.el7.x86_64
0x6e
3
__blk_run_queue
vmlinux 0xffffffff95b68159
Build id: 3.10.0-1160.83.1.el7.x86_64
0x39
4
blk_queue_bio
vmlinux 0xffffffff95b6c083
Build id: 3.10.0-1160.83.1.el7.x86_64
0x3b3
5
generic_make_request
vmlinux 0xffffffff95b69eb7
Build id: 3.10.0-1160.83.1.el7.x86_64
0x157
6
submit_bio
vmlinux 0xffffffff95b6a160
Build id: 3.10.0-1160.83.1.el7.x86_64
0x70
7
xfs_submit_ioend.isra.12
xfs 0xffffffffc0543341
Build id: 3.10.0-1160.83.1.el7.x86_64
0x61
8
xfs_do_writepage
xfs 0xffffffffc0543b54
Build id: 3.10.0-1160.83.1.el7.x86_64
0x494
9
write_cache_pages
vmlinux 0xffffffff959d587c
Build id: 3.10.0-1160.83.1.el7.x86_64
0x21c
10
? xfs_vm_writepages
xfs 0xffffffffc05436c0
Build id: 3.10.0-1160.83.1.el7.x86_64
0xb0
11
? sched_clock_cpu
vmlinux 0xffffffff958e4235
Build id: 3.10.0-1160.83.1.el7.x86_64
0x85
12
? cpumask_next_and
vmlinux 0xffffffff95b9bf95
Build id: 3.10.0-1160.83.1.el7.x86_64
0x35
13
xfs_vm_writepages
xfs 0xffffffffc054367b
Build id: 3.10.0-1160.83.1.el7.x86_64
0x6b
14
do_writepages
vmlinux 0xffffffff959d6921
Build id: 3.10.0-1160.83.1.el7.x86_64
0x21
15
__writeback_single_inode
vmlinux 0xffffffff95a8c790
Build id: 3.10.0-1160.83.1.el7.x86_64
0x40
16
writeback_sb_inodes
vmlinux 0xffffffff95a8d35c
Build id: 3.10.0-1160.83.1.el7.x86_64
0x1cc
17
__writeback_inodes_wb
vmlinux 0xffffffff95a8d667
Build id: 3.10.0-1160.83.1.el7.x86_64
0xa7
18
wb_writeback
vmlinux 0xffffffff95a8db43
Build id: 3.10.0-1160.83.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffff95a78abc
Build id: 3.10.0-1160.83.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff95a8e74f
Build id: 3.10.0-1160.83.1.el7.x86_64
0x2cf
21
process_one_work
vmlinux 0xffffffff958c31df
Build id: 3.10.0-1160.83.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff958c4326
Build id: 3.10.0-1160.83.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff958c4200
Build id: 3.10.0-1160.83.1.el7.x86_64
0x2b0
24
kthread
vmlinux 0xffffffff958cb511
Build id: 3.10.0-1160.83.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff958cb440
Build id: 3.10.0-1160.83.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff95fc51f7
Build id: 3.10.0-1160.83.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffff958cb440
Build id: 3.10.0-1160.83.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u256:1:124623]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock ext4 mbcache coretemp jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr joydev vmw_balloon sg parport_pc parport vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix ahci drm libahci mptspi scsi_transport_spi crct10dif_pclmul
crct10dif_common mptscsih libata crc32c_intel serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 124623 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.83.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8c2cf3b0d280 ti: ffff8c2cf3244000 task.ti: ffff8c2cf3244000
RIP: 0010:[<ffffffff95d0c315>]  [<ffffffff95d0c315>] scsi_request_fn+0x135/0x680
RSP: 0000:ffff8c2cf3247820  EFLAGS: 00000292
RAX: 0000000000000000 RBX: 0000000000000082 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000082 RDI: ffff8c2af5758450
RBP: ffff8c2cf3247878 R08: ffffffff96936ee8 R09: 0000000000000000
R10: ffff8c2af50b1000 R11: ffff8c2cf428e4c0 R12: ffffffff958b3abc
R13: ffff8c2cf32477d8 R14: 0000000000000082 R15: ffff8c2af5758548
FS:  0000000000000000(0000) GS:ffff8c2cf5600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5d85e91000 CR3: 000000022fd16000 CR4: 00000000001607f0
Call Trace:
[<ffffffff95b8db9e>] ? deadline_add_request+0x6e/0x90
[<ffffffff95b68159>] __blk_run_queue+0x39/0x50
[<ffffffff95b6c083>] blk_queue_bio+0x3b3/0x400
[<ffffffff95b69eb7>] generic_make_request+0x157/0x390
[<ffffffff95b6a160>] submit_bio+0x70/0x150
[<ffffffffc0543341>] xfs_submit_ioend.isra.12+0x61/0xf0 [xfs]
[<ffffffffc0543b54>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff959d587c>] write_cache_pages+0x21c/0x480
[<ffffffffc05436c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffff958e4235>] ? sched_clock_cpu+0x85/0xc0
[<ffffffff95b9bf95>] ? cpumask_next_and+0x35/0x50
[<ffffffffc054367b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffff959d6921>] do_writepages+0x21/0x50
[<ffffffff95a8c790>] __writeback_single_inode+0x40/0x260
[<ffffffff95a8d35c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff95a8d667>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff95a8db43>] wb_writeback+0x263/0x2f0
[<ffffffff95a78abc>] ? get_nr_inodes+0x4c/0x70
[<ffffffff95a8e74f>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffff958c31df>] process_one_work+0x17f/0x440
[<ffffffff958c4326>] worker_thread+0x126/0x3c0
[<ffffffff958c4200>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff958cb511>] kthread+0xd1/0xe0
[<ffffffff958cb440>] ? insert_kthread_work+0x40/0x40
[<ffffffff95fc51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff958cb440>] ? insert_kthread_work+0x40/0x40
Code: 8b 85 40 04 00 00 a8 02 0f 85 18 01 00 00 4c 89 f7 e8 10 f2 e5 ff 49 8b bd 58 04 00 00 c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <4d> 8b be 00 01 00 00 4d 85 ff 0f 84 3d 04 00 00 49 8b 85 40 04 
Complete report #1181857
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9e3ba385
Build id: 3.10.0-1160.119.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc01176a2
Build id: 3.10.0-1160.119.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff9e10bb80
Build id: 3.10.0-1160.119.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff9e103030
Build id: 3.10.0-1160.119.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9e10cbdc
Build id: 3.10.0-1160.119.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9df8e21e
Build id: 3.10.0-1160.119.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9df687d9
Build id: 3.10.0-1160.119.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9df6c703
Build id: 3.10.0-1160.119.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9df6a537
Build id: 3.10.0-1160.119.1.el7.x86_64
0x157
10
submit_bio
vmlinux 0xffffffff9df6a7e0
Build id: 3.10.0-1160.119.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0380341
Build id: 3.10.0-1160.119.1.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc0380b54
Build id: 3.10.0-1160.119.1.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffff9ddd5a5c
Build id: 3.10.0-1160.119.1.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc03806c0
Build id: 3.10.0-1160.119.1.el7.x86_64
0xb0
15
xfs_vm_writepages
xfs 0xffffffffc038067b
Build id: 3.10.0-1160.119.1.el7.x86_64
0x6b
16
do_writepages
vmlinux 0xffffffff9ddd6b01
Build id: 3.10.0-1160.119.1.el7.x86_64
0x21
17
__writeback_single_inode
vmlinux 0xffffffff9de8cd40
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
18
? wake_up_bit
vmlinux 0xffffffff9dccc745
Build id: 3.10.0-1160.119.1.el7.x86_64
0x25
19
writeback_sb_inodes
vmlinux 0xffffffff9de8d90c
Build id: 3.10.0-1160.119.1.el7.x86_64
0x1cc
20
__writeback_inodes_wb
vmlinux 0xffffffff9de8dc17
Build id: 3.10.0-1160.119.1.el7.x86_64
0xa7
21
wb_writeback
vmlinux 0xffffffff9de8e0f3
Build id: 3.10.0-1160.119.1.el7.x86_64
0x263
22
bdi_writeback_workfn
vmlinux 0xffffffff9de8ecff
Build id: 3.10.0-1160.119.1.el7.x86_64
0x2cf
23
process_one_work
vmlinux 0xffffffff9dcc32ef
Build id: 3.10.0-1160.119.1.el7.x86_64
0x17f
24
worker_thread
vmlinux 0xffffffff9dcc4436
Build id: 3.10.0-1160.119.1.el7.x86_64
0x126
25
? manage_workers.isra.26
vmlinux 0xffffffff9dcc4310
Build id: 3.10.0-1160.119.1.el7.x86_64
0x2b0
26
kthread
vmlinux 0xffffffff9dccb621
Build id: 3.10.0-1160.119.1.el7.x86_64
0xd1
27
? insert_kthread_work
vmlinux 0xffffffff9dccb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
28
ret_from_fork_nospec_begin
vmlinux 0xffffffff9e3c51dd
Build id: 3.10.0-1160.119.1.el7.x86_64
0x7
29
? insert_kthread_work
vmlinux 0xffffffff9dccb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u16:1:2471]
Modules linked in: binfmt_misc tcp_diag inet_diag veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon sg joydev pcspkr vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt sd_mod fb_sys_fops crc_t10dif ttm crct10dif_generic ahci nfit libahci drm ata_piix
libnvdimm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 2471 Comm: kworker/u16:1 Tainted: G             L ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8ca9b368d280 ti: ffff8ca95dd80000 task.ti: ffff8ca95dd80000
RIP: 0010:[<ffffffff9e3ba385>]  [<ffffffff9e3ba385>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8ca95dd83770  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff9dfc9ec3 RCX: ffff8caa1f0b9060
RDX: ffffa9b1035b0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8ca95dd83770 R08: 0000000000000000 R09: ffff8ca7d700d800
R10: ffff8caa1f107c00 R11: ffff8caa1a1fd000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8ca8d2b0a300 R15: ffffffff9e10a7b9
FS:  0000000000000000(0000) GS:ffff8caa3fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f50302da000 CR3: 00000007946f4000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffc01176a2>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff9e10bb80>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff9e103030>] scsi_dispatch_cmd+0xb0/0x250
[<ffffffff9e10cbdc>] scsi_request_fn+0x4ac/0x680
[<ffffffff9df8e21e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9df687d9>] __blk_run_queue+0x39/0x50
[<ffffffff9df6c703>] blk_queue_bio+0x3b3/0x400
[<ffffffff9df6a537>] generic_make_request+0x157/0x390
[<ffffffff9df6a7e0>] submit_bio+0x70/0x150
[<ffffffffc0380341>] xfs_submit_ioend.isra.12+0x61/0xf0 [xfs]
[<ffffffffc0380b54>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff9ddd5a5c>] write_cache_pages+0x21c/0x480
[<ffffffffc03806c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffffc038067b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffff9ddd6b01>] do_writepages+0x21/0x50
[<ffffffff9de8cd40>] __writeback_single_inode+0x40/0x260
[<ffffffff9dccc745>] ? wake_up_bit+0x25/0x30
[<ffffffff9de8d90c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff9de8dc17>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff9de8e0f3>] wb_writeback+0x263/0x2f0
[<ffffffff9de8ecff>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffff9dcc32ef>] process_one_work+0x17f/0x440
[<ffffffff9dcc4436>] worker_thread+0x126/0x3c0
[<ffffffff9dcc4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff9dccb621>] kthread+0xd1/0xe0
[<ffffffff9dccb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff9e3c51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff9dccb550>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 cc cc cc cc 0f 1f 44 00 00 55 48 89 e5 48 
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
Complete report #1114983
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04543be
Build id: 3.10.0-1160.36.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc047671e
Build id: 3.10.0-1160.36.2.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc046f2e0
Build id: 3.10.0-1160.36.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff878e4810
Build id: 3.10.0-1160.36.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff878edf3c
Build id: 3.10.0-1160.36.2.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff87778b9e
Build id: 3.10.0-1160.36.2.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff87753ee9
Build id: 3.10.0-1160.36.2.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff87757c93
Build id: 3.10.0-1160.36.2.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff87755b77
Build id: 3.10.0-1160.36.2.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff87755e20
Build id: 3.10.0-1160.36.2.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc06c9f31
Build id: 3.10.0-1160.36.2.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc06ca26f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff875cb991
Build id: 3.10.0-1160.36.2.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8767dec0
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffff874c6ef5
Build id: 3.10.0-1160.36.2.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffff8767ea44
Build id: 3.10.0-1160.36.2.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff8767ed4f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff8767f223
Build id: 3.10.0-1160.36.2.el7.x86_64
0x263
19
bdi_writeback_workfn
vmlinux 0xffffffff8767fe1b
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffff874bde6f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff874bef86
Build id: 3.10.0-1160.36.2.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff874bee60
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff874c5e41
Build id: 3.10.0-1160.36.2.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff874c5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff87b95ddd
Build id: 3.10.0-1160.36.2.el7.x86_64
0x7
26
? insert_kthread_work
vmlinux 0xffffffff874c5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [kworker/u8:2:28279]
Modules linked in: openvswitch xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_17306(E) ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev parport_pc sg parport
i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul sysimgblt crct10dif_common fb_sys_fops crc32c_intel ttm serio_raw ata_piix drm nfit vmxnet3 libnvdimm libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 28279 Comm: kworker/u8:2 Tainted: P            EL ------------   3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff90131cc69080 ti: ffff9011f5c04000 task.ti: ffff9011f5c04000
RIP: 0010:[<ffffffffc04543be>]  [<ffffffffc04543be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9011f5c07888  EFLAGS: 00000246
RAX: ffffaad543600000 RBX: ffff900fc4ada1a0 RCX: 000000000000002c
RDX: ffff9013627fa200 RSI: ffff90134d994000 RDI: 000000000000000e
RBP: ffff9011f5c07898 R08: 0000000000000002 R09: 00000000340f4fff
R10: ffff901360b82400 R11: ffff9013465cbdc0 R12: ffff9012f0e51680
R13: ffff9011f5c077fc R14: ffffffff8775c8ab R15: ffff9011f5c07840
FS:  0000000000000000(0000) GS:ffff90137fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1df909ec60 CR3: 00000006249d0000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc047671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc046f2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff878e4810>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff878edf3c>] scsi_request_fn+0x4ac/0x680
[<ffffffff87778b9e>] ? deadline_add_request+0x6e/0x90
[<ffffffff87753ee9>] __blk_run_queue+0x39/0x50
[<ffffffff87757c93>] blk_queue_bio+0x3b3/0x400
[<ffffffff87755b77>] generic_make_request+0x147/0x380
[<ffffffff87755e20>] submit_bio+0x70/0x150
[<ffffffffc06c9f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc06ca26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff875cb991>] do_writepages+0x21/0x50
[<ffffffff8767dec0>] __writeback_single_inode+0x40/0x260
[<ffffffff874c6ef5>] ? wake_up_bit+0x25/0x30
[<ffffffff8767ea44>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8767ed4f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8767f223>] wb_writeback+0x263/0x2f0
[<ffffffff8767fe1b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff874bde6f>] process_one_work+0x17f/0x440
[<ffffffff874bef86>] worker_thread+0x126/0x3c0
[<ffffffff874bee60>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff874c5e41>] kthread+0xd1/0xe0
[<ffffffff874c5d70>] ? insert_kthread_work+0x40/0x40
[<ffffffff87b95ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff874c5d70>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 46 
Complete report #1017572
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01363be
Build id: 3.10.0-1160.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc012a71e
Build id: 3.10.0-1160.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc015b2e0
Build id: 3.10.0-1160.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c8e3980
Build id: 3.10.0-1160.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c8ed0ac
Build id: 3.10.0-1160.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8c777f8e
Build id: 3.10.0-1160.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8c7534d9
Build id: 3.10.0-1160.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff8c7571e3
Build id: 3.10.0-1160.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff8c755167
Build id: 3.10.0-1160.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff8c755410
Build id: 3.10.0-1160.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0334f31
Build id: 3.10.0-1160.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc0335724
Build id: 3.10.0-1160.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffff8c5c9d8c
Build id: 3.10.0-1160.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc0335290
Build id: 3.10.0-1160.el7.x86_64
0xa0
15
? set_next_entity
vmlinux 0xffffffff8c4e055c
Build id: 3.10.0-1160.el7.x86_64
0x3c
16
? __switch_to
vmlinux 0xffffffff8c42b59e
Build id: 3.10.0-1160.el7.x86_64
0xce
17
xfs_vm_writepages
xfs 0xffffffffc033525b
Build id: 3.10.0-1160.el7.x86_64
0x6b
18
do_writepages
vmlinux 0xffffffff8c5cade1
Build id: 3.10.0-1160.el7.x86_64
0x21
19
__writeback_single_inode
vmlinux 0xffffffff8c67d6f0
Build id: 3.10.0-1160.el7.x86_64
0x40
20
writeback_sb_inodes
vmlinux 0xffffffff8c67e274
Build id: 3.10.0-1160.el7.x86_64
0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffff8c67e57f
Build id: 3.10.0-1160.el7.x86_64
0x9f
22
wb_writeback
vmlinux 0xffffffff8c67ea53
Build id: 3.10.0-1160.el7.x86_64
0x263
23
bdi_writeback_workfn
vmlinux 0xffffffff8c67f54c
Build id: 3.10.0-1160.el7.x86_64
0x1cc
24
process_one_work
vmlinux 0xffffffff8c4bdc4f
Build id: 3.10.0-1160.el7.x86_64
0x17f
25
worker_thread
vmlinux 0xffffffff8c4bed66
Build id: 3.10.0-1160.el7.x86_64
0x126
26
? manage_workers.isra.26
vmlinux 0xffffffff8c4bec40
Build id: 3.10.0-1160.el7.x86_64
0x2a0
27
kthread
vmlinux 0xffffffff8c4c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
28
? insert_kthread_work
vmlinux 0xffffffff8c4c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff8cb93df7
Build id: 3.10.0-1160.el7.x86_64
0x21
30
? insert_kthread_work
vmlinux 0xffffffff8c4c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/u256:0:100620]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm crc32_pclmul snd_timer snd ghash_clmulni_intel ppdev aesni_intel soundcore lrw vmw_balloon gf128mul glue_helper pcspkr ablk_helper sg cryptd joydev parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm crct10dif_pclmul crct10dif_common crc32c_intel e1000 mptspi ata_piix scsi_transport_spi mptscsih mptbase libata nfit serio_raw libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 100620 Comm: kworker/u256:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9f727dea2100 ti: ffff9f71b8d58000 task.ti: ffff9f71b8d58000
RIP: 0010:[<ffffffffc01363be>]  [<ffffffffc01363be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9f71b8d5b708  EFLAGS: 00000246
RAX: ffffc12943480000 RBX: ffff9f70b4d1d000 RCX: 000000000000000f
RDX: ffff9f70b5d9be00 RSI: ffff9f752d52f000 RDI: 000000000000000e
RBP: ffff9f71b8d5b718 R08: 0000000000000002 R09: 000000032258afff
R10: ffff9f70b5c93800 R11: ffff9f70b5fbed80 R12: ffff9f71f47f4300
R13: ffff9f71b8d5b67c R14: ffffffff8c75bdfb R15: ffff9f71b8d5b6c0
FS:  0000000000000000(0000) GS:ffff9f752b640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc3aec04f00 CR3: 000000017e3ba000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc012a71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc015b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8c8e3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c8ed0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffff8c777f8e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8c7534d9>] __blk_run_queue+0x39/0x50
[<ffffffff8c7571e3>] blk_queue_bio+0x3b3/0x400
[<ffffffff8c755167>] generic_make_request+0x147/0x380
[<ffffffff8c755410>] submit_bio+0x70/0x150
[<ffffffffc0334f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc0335724>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff8c5c9d8c>] write_cache_pages+0x21c/0x470
[<ffffffffc0335290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff8c4e055c>] ? set_next_entity+0x3c/0xe0
[<ffffffff8c42b59e>] ? __switch_to+0xce/0x580
[<ffffffffc033525b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff8c5cade1>] do_writepages+0x21/0x50
[<ffffffff8c67d6f0>] __writeback_single_inode+0x40/0x260
[<ffffffff8c67e274>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8c67e57f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8c67ea53>] wb_writeback+0x263/0x2f0
[<ffffffff8c67f54c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff8c4bdc4f>] process_one_work+0x17f/0x440
[<ffffffff8c4bed66>] worker_thread+0x126/0x3c0
[<ffffffff8c4bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8c4c5c21>] kthread+0xd1/0xe0
[<ffffffff8c4c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8cb93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8c4c5b50>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 43 14 
Complete report #986368
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03f03be
Build id: 3.10.0-1127.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc041271e
Build id: 3.10.0-1127.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04622e0
Build id: 3.10.0-1127.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb2ee2850
Build id: 3.10.0-1127.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb2eebefc
Build id: 3.10.0-1127.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb2d7709e
Build id: 3.10.0-1127.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffb2d526f9
Build id: 3.10.0-1127.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffb2d56403
Build id: 3.10.0-1127.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffb2d54387
Build id: 3.10.0-1127.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffb2d54630
Build id: 3.10.0-1127.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc068f071
Build id: 3.10.0-1127.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc068f864
Build id: 3.10.0-1127.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffffb2bca15c
Build id: 3.10.0-1127.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc068f3d0
Build id: 3.10.0-1127.el7.x86_64
0xa0
15
xfs_vm_writepages
xfs 0xffffffffc068f39b
Build id: 3.10.0-1127.el7.x86_64
0x6b
16
do_writepages
vmlinux 0xffffffffb2bcb1b1
Build id: 3.10.0-1127.el7.x86_64
0x21
17
__writeback_single_inode
vmlinux 0xffffffffb2c7cbd0
Build id: 3.10.0-1127.el7.x86_64
0x40
18
? wake_up_bit
vmlinux 0xffffffffb2ac7745
Build id: 3.10.0-1127.el7.x86_64
0x25
19
writeback_sb_inodes
vmlinux 0xffffffffb2c7d764
Build id: 3.10.0-1127.el7.x86_64
0x1c4
20
__writeback_inodes_wb
vmlinux 0xffffffffb2c7da6f
Build id: 3.10.0-1127.el7.x86_64
0x9f
21
wb_writeback
vmlinux 0xffffffffb2c7df53
Build id: 3.10.0-1127.el7.x86_64
0x263
22
? get_nr_inodes
vmlinux 0xffffffffb2c69bac
Build id: 3.10.0-1127.el7.x86_64
0x4c
23
bdi_writeback_workfn
vmlinux 0xffffffffb2c7eb4b
Build id: 3.10.0-1127.el7.x86_64
0x2cb
24
process_one_work
vmlinux 0xffffffffb2abe6bf
Build id: 3.10.0-1127.el7.x86_64
0x17f
25
worker_thread
vmlinux 0xffffffffb2abf7d6
Build id: 3.10.0-1127.el7.x86_64
0x126
26
? manage_workers.isra.26
vmlinux 0xffffffffb2abf6b0
Build id: 3.10.0-1127.el7.x86_64
0x2a0
27
kthread
vmlinux 0xffffffffb2ac6691
Build id: 3.10.0-1127.el7.x86_64
0xd1
28
? insert_kthread_work
vmlinux 0xffffffffb2ac65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffffb3192d37
Build id: 3.10.0-1127.el7.x86_64
0x21
30
? insert_kthread_work
vmlinux 0xffffffffb2ac65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 95s! [kworker/u256:2:7819]
Modules linked in: binfmt_misc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec sb_edac ac97_bus iosf_mbi crc32_pclmul snd_seq ghash_clmulni_intel aesni_intel snd_seq_device snd_pcm lrw gf128mul ppdev glue_helper snd_timer snd vmw_balloon ablk_helper cryptd soundcore pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata e1000 crct10dif_pclmul crct10dif_common nfit crc32c_intel mptspi libnvdimm scsi_transport_spi mptscsih mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 7819 Comm: kworker/u256:2 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff99139474d230 ti: ffff991281ebc000 task.ti: ffff991281ebc000
RIP: 0010:[<ffffffffc03f03be>]  [<ffffffffc03f03be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff991281ebf708  EFLAGS: 00000246
RAX: ffffb75ec09c0000 RBX: ffff99133a1bd000 RCX: 0000000000000005
RDX: ffff9912b67a8400 RSI: ffff9913b2909000 RDI: 000000000000000e
RBP: ffff991281ebf718 R08: 0000000000000002 R09: 00000000b72cdfff
R10: ffff9912b5d7c400 R11: ffff991338741880 R12: ffff9913b3d92d00
R13: ffff991281ebf67c R14: ffffffffb2d5b01b R15: ffff991281ebf6c0
FS:  0000000000000000(0000) GS:ffff9913b9640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff5b94e3000 CR3: 0000000001e1a000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc041271e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04622e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb2ee2850>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb2eebefc>] scsi_request_fn+0x4ac/0x660
[<ffffffffb2d7709e>] ? deadline_add_request+0x6e/0x90
[<ffffffffb2d526f9>] __blk_run_queue+0x39/0x50
[<ffffffffb2d56403>] blk_queue_bio+0x3b3/0x400
[<ffffffffb2d54387>] generic_make_request+0x147/0x380
[<ffffffffb2d54630>] submit_bio+0x70/0x150
[<ffffffffc068f071>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc068f864>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffb2bca15c>] write_cache_pages+0x21c/0x470
[<ffffffffc068f3d0>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffc068f39b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb2bcb1b1>] do_writepages+0x21/0x50
[<ffffffffb2c7cbd0>] __writeback_single_inode+0x40/0x260
[<ffffffffb2ac7745>] ? wake_up_bit+0x25/0x30
[<ffffffffb2c7d764>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb2c7da6f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb2c7df53>] wb_writeback+0x263/0x2f0
[<ffffffffb2c69bac>] ? get_nr_inodes+0x4c/0x70
[<ffffffffb2c7eb4b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb2abe6bf>] process_one_work+0x17f/0x440
[<ffffffffb2abf7d6>] worker_thread+0x126/0x3c0
[<ffffffffb2abf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb2ac6691>] kthread+0xd1/0xe0
[<ffffffffb2ac65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb3192d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb2ac65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 3f 
Complete report #957430
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03383be
Build id: 3.10.0-1127.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc035a71e
Build id: 3.10.0-1127.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03722e0
Build id: 3.10.0-1127.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa36e2850
Build id: 3.10.0-1127.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa36ebefc
Build id: 3.10.0-1127.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffa357709e
Build id: 3.10.0-1127.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa35526f9
Build id: 3.10.0-1127.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa3556403
Build id: 3.10.0-1127.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa3554387
Build id: 3.10.0-1127.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffa3554630
Build id: 3.10.0-1127.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc05c1071
Build id: 3.10.0-1127.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc05c1864
Build id: 3.10.0-1127.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffffa33ca15c
Build id: 3.10.0-1127.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc05c13d0
Build id: 3.10.0-1127.el7.x86_64
0xa0
15
? pick_next_entity
vmlinux 0xffffffffa32e2e89
Build id: 3.10.0-1127.el7.x86_64
0xa9
16
? set_next_entity
vmlinux 0xffffffffa32e103c
Build id: 3.10.0-1127.el7.x86_64
0x3c
17
? __switch_to
vmlinux 0xffffffffa322b59e
Build id: 3.10.0-1127.el7.x86_64
0xce
18
xfs_vm_writepages
xfs 0xffffffffc05c139b
Build id: 3.10.0-1127.el7.x86_64
0x6b
19
do_writepages
vmlinux 0xffffffffa33cb1b1
Build id: 3.10.0-1127.el7.x86_64
0x21
20
__writeback_single_inode
vmlinux 0xffffffffa347cbd0
Build id: 3.10.0-1127.el7.x86_64
0x40
21
writeback_sb_inodes
vmlinux 0xffffffffa347d764
Build id: 3.10.0-1127.el7.x86_64
0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffffa347da6f
Build id: 3.10.0-1127.el7.x86_64
0x9f
23
wb_writeback
vmlinux 0xffffffffa347df53
Build id: 3.10.0-1127.el7.x86_64
0x263
24
bdi_writeback_workfn
vmlinux 0xffffffffa347eb4b
Build id: 3.10.0-1127.el7.x86_64
0x2cb
25
process_one_work
vmlinux 0xffffffffa32be6bf
Build id: 3.10.0-1127.el7.x86_64
0x17f
26
worker_thread
vmlinux 0xffffffffa32bf7d6
Build id: 3.10.0-1127.el7.x86_64
0x126
27
? manage_workers.isra.26
vmlinux 0xffffffffa32bf6b0
Build id: 3.10.0-1127.el7.x86_64
0x2a0
28
kthread
vmlinux 0xffffffffa32c6691
Build id: 3.10.0-1127.el7.x86_64
0xd1
29
? insert_kthread_work
vmlinux 0xffffffffa32c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffffa3992d1d
Build id: 3.10.0-1127.el7.x86_64
0x7
31
? insert_kthread_work
vmlinux 0xffffffffa32c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 107s! [kworker/u256:1:49047]
Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm dns_resolver xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables tun xt_conntrack iptable_filter vmw_vsock_vmci_transport vsock xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_timer vmw_balloon snd sg crc32_pclmul soundcore ghash_clmulni_intel aesni_intel pcspkr lrw gf128mul glue_helper ablk_helper cryptd joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx ata_piix drm_kms_helper syscopyarea
sysfillrect sysimgblt libata fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm nfit e1000 serio_raw libnvdimm mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 49047 Comm: kworker/u256:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff911fba8bd230 ti: ffff911f85268000 task.ti: ffff911f85268000
RIP: 0010:[<ffffffffc03383be>]  [<ffffffffc03383be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff911f8526b708  EFLAGS: 00000246
RAX: ffffae0a83200000 RBX: ffff911b35cec000 RCX: 0000000000000000
RDX: ffff911b35db9200 RSI: ffff91202380a000 RDI: 000000000000000e
RBP: ffff911f8526b718 R08: 0000000000000002 R09: 0000000073306fff
R10: ffff911c772f6400 R11: ffff91202310fd40 R12: ffff911e6ba2ea00
R13: ffff911f8526b67c R14: ffffffffa355b01b R15: ffff911f8526b6c0
FS:  0000000000000000(0000) GS:ffff912029600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe966e3c000 CR3: 0000000510bfe000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc035a71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03722e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa36e2850>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa36ebefc>] scsi_request_fn+0x4ac/0x660
[<ffffffffa357709e>] ? deadline_add_request+0x6e/0x90
[<ffffffffa35526f9>] __blk_run_queue+0x39/0x50
[<ffffffffa3556403>] blk_queue_bio+0x3b3/0x400
[<ffffffffa3554387>] generic_make_request+0x147/0x380
[<ffffffffa3554630>] submit_bio+0x70/0x150
[<ffffffffc05c1071>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc05c1864>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffa33ca15c>] write_cache_pages+0x21c/0x470
[<ffffffffc05c13d0>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffa32e2e89>] ? pick_next_entity+0xa9/0x190
[<ffffffffa32e103c>] ? set_next_entity+0x3c/0xe0
[<ffffffffa322b59e>] ? __switch_to+0xce/0x580
[<ffffffffc05c139b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffa33cb1b1>] do_writepages+0x21/0x50
[<ffffffffa347cbd0>] __writeback_single_inode+0x40/0x260
[<ffffffffa347d764>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa347da6f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa347df53>] wb_writeback+0x263/0x2f0
[<ffffffffa347eb4b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa32be6bf>] process_one_work+0x17f/0x440
[<ffffffffa32bf7d6>] worker_thread+0x126/0x3c0
[<ffffffffa32bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa32c6691>] kthread+0xd1/0xe0
[<ffffffffa32c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa3992d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa32c65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 34 
Complete report #945018
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff91dba385
Build id: 3.10.0-1160.83.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc01846a2
Build id: 3.10.0-1160.83.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff91b0b630
Build id: 3.10.0-1160.83.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff91b02ae0
Build id: 3.10.0-1160.83.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff91b0c68c
Build id: 3.10.0-1160.83.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9198db9e
Build id: 3.10.0-1160.83.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff91968159
Build id: 3.10.0-1160.83.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9196c083
Build id: 3.10.0-1160.83.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff91969eb7
Build id: 3.10.0-1160.83.1.el7.x86_64
0x157
10
submit_bio
vmlinux 0xffffffff9196a160
Build id: 3.10.0-1160.83.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc03cf341
Build id: 3.10.0-1160.83.1.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc03cfb54
Build id: 3.10.0-1160.83.1.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffff917d587c
Build id: 3.10.0-1160.83.1.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc03cf6c0
Build id: 3.10.0-1160.83.1.el7.x86_64
0xb0
15
? __enqueue_entity
vmlinux 0xffffffff916e5bf8
Build id: 3.10.0-1160.83.1.el7.x86_64
0x78
16
? put_prev_entity
vmlinux 0xffffffff916ea09b
Build id: 3.10.0-1160.83.1.el7.x86_64
0x8b
17
xfs_vm_writepages
xfs 0xffffffffc03cf67b
Build id: 3.10.0-1160.83.1.el7.x86_64
0x6b
18
do_writepages
vmlinux 0xffffffff917d6921
Build id: 3.10.0-1160.83.1.el7.x86_64
0x21
19
__writeback_single_inode
vmlinux 0xffffffff9188c790
Build id: 3.10.0-1160.83.1.el7.x86_64
0x40
20
writeback_sb_inodes
vmlinux 0xffffffff9188d35c
Build id: 3.10.0-1160.83.1.el7.x86_64
0x1cc
21
__writeback_inodes_wb
vmlinux 0xffffffff9188d667
Build id: 3.10.0-1160.83.1.el7.x86_64
0xa7
22
wb_writeback
vmlinux 0xffffffff9188db43
Build id: 3.10.0-1160.83.1.el7.x86_64
0x263
23
bdi_writeback_workfn
vmlinux 0xffffffff9188e74f
Build id: 3.10.0-1160.83.1.el7.x86_64
0x2cf
24
process_one_work
vmlinux 0xffffffff916c31df
Build id: 3.10.0-1160.83.1.el7.x86_64
0x17f
25
worker_thread
vmlinux 0xffffffff916c4326
Build id: 3.10.0-1160.83.1.el7.x86_64
0x126
26
? manage_workers.isra.26
vmlinux 0xffffffff916c4200
Build id: 3.10.0-1160.83.1.el7.x86_64
0x2b0
27
kthread
vmlinux 0xffffffff916cb511
Build id: 3.10.0-1160.83.1.el7.x86_64
0xd1
28
? insert_kthread_work
vmlinux 0xffffffff916cb440
Build id: 3.10.0-1160.83.1.el7.x86_64
0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff91dc51dd
Build id: 3.10.0-1160.83.1.el7.x86_64
0x7
30
? insert_kthread_work
vmlinux 0xffffffff916cb440
Build id: 3.10.0-1160.83.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/u4:2:6402]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_16108(E) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon pcspkr joydev sg vmw_vmci parport_pc parport i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel drm ata_piix serio_raw nfit libnvdimm libata vmxnet3 vmw_pvscsi
drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 6402 Comm: kworker/u4:2 Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.83.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff905120dd2100 ti: ffff9051584cc000 task.ti: ffff9051584cc000
RIP: 0010:[<ffffffff91dba385>]  [<ffffffff91dba385>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9051584cf770  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff919c9843 RCX: ffffffff92234f40
RDX: ffffa9eb41b80000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff9051584cf770 R08: 0000000000000001 R09: 0000000000000000
R10: ffff90526d399c00 R11: ffff90526eadf000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff904f9b0dd200 R15: ffffffff91b0a269
FS:  0000000000000000(0000) GS:ffff90527fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0a380fa9d0 CR3: 000000018365a000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc01846a2>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff91b0b630>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff91b02ae0>] scsi_dispatch_cmd+0xb0/0x250
[<ffffffff91b0c68c>] scsi_request_fn+0x4ac/0x680
[<ffffffff9198db9e>] ? deadline_add_request+0x6e/0x90
[<ffffffff91968159>] __blk_run_queue+0x39/0x50
[<ffffffff9196c083>] blk_queue_bio+0x3b3/0x400
[<ffffffff91969eb7>] generic_make_request+0x157/0x390
[<ffffffff9196a160>] submit_bio+0x70/0x150
[<ffffffffc03cf341>] xfs_submit_ioend.isra.12+0x61/0xf0 [xfs]
[<ffffffffc03cfb54>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff917d587c>] write_cache_pages+0x21c/0x480
[<ffffffffc03cf6c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffff916e5bf8>] ? __enqueue_entity+0x78/0x80
[<ffffffff916ea09b>] ? put_prev_entity+0x8b/0x400
[<ffffffffc03cf67b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffff917d6921>] do_writepages+0x21/0x50
[<ffffffff9188c790>] __writeback_single_inode+0x40/0x260
[<ffffffff9188d35c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff9188d667>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff9188db43>] wb_writeback+0x263/0x2f0
[<ffffffff9188e74f>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffff916c31df>] process_one_work+0x17f/0x440
[<ffffffff916c4326>] worker_thread+0x126/0x3c0
[<ffffffff916c4200>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff916cb511>] kthread+0xd1/0xe0
[<ffffffff916cb440>] ? insert_kthread_work+0x40/0x40
[<ffffffff91dc51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff916cb440>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 cc cc cc cc 66 66 66 66 90 55 48 89 e5 48 
Complete report #895966
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb19817c5
Build id: 3.10.0-1062.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc03ab632
Build id: 3.10.0-1062.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb16e4e90
Build id: 3.10.0-1062.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb16dc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb16e5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffb15716de
Build id: 3.10.0-1062.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffb154cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffb1550c63
Build id: 3.10.0-1062.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffb154ebf7
Build id: 3.10.0-1062.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffb154eea0
Build id: 3.10.0-1062.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0a15cd1
Build id: 3.10.0-1062.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc0a1600f
Build id: 3.10.0-1062.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffb13c8d31
Build id: 3.10.0-1062.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffb14776c0
Build id: 3.10.0-1062.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffffb12c6185
Build id: 3.10.0-1062.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffffb1478254
Build id: 3.10.0-1062.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffb147855f
Build id: 3.10.0-1062.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffb1478a43
Build id: 3.10.0-1062.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffb146490c
Build id: 3.10.0-1062.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffb147963b
Build id: 3.10.0-1062.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffb12bd0ff
Build id: 3.10.0-1062.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffb12be216
Build id: 3.10.0-1062.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffb12be0f0
Build id: 3.10.0-1062.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffb12c50d1
Build id: 3.10.0-1062.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffb12c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffb198bd37
Build id: 3.10.0-1062.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffffb12c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 31s! [kworker/u8:3:16389]
Modules linked in: tcp_lp fuse nfsv3 nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi iscsi_target_mod ip_vs target_core_mod nf_conntrack vmw_vsock_vmci_transport vsock sb_edac coretemp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c zfs(POE) zunicode(POE) zlua(POE) sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif
crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect ata_piix sysimgblt crct10dif_pclmul fb_sys_fops crct10dif_common crc32c_intel ttm serio_raw drm vmxnet3 libata vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod zcommon(POE) znvpair(POE) zavl(POE) icp(POE) spl(OE)
CPU: 2 PID: 16389 Comm: kworker/u8:3 Kdump: loaded Tainted: P           OEL ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff97ed1b261070 ti: ffff97eb27034000 task.ti: ffff97eb27034000
connection7:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4897108681, last ping 4897108680, now 4897141782
RIP: 0010:[<ffffffffb19817c5>]  [<ffffffffb19817c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff97eb270378f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffb15ab803 RCX: ffff97ecf02f8060
connection7:0: detected conn error (1022)
RDX: ffffa76580e90000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff97eb270378f0 R08: 0000000000000000 R09: ffff97ed20fe9b00
R10: ffff97ed310e30d8 R11: ffff97ecf0297800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff97eb6edffe00 R15: ffffffffb16e3be9
FS:  0000000000000000(0000) GS:ffff97ed3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8e6b64e000 CR3: 00000001c8ef6000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc03ab632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffb16e4e90>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffb16dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb16e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffb15716de>] ? deadline_add_request+0x6e/0x90
[<ffffffffb154cf59>] __blk_run_queue+0x39/0x50
[<ffffffffb1550c63>] blk_queue_bio+0x3b3/0x400
[<ffffffffb154ebf7>] generic_make_request+0x147/0x380
[<ffffffffb154eea0>] submit_bio+0x70/0x150
[<ffffffffc0a15cd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc0a1600f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffb13c8d31>] do_writepages+0x21/0x50
[<ffffffffb14776c0>] __writeback_single_inode+0x40/0x260
[<ffffffffb12c6185>] ? wake_up_bit+0x25/0x30
[<ffffffffb1478254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb147855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb1478a43>] wb_writeback+0x263/0x2f0
[<ffffffffb146490c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffb147963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb12bd0ff>] process_one_work+0x17f/0x440
[<ffffffffb12be216>] worker_thread+0x126/0x3c0
[<ffffffffb12be0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb12c50d1>] kthread+0xd1/0xe0
[<ffffffffb12c5000>] ? insert_kthread_work+0x40/0x40
[<ffffffffb198bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb12c5000>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #781532
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc021f3be
Build id: 3.10.0-1062.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc028e71e
Build id: 3.10.0-1062.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02872e0
Build id: 3.10.0-1062.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffae0dc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffae0e5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffadf716de
Build id: 3.10.0-1062.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffadf4cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffadf50c63
Build id: 3.10.0-1062.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffadf4ebf7
Build id: 3.10.0-1062.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffadf4eea0
Build id: 3.10.0-1062.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc044bcd1
Build id: 3.10.0-1062.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc044c4c4
Build id: 3.10.0-1062.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffffaddc7cec
Build id: 3.10.0-1062.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc044c030
Build id: 3.10.0-1062.el7.x86_64
0xa0
15
? submit_bio
vmlinux 0xffffffffadf4eea0
Build id: 3.10.0-1062.el7.x86_64
0x70
16
? __switch_to
vmlinux 0xffffffffadc2b59e
Build id: 3.10.0-1062.el7.x86_64
0xce
17
xfs_vm_writepages
xfs 0xffffffffc044bffb
Build id: 3.10.0-1062.el7.x86_64
0x6b
18
do_writepages
vmlinux 0xffffffffaddc8d31
Build id: 3.10.0-1062.el7.x86_64
0x21
19
__writeback_single_inode
vmlinux 0xffffffffade776c0
Build id: 3.10.0-1062.el7.x86_64
0x40
20
writeback_sb_inodes
vmlinux 0xffffffffade78254
Build id: 3.10.0-1062.el7.x86_64
0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffffade7855f
Build id: 3.10.0-1062.el7.x86_64
0x9f
22
wb_writeback
vmlinux 0xffffffffade78a43
Build id: 3.10.0-1062.el7.x86_64
0x263
23
bdi_writeback_workfn
vmlinux 0xffffffffade7963b
Build id: 3.10.0-1062.el7.x86_64
0x2cb
24
process_one_work
vmlinux 0xffffffffadcbd0ff
Build id: 3.10.0-1062.el7.x86_64
0x17f
25
worker_thread
vmlinux 0xffffffffadcbe216
Build id: 3.10.0-1062.el7.x86_64
0x126
26
? manage_workers.isra.26
vmlinux 0xffffffffadcbe0f0
Build id: 3.10.0-1062.el7.x86_64
0x2a0
27
kthread
vmlinux 0xffffffffadcc50d1
Build id: 3.10.0-1062.el7.x86_64
0xd1
28
? insert_kthread_work
vmlinux 0xffffffffadcc5000
Build id: 3.10.0-1062.el7.x86_64
0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffffae38bd1d
Build id: 3.10.0-1062.el7.x86_64
0x7
30
? insert_kthread_work
vmlinux 0xffffffffadcc5000
Build id: 3.10.0-1062.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u256:2:27071]
Modules linked in: xt_nat veth binfmt_misc tcp_lp nls_utf8 isofs rfcomm fuse nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev vmw_balloon crc32_pclmul ghash_clmulni_intel btusb snd_ens1371 btrtl snd_rawmidi btbcm snd_ac97_codec btintel bluetooth ac97_bus aesni_intel snd_seq snd_seq_device lrw gf128mul snd_pcm glue_helper ablk_helper cryptd snd_timer joydev rfkill snd pcspkr sg vmw_vmci soundcore i2c_piix4 parport_pc parport
ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix crct10dif_pclmul crct10dif_common nfit scsi_transport_spi crc32c_intel mptscsih libata mptbase libnvdimm serio_raw e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 27071 Comm: kworker/u256:2 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffffa110f1fb62a0 ti: ffffa11120e60000 task.ti: ffffa11120e60000
RIP: 0010:[<ffffffffc021f3be>]  [<ffffffffc021f3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffffa11120e63708  EFLAGS: 00000246
RAX: ffffbb9c81980000 RBX: ffffa10f94e93288 RCX: 000000000000000a
RDX: ffffa1112b6b0200 RSI: ffffa1112d7e1000 RDI: 000000000000000e
RBP: ffffa11120e63718 R08: 0000000000000003 R09: 000000032912a3ff
R10: ffffa1112d2250d8 R11: ffffa11129908700 R12: ffffa1111d698d80
R13: ffffa11120e6367c R14: ffffffffadf557ab R15: ffffa11120e636c0
FS:  0000000000000000(0000) GS:ffffa11131600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff2ac045480 CR3: 0000000322b70000 CR4: 00000000007607f0
PKRU: 00000000
Call Trace:
[<ffffffffc028e71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02872e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffae0dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffae0e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffadf716de>] ? deadline_add_request+0x6e/0x90
[<ffffffffadf4cf59>] __blk_run_queue+0x39/0x50
[<ffffffffadf50c63>] blk_queue_bio+0x3b3/0x400
[<ffffffffadf4ebf7>] generic_make_request+0x147/0x380
[<ffffffffadf4eea0>] submit_bio+0x70/0x150
[<ffffffffc044bcd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc044c4c4>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffaddc7cec>] write_cache_pages+0x21c/0x470
[<ffffffffc044c030>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffadf4eea0>] ? submit_bio+0x70/0x150
[<ffffffffadc2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc044bffb>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffaddc8d31>] do_writepages+0x21/0x50
[<ffffffffade776c0>] __writeback_single_inode+0x40/0x260
[<ffffffffade78254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffade7855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffade78a43>] wb_writeback+0x263/0x2f0
[<ffffffffade7963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffadcbd0ff>] process_one_work+0x17f/0x440
[<ffffffffadcbe216>] worker_thread+0x126/0x3c0
[<ffffffffadcbe0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffadcc50d1>] kthread+0xd1/0xe0
[<ffffffffadcc5000>] ? insert_kthread_work+0x40/0x40
[<ffffffffae38bd1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffadcc5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 22 
Complete report #658149
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff86fba385
Build id: 3.10.0-1160.88.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc032e6a2
Build id: 3.10.0-1160.88.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff86d0bb30
Build id: 3.10.0-1160.88.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff86d02fe0
Build id: 3.10.0-1160.88.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff86d0cb8c
Build id: 3.10.0-1160.88.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff86b8e08e
Build id: 3.10.0-1160.88.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff86b68649
Build id: 3.10.0-1160.88.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff86b6c573
Build id: 3.10.0-1160.88.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff86b6a3a7
Build id: 3.10.0-1160.88.1.el7.x86_64
0x157
10
submit_bio
vmlinux 0xffffffff86b6a650
Build id: 3.10.0-1160.88.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc05a1341
Build id: 3.10.0-1160.88.1.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc05a1b54
Build id: 3.10.0-1160.88.1.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffff869d599c
Build id: 3.10.0-1160.88.1.el7.x86_64
0x21c
14
? check_preempt_curr
vmlinux 0xffffffff868dd212
Build id: 3.10.0-1160.88.1.el7.x86_64
0x92
15
? xfs_vm_writepages
xfs 0xffffffffc05a16c0
Build id: 3.10.0-1160.88.1.el7.x86_64
0xb0
16
? cpumask_next_and
vmlinux 0xffffffff86b9c485
Build id: 3.10.0-1160.88.1.el7.x86_64
0x35
17
xfs_vm_writepages
xfs 0xffffffffc05a167b
Build id: 3.10.0-1160.88.1.el7.x86_64
0x6b
18
do_writepages
vmlinux 0xffffffff869d6a41
Build id: 3.10.0-1160.88.1.el7.x86_64
0x21
19
__writeback_single_inode
vmlinux 0xffffffff86a8cc80
Build id: 3.10.0-1160.88.1.el7.x86_64
0x40
20
writeback_sb_inodes
vmlinux 0xffffffff86a8d84c
Build id: 3.10.0-1160.88.1.el7.x86_64
0x1cc
21
__writeback_inodes_wb
vmlinux 0xffffffff86a8db57
Build id: 3.10.0-1160.88.1.el7.x86_64
0xa7
22
wb_writeback
vmlinux 0xffffffff86a8e033
Build id: 3.10.0-1160.88.1.el7.x86_64
0x263
23
? get_nr_inodes
vmlinux 0xffffffff86a78ffc
Build id: 3.10.0-1160.88.1.el7.x86_64
0x4c
24
bdi_writeback_workfn
vmlinux 0xffffffff86a8ec3f
Build id: 3.10.0-1160.88.1.el7.x86_64
0x2cf
25
process_one_work
vmlinux 0xffffffff868c32ef
Build id: 3.10.0-1160.88.1.el7.x86_64
0x17f
26
worker_thread
vmlinux 0xffffffff868c4436
Build id: 3.10.0-1160.88.1.el7.x86_64
0x126
27
? manage_workers.isra.26
vmlinux 0xffffffff868c4310
Build id: 3.10.0-1160.88.1.el7.x86_64
0x2b0
28
kthread
vmlinux 0xffffffff868cb621
Build id: 3.10.0-1160.88.1.el7.x86_64
0xd1
29
? insert_kthread_work
vmlinux 0xffffffff868cb550
Build id: 3.10.0-1160.88.1.el7.x86_64
0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffff86fc51dd
Build id: 3.10.0-1160.88.1.el7.x86_64
0x7
31
? insert_kthread_work
vmlinux 0xffffffff868cb550
Build id: 3.10.0-1160.88.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:30:411]
Modules linked in: vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg vmw_balloon pcspkr joydev i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea crc_t10dif sysfillrect crct10dif_generic sysimgblt fb_sys_fops ttm ahci nfit libnvdimm drm ata_piix libahci crct10dif_pclmul crct10dif_common crc32c_intel libata vmxnet3 serio_raw vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 411 Comm: kworker/u8:30 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9423e97f9080 ti: ffff9423e8c00000 task.ti: ffff9423e8c00000
RIP: 0010:[<ffffffff86fba385>]  [<ffffffff86fba385>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9423e8c03770  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff86bc9d43 RCX: ffff942135e51060
RDX: ffffb79d41c90000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff9423e8c03770 R08: 0000000000000000 R09: ffff9423ed039d00
R10: ffff9423ea7c1800 R11: ffff941ff5c82000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9423a3e1ef00 R15: ffffffff86d0a769
FS:  0000000000000000(0000) GS:ffff9423ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f50a3af3000 CR3: 000000042ab24000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc032e6a2>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff86d0bb30>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff86d02fe0>] scsi_dispatch_cmd+0xb0/0x250
[<ffffffff86d0cb8c>] scsi_request_fn+0x4ac/0x680
[<ffffffff86b8e08e>] ? deadline_add_request+0x6e/0x90
[<ffffffff86b68649>] __blk_run_queue+0x39/0x50
[<ffffffff86b6c573>] blk_queue_bio+0x3b3/0x400
[<ffffffff86b6a3a7>] generic_make_request+0x157/0x390
[<ffffffff86b6a650>] submit_bio+0x70/0x150
[<ffffffffc05a1341>] xfs_submit_ioend.isra.12+0x61/0xf0 [xfs]
[<ffffffffc05a1b54>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff869d599c>] write_cache_pages+0x21c/0x480
[<ffffffff868dd212>] ? check_preempt_curr+0x92/0xa0
[<ffffffffc05a16c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffff86b9c485>] ? cpumask_next_and+0x35/0x50
[<ffffffffc05a167b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffff869d6a41>] do_writepages+0x21/0x50
[<ffffffff86a8cc80>] __writeback_single_inode+0x40/0x260
[<ffffffff86a8d84c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff86a8db57>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff86a8e033>] wb_writeback+0x263/0x2f0
[<ffffffff86a78ffc>] ? get_nr_inodes+0x4c/0x70
[<ffffffff86a8ec3f>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffff868c32ef>] process_one_work+0x17f/0x440
[<ffffffff868c4436>] worker_thread+0x126/0x3c0
[<ffffffff868c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff868cb621>] kthread+0xd1/0xe0
[<ffffffff868cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff86fc51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff868cb550>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 cc cc cc cc 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #513721
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00ed3be
Build id: 3.10.0-1062.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc011871e
Build id: 3.10.0-1062.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00e62e0
Build id: 3.10.0-1062.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff848dc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff848e5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff847716de
Build id: 3.10.0-1062.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8474cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff84750c63
Build id: 3.10.0-1062.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff8474ebf7
Build id: 3.10.0-1062.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff8474eea0
Build id: 3.10.0-1062.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0367cd1
Build id: 3.10.0-1062.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc03684c4
Build id: 3.10.0-1062.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffff845c7cec
Build id: 3.10.0-1062.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc0368030
Build id: 3.10.0-1062.el7.x86_64
0xa0
15
? submit_bio
vmlinux 0xffffffff8474eea0
Build id: 3.10.0-1062.el7.x86_64
0x70
16
xfs_vm_writepages
xfs 0xffffffffc0367ffb
Build id: 3.10.0-1062.el7.x86_64
0x6b
17
do_writepages
vmlinux 0xffffffff845c8d31
Build id: 3.10.0-1062.el7.x86_64
0x21
18
__writeback_single_inode
vmlinux 0xffffffff846776c0
Build id: 3.10.0-1062.el7.x86_64
0x40
19
? wake_up_bit
vmlinux 0xffffffff844c6185
Build id: 3.10.0-1062.el7.x86_64
0x25
20
writeback_sb_inodes
vmlinux 0xffffffff84678254
Build id: 3.10.0-1062.el7.x86_64
0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffff8467855f
Build id: 3.10.0-1062.el7.x86_64
0x9f
22
wb_writeback
vmlinux 0xffffffff84678a43
Build id: 3.10.0-1062.el7.x86_64
0x263
23
bdi_writeback_workfn
vmlinux 0xffffffff8467963b
Build id: 3.10.0-1062.el7.x86_64
0x2cb
24
process_one_work
vmlinux 0xffffffff844bd0ff
Build id: 3.10.0-1062.el7.x86_64
0x17f
25
worker_thread
vmlinux 0xffffffff844be216
Build id: 3.10.0-1062.el7.x86_64
0x126
26
? manage_workers.isra.26
vmlinux 0xffffffff844be0f0
Build id: 3.10.0-1062.el7.x86_64
0x2a0
27
kthread
vmlinux 0xffffffff844c50d1
Build id: 3.10.0-1062.el7.x86_64
0xd1
28
? insert_kthread_work
vmlinux 0xffffffff844c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff84b8bd37
Build id: 3.10.0-1062.el7.x86_64
0x21
30
? insert_kthread_work
vmlinux 0xffffffff844c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [kworker/u256:1:119]
Modules linked in: vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon ppdev cryptd pcspkr sg joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih nfit libnvdimm libata e1000 crct10dif_pclmul crct10dif_common mptbase crc32c_intel drm_panel_orientation_quirks serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 119 Comm: kworker/u256:1 Tainted: G        W    L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff981a99605230 ti: ffff981a996d8000 task.ti: ffff981a996d8000
RIP: 0010:[<ffffffffc00ed3be>]  [<ffffffffc00ed3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff981a996db708  EFLAGS: 00000246
RAX: ffffa7e2031c0000 RBX: ffff981a867f4000 RCX: 0000000000000010
RDX: ffff981a97ca4e00 RSI: ffff981a9de79000 RDI: 000000000000000e
RBP: ffff981a996db718 R08: 0000000000000002 R09: 00000008158c4fff
R10: ffff981a996110d8 R11: ffff981a963c01c0 R12: ffff981a8e292a00
R13: ffff981a996db67c R14: ffffffff847557ab R15: ffff981a996db6c0
FS:  0000000000000000(0000) GS:ffff981a9d740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe8a1f336d0 CR3: 00000005d5c10000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc011871e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00e62e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff848dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff848e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff847716de>] ? deadline_add_request+0x6e/0x90
[<ffffffff8474cf59>] __blk_run_queue+0x39/0x50
[<ffffffff84750c63>] blk_queue_bio+0x3b3/0x400
[<ffffffff8474ebf7>] generic_make_request+0x147/0x380
[<ffffffff8474eea0>] submit_bio+0x70/0x150
[<ffffffffc0367cd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc03684c4>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffff845c7cec>] write_cache_pages+0x21c/0x470
[<ffffffffc0368030>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff8474eea0>] ? submit_bio+0x70/0x150
[<ffffffffc0367ffb>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff845c8d31>] do_writepages+0x21/0x50
[<ffffffff846776c0>] __writeback_single_inode+0x40/0x260
[<ffffffff844c6185>] ? wake_up_bit+0x25/0x30
[<ffffffff84678254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8467855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff84678a43>] wb_writeback+0x263/0x2f0
[<ffffffff8467963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff844bd0ff>] process_one_work+0x17f/0x440
[<ffffffff844be216>] worker_thread+0x126/0x3c0
[<ffffffff844be0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff844c50d1>] kthread+0xd1/0xe0
[<ffffffff844c5000>] ? insert_kthread_work+0x40/0x40
[<ffffffff84b8bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff844c5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 b3 0f 
Complete report #453083
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02af3be
Build id: 3.10.0-1160.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02d171e
Build id: 3.10.0-1160.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02ca2e0
Build id: 3.10.0-1160.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb8ce3980
Build id: 3.10.0-1160.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb8ced0ac
Build id: 3.10.0-1160.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb8b77f8e
Build id: 3.10.0-1160.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffb8b534d9
Build id: 3.10.0-1160.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffb8b571e3
Build id: 3.10.0-1160.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffb8b55167
Build id: 3.10.0-1160.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffb8b55410
Build id: 3.10.0-1160.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc054df31
Build id: 3.10.0-1160.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc054e724
Build id: 3.10.0-1160.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffffb89c9d8c
Build id: 3.10.0-1160.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc054e290
Build id: 3.10.0-1160.el7.x86_64
0xa0
15
? __enqueue_entity
vmlinux 0xffffffffb88dfb48
Build id: 3.10.0-1160.el7.x86_64
0x78
16
? put_prev_entity
vmlinux 0xffffffffb88e5f1b
Build id: 3.10.0-1160.el7.x86_64
0x8b
17
? __switch_to
vmlinux 0xffffffffb882b59e
Build id: 3.10.0-1160.el7.x86_64
0xce
18
xfs_vm_writepages
xfs 0xffffffffc054e25b
Build id: 3.10.0-1160.el7.x86_64
0x6b
19
do_writepages
vmlinux 0xffffffffb89cade1
Build id: 3.10.0-1160.el7.x86_64
0x21
20
__writeback_single_inode
vmlinux 0xffffffffb8a7d6f0
Build id: 3.10.0-1160.el7.x86_64
0x40
21
writeback_sb_inodes
vmlinux 0xffffffffb8a7e274
Build id: 3.10.0-1160.el7.x86_64
0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffffb8a7e57f
Build id: 3.10.0-1160.el7.x86_64
0x9f
23
wb_writeback
vmlinux 0xffffffffb8a7ea53
Build id: 3.10.0-1160.el7.x86_64
0x263
24
bdi_writeback_workfn
vmlinux 0xffffffffb8a7f64b
Build id: 3.10.0-1160.el7.x86_64
0x2cb
25
process_one_work
vmlinux 0xffffffffb88bdc4f
Build id: 3.10.0-1160.el7.x86_64
0x17f
26
worker_thread
vmlinux 0xffffffffb88bed66
Build id: 3.10.0-1160.el7.x86_64
0x126
27
? manage_workers.isra.26
vmlinux 0xffffffffb88bec40
Build id: 3.10.0-1160.el7.x86_64
0x2a0
28
kthread
vmlinux 0xffffffffb88c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
29
? insert_kthread_work
vmlinux 0xffffffffb88c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffffb8f93df7
Build id: 3.10.0-1160.el7.x86_64
0x21
31
? insert_kthread_work
vmlinux 0xffffffffb88c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 27s! [kworker/u256:0:5275]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi coretemp snd_ac97_codec ac97_bus snd_seq btusb btrtl snd_seq_device iosf_mbi btbcm btintel crc32_pclmul bluetooth ghash_clmulni_intel snd_pcm aesni_intel lrw ppdev gf128mul snd_timer snd glue_helper soundcore ablk_helper rfkill cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix serio_raw drm libata nfit e1000 mptspi scsi_transport_spi libnvdimm mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 5275 Comm: kworker/u256:0 Kdump: loaded Not tainted 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9f78ddb28000 ti: ffff9f78b4e08000 task.ti: ffff9f78b4e08000
RIP: 0010:[<ffffffffc02af3be>]  [<ffffffffc02af3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9f78b4e0b708  EFLAGS: 00000246
RAX: ffffb47340440000 RBX: ffff9f78f34bc000 RCX: 0000000000000005
RDX: ffff9f78b54fc000 RSI: ffff9f78b5af3000 RDI: 000000000000000e
RBP: ffff9f78b4e0b718 R08: 0000000000000002 R09: 000000000da43fff
R10: ffff9f78b4d6d000 R11: ffff9f78cf952d80 R12: ffff9f788e97a700
R13: ffff9f78b4e0b67c R14: ffffffffb8b5bdfb R15: ffff9f78b4e0b6c0
FS:  0000000000000000(0000) GS:ffff9f78fb640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f23d7f40000 CR3: 000000004df2e000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc02d171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02ca2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb8ce3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb8ced0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffffb8b77f8e>] ? deadline_add_request+0x6e/0x90
[<ffffffffb8b534d9>] __blk_run_queue+0x39/0x50
[<ffffffffb8b571e3>] blk_queue_bio+0x3b3/0x400
[<ffffffffb8b55167>] generic_make_request+0x147/0x380
[<ffffffffb8b55410>] submit_bio+0x70/0x150
[<ffffffffc054df31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc054e724>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffb89c9d8c>] write_cache_pages+0x21c/0x470
[<ffffffffc054e290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffb88dfb48>] ? __enqueue_entity+0x78/0x80
[<ffffffffb88e5f1b>] ? put_prev_entity+0x8b/0x400
[<ffffffffb882b59e>] ? __switch_to+0xce/0x580
[<ffffffffc054e25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb89cade1>] do_writepages+0x21/0x50
[<ffffffffb8a7d6f0>] __writeback_single_inode+0x40/0x260
[<ffffffffb8a7e274>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb8a7e57f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb8a7ea53>] wb_writeback+0x263/0x2f0
[<ffffffffb8a7f64b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb88bdc4f>] process_one_work+0x17f/0x440
[<ffffffffb88bed66>] worker_thread+0x126/0x3c0
[<ffffffffb88bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb88c5c21>] kthread+0xd1/0xe0
[<ffffffffb88c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb8f93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb88c5b50>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 2b 
Complete report #440767
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01bf3be
Build id: 3.10.0-1160.66.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc026e71e
Build id: 3.10.0-1160.66.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02792e0
Build id: 3.10.0-1160.66.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffaeae4650
Build id: 3.10.0-1160.66.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffaeaedd7c
Build id: 3.10.0-1160.66.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffae9789de
Build id: 3.10.0-1160.66.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffae953d09
Build id: 3.10.0-1160.66.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffae957ad3
Build id: 3.10.0-1160.66.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffae9559b7
Build id: 3.10.0-1160.66.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffae955c60
Build id: 3.10.0-1160.66.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0470f31
Build id: 3.10.0-1160.66.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc047126f
Build id: 3.10.0-1160.66.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffae7cb9d1
Build id: 3.10.0-1160.66.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffae87df40
Build id: 3.10.0-1160.66.1.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffffae6c7045
Build id: 3.10.0-1160.66.1.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffffae87eac4
Build id: 3.10.0-1160.66.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffae87edcf
Build id: 3.10.0-1160.66.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffae87f2a3
Build id: 3.10.0-1160.66.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffae86ac1c
Build id: 3.10.0-1160.66.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffae87fe9b
Build id: 3.10.0-1160.66.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffae6bdfbf
Build id: 3.10.0-1160.66.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffae6bf0d6
Build id: 3.10.0-1160.66.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffae6befb0
Build id: 3.10.0-1160.66.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffae6c5f91
Build id: 3.10.0-1160.66.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffae6c5ec0
Build id: 3.10.0-1160.66.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffaed99ddd
Build id: 3.10.0-1160.66.1.el7.x86_64
0x7
27
? insert_kthread_work
vmlinux 0xffffffffae6c5ec0
Build id: 3.10.0-1160.66.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [kworker/u128:3:61123]
Modules linked in: tcp_diag udp_diag inet_diag nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc zfs(POE) zunicode(POE) zzstd(OE) zlua(OE) zcommon(POE) znvpair(POE) zavl(POE) icp(POE) iosf_mbi crc32_pclmul spl(OE) ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect ata_piix sysimgblt fb_sys_fops ttm nfit libnvdimm drm mptspi scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common crc32c_intel libata mptbase e1000 serio_raw drm_panel_orientation_quirks
floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: fileaccess_mod_100600201]
CPU: 3 PID: 61123 Comm: kworker/u128:3 Kdump: loaded Tainted: P           OEL ------------   3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9a1f2d0be300 ti: ffff9a22a772c000 task.ti: ffff9a22a772c000
RIP: 0010:[<ffffffffc01bf3be>]  [<ffffffffc01bf3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9a22a772f888  EFLAGS: 00000246
RAX: ffffb2d741900000 RBX: ffff9a20dabc4fd0 RCX: 0000000000000000
RDX: ffff9a22a97a2400 RSI: ffff9a22a9433000 RDI: 000000000000000e
RBP: ffff9a22a772f898 R08: 0000000000000003 R09: 00000004228cbfff
R10: ffff9a22a920b800 R11: ffff9a213e2addc0 R12: ffff9a20fa691c80
R13: ffff9a22a772f7fc R14: ffffffffae95c6eb R15: ffff9a22a772f840
FS:  0000000000000000(0000) GS:ffff9a22ae6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f72a88a8000 CR3: 00000000ba6ec000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc026e71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02792e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffaeae4650>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffaeaedd7c>] scsi_request_fn+0x4ac/0x680
[<ffffffffae9789de>] ? deadline_add_request+0x6e/0x90
[<ffffffffae953d09>] __blk_run_queue+0x39/0x50
[<ffffffffae957ad3>] blk_queue_bio+0x3b3/0x400
[<ffffffffae9559b7>] generic_make_request+0x147/0x380
[<ffffffffae955c60>] submit_bio+0x70/0x150
[<ffffffffc0470f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc047126f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffae7cb9d1>] do_writepages+0x21/0x50
[<ffffffffae87df40>] __writeback_single_inode+0x40/0x260
[<ffffffffae6c7045>] ? wake_up_bit+0x25/0x30
[<ffffffffae87eac4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffae87edcf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffae87f2a3>] wb_writeback+0x263/0x2f0
[<ffffffffae86ac1c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffae87fe9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffae6bdfbf>] process_one_work+0x17f/0x440
[<ffffffffae6bf0d6>] worker_thread+0x126/0x3c0
[<ffffffffae6befb0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffae6c5f91>] kthread+0xd1/0xe0
[<ffffffffae6c5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffffaed99ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffae6c5ec0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 1c 
Complete report #406150
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb398e9f5
Build id: 3.10.0-1160.62.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0317632
Build id: 3.10.0-1160.62.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb36ecd50
Build id: 3.10.0-1160.62.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb36e4640
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb36edd6c
Build id: 3.10.0-1160.62.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb35789ce
Build id: 3.10.0-1160.62.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffb3553cf9
Build id: 3.10.0-1160.62.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffb3557ac3
Build id: 3.10.0-1160.62.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffb35559a7
Build id: 3.10.0-1160.62.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffb3555c50
Build id: 3.10.0-1160.62.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc057ef31
Build id: 3.10.0-1160.62.1.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc057f724
Build id: 3.10.0-1160.62.1.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffffb33ca96c
Build id: 3.10.0-1160.62.1.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc057f290
Build id: 3.10.0-1160.62.1.el7.x86_64
0xa0
15
xfs_vm_writepages
xfs 0xffffffffc057f25b
Build id: 3.10.0-1160.62.1.el7.x86_64
0x6b
16
do_writepages
vmlinux 0xffffffffb33cb9c1
Build id: 3.10.0-1160.62.1.el7.x86_64
0x21
17
__writeback_single_inode
vmlinux 0xffffffffb347df30
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
18
? wake_up_bit
vmlinux 0xffffffffb32c7065
Build id: 3.10.0-1160.62.1.el7.x86_64
0x25
19
writeback_sb_inodes
vmlinux 0xffffffffb347eab4
Build id: 3.10.0-1160.62.1.el7.x86_64
0x1c4
20
__writeback_inodes_wb
vmlinux 0xffffffffb347edbf
Build id: 3.10.0-1160.62.1.el7.x86_64
0x9f
21
wb_writeback
vmlinux 0xffffffffb347f293
Build id: 3.10.0-1160.62.1.el7.x86_64
0x263
22
bdi_writeback_workfn
vmlinux 0xffffffffb347fe8b
Build id: 3.10.0-1160.62.1.el7.x86_64
0x2cb
23
process_one_work
vmlinux 0xffffffffb32bdfdf
Build id: 3.10.0-1160.62.1.el7.x86_64
0x17f
24
worker_thread
vmlinux 0xffffffffb32bf0f6
Build id: 3.10.0-1160.62.1.el7.x86_64
0x126
25
? manage_workers.isra.26
vmlinux 0xffffffffb32befd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x2a0
26
kthread
vmlinux 0xffffffffb32c5fb1
Build id: 3.10.0-1160.62.1.el7.x86_64
0xd1
27
? insert_kthread_work
vmlinux 0xffffffffb32c5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
28
ret_from_fork_nospec_begin
vmlinux 0xffffffffb3999df7
Build id: 3.10.0-1160.62.1.el7.x86_64
0x21
29
? insert_kthread_work
vmlinux 0xffffffffb32c5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/u4:3:6824]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13109(E) tcp_lp falcon_lsm_pinned_12905(E) nfsv3 nfs_acl nfs lockd grace fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg parport_pc vmw_vmci parport i2c_piix4 binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic
drm_kms_helper ahci crct10dif_pclmul crct10dif_common syscopyarea sysfillrect ata_piix libahci sysimgblt fb_sys_fops crc32c_intel ttm serio_raw libata drm vmxnet3 nfit libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 1 PID: 6824 Comm: kworker/u4:3 Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff918bb8db3180 ti: ffff918b7b7c8000 task.ti: ffff918b7b7c8000
RIP: 0010:[<ffffffffb398e9f5>]  [<ffffffffb398e9f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff918b7b7cb770  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffffb35b2d63 RCX: ffffffffb3e34f40
RDX: ffffb6c540780000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff918b7b7cb770 R08: 0000000000000001 R09: 0000000000000000
R10: ffff918b34815c00 R11: ffff918c33752800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff918ba9f9f980 R15: ffffffffb36eba09
FS:  0000000000000000(0000) GS:ffff918c3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006dee1c CR3: 00000000b620a000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc0317632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffb36ecd50>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffb36e4640>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb36edd6c>] scsi_request_fn+0x4ac/0x680
[<ffffffffb35789ce>] ? deadline_add_request+0x6e/0x90
[<ffffffffb3553cf9>] __blk_run_queue+0x39/0x50
[<ffffffffb3557ac3>] blk_queue_bio+0x3b3/0x400
[<ffffffffb35559a7>] generic_make_request+0x147/0x380
[<ffffffffb3555c50>] submit_bio+0x70/0x150
[<ffffffffc057ef31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc057f724>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffb33ca96c>] write_cache_pages+0x21c/0x470
[<ffffffffc057f290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffc057f25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb33cb9c1>] do_writepages+0x21/0x50
[<ffffffffb347df30>] __writeback_single_inode+0x40/0x260
[<ffffffffb32c7065>] ? wake_up_bit+0x25/0x30
[<ffffffffb347eab4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb347edbf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb347f293>] wb_writeback+0x263/0x2f0
[<ffffffffb347fe8b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb32bdfdf>] process_one_work+0x17f/0x440
[<ffffffffb32bf0f6>] worker_thread+0x126/0x3c0
[<ffffffffb32befd0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb32c5fb1>] kthread+0xd1/0xe0
[<ffffffffb32c5ee0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb3999df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb32c5ee0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #393141
Frame # Function Binary Source or offset Line
1
scsi_dispatch_cmd
vmlinux 0xffffffff954e4857
Build id: 3.10.0-1160.45.1.el7.x86_64
0xa7
2
scsi_request_fn
vmlinux 0xffffffff954edf8c
Build id: 3.10.0-1160.45.1.el7.x86_64
0x4ac
3
? deadline_add_request
vmlinux 0xffffffff95378bee
Build id: 3.10.0-1160.45.1.el7.x86_64
0x6e
4
__blk_run_queue
vmlinux 0xffffffff95353f39
Build id: 3.10.0-1160.45.1.el7.x86_64
0x39
5
blk_queue_bio
vmlinux 0xffffffff95357ce3
Build id: 3.10.0-1160.45.1.el7.x86_64
0x3b3
6
generic_make_request
vmlinux 0xffffffff95355bc7
Build id: 3.10.0-1160.45.1.el7.x86_64
0x147
7
submit_bio
vmlinux 0xffffffff95355e70
Build id: 3.10.0-1160.45.1.el7.x86_64
0x70
8
xfs_submit_ioend.isra.12
xfs 0xffffffffc06abf31
Build id: 3.10.0-1160.45.1.el7.x86_64
0x61
9
xfs_vm_writepages
xfs 0xffffffffc06ac26f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x7f
10
do_writepages
vmlinux 0xffffffff951cb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64
0x21
11
__writeback_single_inode
vmlinux 0xffffffff9527df10
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
12
? wake_up_bit
vmlinux 0xffffffff950c6f15
Build id: 3.10.0-1160.45.1.el7.x86_64
0x25
13
writeback_sb_inodes
vmlinux 0xffffffff9527ea94
Build id: 3.10.0-1160.45.1.el7.x86_64
0x1c4
14
__writeback_inodes_wb
vmlinux 0xffffffff9527ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x9f
15
wb_writeback
vmlinux 0xffffffff9527f273
Build id: 3.10.0-1160.45.1.el7.x86_64
0x263
16
? get_nr_inodes
vmlinux 0xffffffff9526ae6c
Build id: 3.10.0-1160.45.1.el7.x86_64
0x4c
17
bdi_writeback_workfn
vmlinux 0xffffffff9527fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64
0x2cb
18
process_one_work
vmlinux 0xffffffff950bde8f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x17f
19
worker_thread
vmlinux 0xffffffff950befa6
Build id: 3.10.0-1160.45.1.el7.x86_64
0x126
20
? manage_workers.isra.26
vmlinux 0xffffffff950bee80
Build id: 3.10.0-1160.45.1.el7.x86_64
0x2a0
21
kthread
vmlinux 0xffffffff950c5e61
Build id: 3.10.0-1160.45.1.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffff950c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffff95795df7
Build id: 3.10.0-1160.45.1.el7.x86_64
0x21
24
? insert_kthread_work
vmlinux 0xffffffff950c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u2:0:27818]
Modules linked in: binfmt_misc devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock squashfs loop coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr joydev vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c
 sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif drm_kms_helper crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel serio_raw mptspi libata scsi_transport_spi mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 27818 Comm: kworker/u2:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff914f38223180 ti: ffff914f20ae8000 task.ti: ffff914f20ae8000
RIP: 0010:[<ffffffff954e4857>]  [<ffffffff954e4857>] scsi_dispatch_cmd+0xa7/0x240
RSP: 0018:ffff914f20aeb978  EFLAGS: 00010246
RAX: ffffffffc051e280 RBX: ffffffff950ada83 RCX: ffff91502d1dc298
RDX: 000000000000000a RSI: ffff915078b2c380 RDI: ffff915074791000
RBP: ffff914f20aeb990 R08: 0000000000000010 R09: 0000000000000000
R10: ffff914ef5f97800 R11: ffff915078b2c380 R12: ffff914ef5e7d348
R13: ffffffff96103bc0 R14: ffffffff96103bc0 R15: ffff914ef5e7d348
FS:  0000000000000000(0000) GS:ffff91507fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb694b0ea10 CR3: 00000000b43e8000 CR4: 00000000000407f0
Call Trace:
 [<ffffffff954edf8c>] scsi_request_fn+0x4ac/0x680
 [<ffffffff95378bee>] ? deadline_add_request+0x6e/0x90
 [<ffffffff95353f39>] __blk_run_queue+0x39/0x50
 [<ffffffff95357ce3>] blk_queue_bio+0x3b3/0x400
 [<ffffffff95355bc7>] generic_make_request+0x147/0x380
 [<ffffffff95355e70>] submit_bio+0x70/0x150
 [<ffffffffc06abf31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
 [<ffffffffc06ac26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
 [<ffffffff951cb9b1>] do_writepages+0x21/0x50
 [<ffffffff9527df10>] __writeback_single_inode+0x40/0x260
 [<ffffffff950c6f15>] ? wake_up_bit+0x25/0x30
 [<ffffffff9527ea94>] writeback_sb_inodes+0x1c4/0x430
 [<ffffffff9527ed9f>] __writeback_inodes_wb+0x9f/0xd0
 [<ffffffff9527f273>] wb_writeback+0x263/0x2f0
 [<ffffffff9526ae6c>] ? get_nr_inodes+0x4c/0x70
 [<ffffffff9527fe6b>] bdi_writeback_workfn+0x2cb/0x460
 [<ffffffff950bde8f>] process_one_work+0x17f/0x440
 [<ffffffff950befa6>] worker_thread+0x126/0x3c0
 [<ffffffff950bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
 [<ffffffff950c5e61>] kthread+0xd1/0xe0
 [<ffffffff950c5d90>] ? insert_kthread_work+0x40/0x40
 [<ffffffff95795df7>] ret_from_fork_nospec_begin+0x21/0x21
 [<ffffffff950c5d90>] ? insert_kthread_work+0x40/0x40
Code: 00 00 66 44 39 c2 77 44 41 83 bd 88 01 00 00 04 0f 84 1d 01 00 00 66 66 66 66 90 49 8b 85 c0 00 00 00 4c 89 ef 48 89 de 45 31 e4 <48> 8b 40 38 e8 f0 36 eb ff 85 c0 41 89 c5 0f 85 85 00 00 00 5b 
Complete report #383563
Frame # Function Binary Source or offset Line
1
pvscsi_queue
vmw_pvscsi 0xffffffffc020f632
Build id: 3.10.0-1160.el7.x86_64
0x4a2
2
? scsi_test_unit_ready
vmlinux 0xffffffff868ec090
Build id: 3.10.0-1160.el7.x86_64
0x170
3
scsi_dispatch_cmd
vmlinux 0xffffffff868e3980
Build id: 3.10.0-1160.el7.x86_64
0xb0
4
scsi_request_fn
vmlinux 0xffffffff868ed0ac
Build id: 3.10.0-1160.el7.x86_64
0x4ac
5
? deadline_add_request
vmlinux 0xffffffff86777f8e
Build id: 3.10.0-1160.el7.x86_64
0x6e
6
__blk_run_queue
vmlinux 0xffffffff867534d9
Build id: 3.10.0-1160.el7.x86_64
0x39
7
blk_queue_bio
vmlinux 0xffffffff867571e3
Build id: 3.10.0-1160.el7.x86_64
0x3b3
8
generic_make_request
vmlinux 0xffffffff86755167
Build id: 3.10.0-1160.el7.x86_64
0x147
9
submit_bio
vmlinux 0xffffffff86755410
Build id: 3.10.0-1160.el7.x86_64
0x70
10
xfs_submit_ioend.isra.12
xfs 0xffffffffc045af31
Build id: 3.10.0-1160.el7.x86_64
0x61
11
xfs_vm_writepages
xfs 0xffffffffc045b26f
Build id: 3.10.0-1160.el7.x86_64
0x7f
12
do_writepages
vmlinux 0xffffffff865cade1
Build id: 3.10.0-1160.el7.x86_64
0x21
13
__writeback_single_inode
vmlinux 0xffffffff8667d6f0
Build id: 3.10.0-1160.el7.x86_64
0x40
14
? wake_up_bit
vmlinux 0xffffffff864c6cd5
Build id: 3.10.0-1160.el7.x86_64
0x25
15
writeback_sb_inodes
vmlinux 0xffffffff8667e274
Build id: 3.10.0-1160.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8667e57f
Build id: 3.10.0-1160.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff8667ea53
Build id: 3.10.0-1160.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8667f54c
Build id: 3.10.0-1160.el7.x86_64
0x1cc
19
process_one_work
vmlinux 0xffffffff864bdc4f
Build id: 3.10.0-1160.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff864bed66
Build id: 3.10.0-1160.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff864bec40
Build id: 3.10.0-1160.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff864c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff864c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff86b93df7
Build id: 3.10.0-1160.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff864c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 27s! [kswapd0:45]
FS:  0000000000000000(0000) GS:ffff945dffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f35d0a1f000 CR3: 0000000236c54000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc020f632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff868ec090>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff868e3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff868ed0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffff86777f8e>] ? deadline_add_request+0x6e/0x90
[<ffffffff867534d9>] __blk_run_queue+0x39/0x50
[<ffffffff867571e3>] blk_queue_bio+0x3b3/0x400
[<ffffffff86755167>] generic_make_request+0x147/0x380
[<ffffffff86755410>] submit_bio+0x70/0x150
[<ffffffffc045af31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc045b26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff865cade1>] do_writepages+0x21/0x50
[<ffffffff8667d6f0>] __writeback_single_inode+0x40/0x260
[<ffffffff864c6cd5>] ? wake_up_bit+0x25/0x30
[<ffffffff8667e274>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8667e57f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8667ea53>] wb_writeback+0x263/0x2f0
[<ffffffff8667f54c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff864bdc4f>] process_one_work+0x17f/0x440
[<ffffffff864bed66>] worker_thread+0x126/0x3c0
[<ffffffff864bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff864c5c21>] kthread+0xd1/0xe0
[<ffffffff864c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff86b93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff864c5b50>] ? insert_kthread_work+0x40/0x40
Complete report #264159
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff88d8b565
Build id: 3.10.0-1160.25.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0279632
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff88aec6b0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff88ae3fa0
Build id: 3.10.0-1160.25.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff88aed6cc
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8897853e
Build id: 3.10.0-1160.25.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff88953939
Build id: 3.10.0-1160.25.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff88957743
Build id: 3.10.0-1160.25.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff889555c7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff88955870
Build id: 3.10.0-1160.25.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc053df31
Build id: 3.10.0-1160.25.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc053e26f
Build id: 3.10.0-1160.25.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff887cb001
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8887d9a0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffff886c6e55
Build id: 3.10.0-1160.25.1.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffff8887e524
Build id: 3.10.0-1160.25.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff8887e82f
Build id: 3.10.0-1160.25.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff8887ed03
Build id: 3.10.0-1160.25.1.el7.x86_64
0x263
19
bdi_writeback_workfn
vmlinux 0xffffffff8887f8fb
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffff886bddcf
Build id: 3.10.0-1160.25.1.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff886beee6
Build id: 3.10.0-1160.25.1.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff886bedc0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff886c5da1
Build id: 3.10.0-1160.25.1.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff886c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff88d95df7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffff886c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [dbus-daemon:1294]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:3:13804]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) vmw_vsock_vmci_transport falcon_lsm_pinned_11906(E) vsock sunrpc crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd pcspkr joydev sg parport_pc i2c_piix4 parport vmw_vmci binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix ahci nfit libahci libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 13804 Comm: kworker/u8:3 Tainted: P            EL ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:4)
task: ffff9f33da152100 ti: ffff9f3386848000 task.ti: ffff9f3386848000
RIP: 0010:[<ffffffff88d8b565>]  [<ffffffff88d8b565>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9f338684b8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff889b28d3 RCX: ffff9f34d2bd9060
RDX: ffffbb05c3610000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9f338684b8f0 R08: 0000000000000000 R09: ffff9f394b439e00
R10: ffff9f34f03c6800 R11: ffff9f34f2798800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9f37dda06f00 R15: ffffffff88aeb369
FS:  0000000000000000(0000) GS:ffff9f39bfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000198119420 CR3: 0000000101630000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc0279632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff88aec6b0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff88ae3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff88aed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffff8897853e>] ? deadline_add_request+0x6e/0x90
[<ffffffff88953939>] __blk_run_queue+0x39/0x50
[<ffffffff88957743>] blk_queue_bio+0x3b3/0x400
[<ffffffff889555c7>] generic_make_request+0x147/0x380
[<ffffffff88955870>] submit_bio+0x70/0x150
[<ffffffffc053df31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc053e26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff887cb001>] do_writepages+0x21/0x50
[<ffffffff8887d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffff886c6e55>] ? wake_up_bit+0x25/0x30
[<ffffffff8887e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8887e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8887ed03>] wb_writeback+0x263/0x2f0
[<ffffffff8887f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff886bddcf>] process_one_work+0x17f/0x440
[<ffffffff886beee6>] worker_thread+0x126/0x3c0
[<ffffffff886bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff886c5da1>] kthread+0xd1/0xe0
[<ffffffff886c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff88d95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff886c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #148712
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb858b4d5
Build id: 3.10.0-1160.24.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0121632
Build id: 3.10.0-1160.24.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb82ec6b0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb82e3fa0
Build id: 3.10.0-1160.24.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb82ed6cc
Build id: 3.10.0-1160.24.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb817853e
Build id: 3.10.0-1160.24.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffb8153939
Build id: 3.10.0-1160.24.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffb8157743
Build id: 3.10.0-1160.24.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffb81555c7
Build id: 3.10.0-1160.24.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffb8155870
Build id: 3.10.0-1160.24.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0367f31
Build id: 3.10.0-1160.24.1.el7.x86_64
0x61
12
xfs_do_writepage
xfs 0xffffffffc0368724
Build id: 3.10.0-1160.24.1.el7.x86_64
0x494
13
write_cache_pages
vmlinux 0xffffffffb7fc9fac
Build id: 3.10.0-1160.24.1.el7.x86_64
0x21c
14
? xfs_vm_writepages
xfs 0xffffffffc0368290
Build id: 3.10.0-1160.24.1.el7.x86_64
0xa0
15
? pick_next_entity
vmlinux 0xffffffffb7ee2499
Build id: 3.10.0-1160.24.1.el7.x86_64
0xa9
16
? set_next_entity
vmlinux 0xffffffffb7ee273c
Build id: 3.10.0-1160.24.1.el7.x86_64
0x3c
17
? __switch_to
vmlinux 0xffffffffb7e2b59e
Build id: 3.10.0-1160.24.1.el7.x86_64
0xce
18
xfs_vm_writepages
xfs 0xffffffffc036825b
Build id: 3.10.0-1160.24.1.el7.x86_64
0x6b
19
do_writepages
vmlinux 0xffffffffb7fcb001
Build id: 3.10.0-1160.24.1.el7.x86_64
0x21
20
__writeback_single_inode
vmlinux 0xffffffffb807d9a0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x40
21
writeback_sb_inodes
vmlinux 0xffffffffb807e524
Build id: 3.10.0-1160.24.1.el7.x86_64
0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffffb807e82f
Build id: 3.10.0-1160.24.1.el7.x86_64
0x9f
23
wb_writeback
vmlinux 0xffffffffb807ed03
Build id: 3.10.0-1160.24.1.el7.x86_64
0x263
24
bdi_writeback_workfn
vmlinux 0xffffffffb807f8fb
Build id: 3.10.0-1160.24.1.el7.x86_64
0x2cb
25
process_one_work
vmlinux 0xffffffffb7ebddcf
Build id: 3.10.0-1160.24.1.el7.x86_64
0x17f
26
worker_thread
vmlinux 0xffffffffb7ebeee6
Build id: 3.10.0-1160.24.1.el7.x86_64
0x126
27
? manage_workers.isra.26
vmlinux 0xffffffffb7ebedc0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x2a0
28
kthread
vmlinux 0xffffffffb7ec5da1
Build id: 3.10.0-1160.24.1.el7.x86_64
0xd1
29
? insert_kthread_work
vmlinux 0xffffffffb7ec5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffffb8595df7
Build id: 3.10.0-1160.24.1.el7.x86_64
0x21
31
? insert_kthread_work
vmlinux 0xffffffffb7ec5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/u4:0:31140]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd pcspkr vmw_balloon joydev sg parport_pc i2c_piix4 parport vmw_vmci binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif vmwgfx crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel drm serio_raw
libata nfit libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 31140 Comm: kworker/u4:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9e6ab6a9c200 ti: ffff9e6c6e6ec000 task.ti: ffff9e6c6e6ec000
RIP: 0010:[<ffffffffb858b4d5>]  [<ffffffffb858b4d5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9e6c6e6ef770  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffffb81b28d3 RCX: ffff9e6ba9fae060
RDX: ffffb3cd01be0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff9e6c6e6ef770 R08: 0000000000000000 R09: ffff9e6ac76bcd00
R10: ffff9e6d716f7c00 R11: ffff9e6d71062800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9e6c7ea1c480 R15: ffffffffb82eb369
FS:  0000000000000000(0000) GS:ffff9e6d7fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ef569f369 CR3: 00000001e49a8000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc0121632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffb82ec6b0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffb82e3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb82ed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffffb817853e>] ? deadline_add_request+0x6e/0x90
[<ffffffffb8153939>] __blk_run_queue+0x39/0x50
[<ffffffffb8157743>] blk_queue_bio+0x3b3/0x400
[<ffffffffb81555c7>] generic_make_request+0x147/0x380
[<ffffffffb8155870>] submit_bio+0x70/0x150
[<ffffffffc0367f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc0368724>] xfs_do_writepage+0x494/0x550 [xfs]
[<ffffffffb7fc9fac>] write_cache_pages+0x21c/0x470
[<ffffffffc0368290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffb7ee2499>] ? pick_next_entity+0xa9/0x190
[<ffffffffb7ee273c>] ? set_next_entity+0x3c/0xe0
[<ffffffffb7e2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc036825b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb7fcb001>] do_writepages+0x21/0x50
[<ffffffffb807d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffffb807e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb807e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb807ed03>] wb_writeback+0x263/0x2f0
[<ffffffffb807f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb7ebddcf>] process_one_work+0x17f/0x440
[<ffffffffb7ebeee6>] worker_thread+0x126/0x3c0
[<ffffffffb7ebedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb7ec5da1>] kthread+0xd1/0xe0
[<ffffffffb7ec5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb8595df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb7ec5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #136841
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9778b495
Build id: 3.10.0-1160.21.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0093632
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff974ec580
Build id: 3.10.0-1160.21.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff974e3e70
Build id: 3.10.0-1160.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff974ed59c
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9737840e
Build id: 3.10.0-1160.21.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff97353809
Build id: 3.10.0-1160.21.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff97357613
Build id: 3.10.0-1160.21.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff97355497
Build id: 3.10.0-1160.21.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff97355740
Build id: 3.10.0-1160.21.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc02baf31
Build id: 3.10.0-1160.21.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc02bb26f
Build id: 3.10.0-1160.21.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff971caef1
Build id: 3.10.0-1160.21.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff9727d870
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffff970c6e55
Build id: 3.10.0-1160.21.1.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffff9727e3f4
Build id: 3.10.0-1160.21.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff9727e6ff
Build id: 3.10.0-1160.21.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff9727ebd3
Build id: 3.10.0-1160.21.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffff9726a7fc
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff9727f7cb
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff970bddcf
Build id: 3.10.0-1160.21.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff970beee6
Build id: 3.10.0-1160.21.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff970bedc0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff970c5da1
Build id: 3.10.0-1160.21.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff970c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff97795df7
Build id: 3.10.0-1160.21.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffff970c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 24s! [kworker/u37:2:1644]
Modules linked in: binfmt_misc tcp_diag inet_diag nf_conntrack_ftp tq_base(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc nfit libnvdimm ppdev coretemp vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic drm ata_piix libata crct10dif_pclmul vmxnet3
crct10dif_common crc32c_intel serio_raw vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 1644 Comm: kworker/u37:2 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8ec17bf75280 ti: ffff8ec2b471c000 task.ti: ffff8ec2b471c000
RIP: 0010:[<ffffffff9778b495>]  [<ffffffff9778b495>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8ec2b471f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff973b27a3 RCX: ffffffff97c34f40
RDX: ffffa56d03660000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8ec2b471f8f0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8ec175f9e400 R11: ffff8ec6143d8000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8ec2b4f1cc00 R15: ffffffff974eb239
FS:  0000000000000000(0000) GS:ffff8ec2e5100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe218009600 CR3: 00000006033f0000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0093632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff974ec580>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff974e3e70>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff974ed59c>] scsi_request_fn+0x4ac/0x680
[<ffffffff9737840e>] ? deadline_add_request+0x6e/0x90
[<ffffffff97353809>] __blk_run_queue+0x39/0x50
[<ffffffff97357613>] blk_queue_bio+0x3b3/0x400
[<ffffffff97355497>] generic_make_request+0x147/0x380
[<ffffffff97355740>] submit_bio+0x70/0x150
[<ffffffffc02baf31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc02bb26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff971caef1>] do_writepages+0x21/0x50
[<ffffffff9727d870>] __writeback_single_inode+0x40/0x260
[<ffffffff970c6e55>] ? wake_up_bit+0x25/0x30
[<ffffffff9727e3f4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9727e6ff>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9727ebd3>] wb_writeback+0x263/0x2f0
[<ffffffff9726a7fc>] ? get_nr_inodes+0x4c/0x70
[<ffffffff9727f7cb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff970bddcf>] process_one_work+0x17f/0x440
[<ffffffff970beee6>] worker_thread+0x126/0x3c0
[<ffffffff970bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff970c5da1>] kthread+0xd1/0xe0
[<ffffffff970c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff97795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff970c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #125909
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02343be
Build id: 3.10.0-1160.21.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc024f71e
Build id: 3.10.0-1160.21.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc026d2e0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff976e3e70
Build id: 3.10.0-1160.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff976ed59c
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9757840e
Build id: 3.10.0-1160.21.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff97553809
Build id: 3.10.0-1160.21.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff97557613
Build id: 3.10.0-1160.21.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff97555497
Build id: 3.10.0-1160.21.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff97555740
Build id: 3.10.0-1160.21.1.el7.x86_64
0x70
11
? __switch_to
vmlinux 0xffffffff9722b59e
Build id: 3.10.0-1160.21.1.el7.x86_64
0xce
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc04b7f31
Build id: 3.10.0-1160.21.1.el7.x86_64
0x61
13
xfs_vm_writepages
xfs 0xffffffffc04b826f
Build id: 3.10.0-1160.21.1.el7.x86_64
0x7f
14
do_writepages
vmlinux 0xffffffff973caef1
Build id: 3.10.0-1160.21.1.el7.x86_64
0x21
15
__writeback_single_inode
vmlinux 0xffffffff9747d870
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
16
? wake_up_bit
vmlinux 0xffffffff972c6e55
Build id: 3.10.0-1160.21.1.el7.x86_64
0x25
17
writeback_sb_inodes
vmlinux 0xffffffff9747e3f4
Build id: 3.10.0-1160.21.1.el7.x86_64
0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffff9747e6ff
Build id: 3.10.0-1160.21.1.el7.x86_64
0x9f
19
wb_writeback
vmlinux 0xffffffff9747ebd3
Build id: 3.10.0-1160.21.1.el7.x86_64
0x263
20
bdi_writeback_workfn
vmlinux 0xffffffff9747f7cb
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff972bddcf
Build id: 3.10.0-1160.21.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff972beee6
Build id: 3.10.0-1160.21.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff972bedc0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff972c5da1
Build id: 3.10.0-1160.21.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff972c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff97995df7
Build id: 3.10.0-1160.21.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffff972c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [kworker/u256:0:117488]
Modules linked in: binfmt_misc ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_REJECT nf_reject_ipv4 xt_limit xt_conntrack nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr sg vmw_balloon joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix drm libahci nfit crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm libata mptspi scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks
dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 117488 Comm: kworker/u256:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff91f672fc6300 ti: ffff91f5f4278000 task.ti: ffff91f5f4278000
RIP: 0010:[<ffffffffc02343be>]  [<ffffffffc02343be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff91f5f427b888  EFLAGS: 00000246
RAX: ffffa43440700000 RBX: ffff91f5a2c4c7d0 RCX: 0000000000000006
RDX: ffff91f5767cae00 RSI: ffff91f575f30000 RDI: 000000000000000e
RBP: ffff91f5f427b898 R08: 0000000000000003 R09: 000000000f41bfff
R10: ffff91f5fbaef800 R11: ffff91f5fb9ef800 R12: ffff91f5f9e72580
R13: ffff91f5f427b7fc R14: ffffffff9755c22b R15: ffff91f5f427b840
FS:  0000000000000000(0000) GS:ffff91f6796c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000563e2e3ffb90 CR3: 0000000048810000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc024f71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc026d2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff976e3e70>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff976ed59c>] scsi_request_fn+0x4ac/0x680
[<ffffffff9757840e>] ? deadline_add_request+0x6e/0x90
[<ffffffff97553809>] __blk_run_queue+0x39/0x50
[<ffffffff97557613>] blk_queue_bio+0x3b3/0x400
[<ffffffff97555497>] generic_make_request+0x147/0x380
[<ffffffff97555740>] submit_bio+0x70/0x150
[<ffffffff9722b59e>] ? __switch_to+0xce/0x580
[<ffffffffc04b7f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04b826f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff973caef1>] do_writepages+0x21/0x50
[<ffffffff9747d870>] __writeback_single_inode+0x40/0x260
[<ffffffff972c6e55>] ? wake_up_bit+0x25/0x30
[<ffffffff9747e3f4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9747e6ff>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9747ebd3>] wb_writeback+0x263/0x2f0
[<ffffffff9747f7cb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff972bddcf>] process_one_work+0x17f/0x440
[<ffffffff972beee6>] worker_thread+0x126/0x3c0
[<ffffffff972bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff972c5da1>] kthread+0xd1/0xe0
[<ffffffff972c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff97995df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff972c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 24 
Complete report #96308
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00923be
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc00c971e
Build id: 3.10.0-1160.15.2.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00e22e0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9dae3e70
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9daed59c
Build id: 3.10.0-1160.15.2.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9d97846e
Build id: 3.10.0-1160.15.2.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9d953869
Build id: 3.10.0-1160.15.2.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9d957673
Build id: 3.10.0-1160.15.2.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9d9554f7
Build id: 3.10.0-1160.15.2.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff9d9557a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0310f31
Build id: 3.10.0-1160.15.2.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc031126f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff9d7caf51
Build id: 3.10.0-1160.15.2.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff9d87d8d0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffff9d6c6ec5
Build id: 3.10.0-1160.15.2.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffff9d87e454
Build id: 3.10.0-1160.15.2.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff9d87e75f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff9d87ec33
Build id: 3.10.0-1160.15.2.el7.x86_64
0x263
19
bdi_writeback_workfn
vmlinux 0xffffffff9d87f82b
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffff9d6bde3f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff9d6bef56
Build id: 3.10.0-1160.15.2.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff9d6bee30
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff9d6c5e11
Build id: 3.10.0-1160.15.2.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff9d6c5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff9dd94df7
Build id: 3.10.0-1160.15.2.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffff9d6c5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [kworker/u256:1:79261]
Modules linked in: tcp_diag inet_diag talpa_vfshook(OE) talpa_pedconnector(OE) talpa_pedevice(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) event_collector_2_1_364486(OE) binfmt_misc xt_physdev br_netfilter ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd pcspkr
sg vmw_balloon joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata nfit crc32c_intel libnvdimm mptspi serio_raw scsi_transport_spi mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 79261 Comm: kworker/u256:1 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff88a4a54f8000 ti: ffff88a4af484000 task.ti: ffff88a4af484000
RIP: 0010:[<ffffffffc00923be>]  [<ffffffffc00923be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff88a4af487888  EFLAGS: 00000246
RAX: ffffa67b40cc0000 RBX: ffff88a478ff2ab0 RCX: 0000000000000015
RDX: ffff88a533c93400 RSI: ffff88a534226000 RDI: 000000000000000e
RBP: ffff88a4af487898 R08: 0000000000000003 R09: 00000001737e6fff
R10: ffff88a533cc8000 R11: ffff88a4e01cbb80 R12: ffff88a4c7eb0900
R13: ffff88a4af4877fc R14: ffffffff9d95c28b R15: ffff88a4af487840
FS:  0000000000000000(0000) GS:ffff88a5356c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f02d0020000 CR3: 00000000babee000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc00c971e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00e22e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9dae3e70>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9daed59c>] scsi_request_fn+0x4ac/0x680
[<ffffffff9d97846e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9d953869>] __blk_run_queue+0x39/0x50
[<ffffffff9d957673>] blk_queue_bio+0x3b3/0x400
[<ffffffff9d9554f7>] generic_make_request+0x147/0x380
[<ffffffff9d9557a0>] submit_bio+0x70/0x150
[<ffffffffc0310f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc031126f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9d7caf51>] do_writepages+0x21/0x50
[<ffffffff9d87d8d0>] __writeback_single_inode+0x40/0x260
[<ffffffff9d6c6ec5>] ? wake_up_bit+0x25/0x30
[<ffffffff9d87e454>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9d87e75f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9d87ec33>] wb_writeback+0x263/0x2f0
[<ffffffff9d87f82b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9d6bde3f>] process_one_work+0x17f/0x440
[<ffffffff9d6bef56>] worker_thread+0x126/0x3c0
[<ffffffff9d6bee30>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9d6c5e11>] kthread+0xd1/0xe0
[<ffffffff9d6c5d40>] ? insert_kthread_work+0x40/0x40
[<ffffffff9dd94df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9d6c5d40>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 0a 
Complete report #77304
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa05883c5
Build id: 3.10.0-1127.8.2.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0196632
Build id: 3.10.0-1127.8.2.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffa02eaff0
Build id: 3.10.0-1127.8.2.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffa02e28e0
Build id: 3.10.0-1127.8.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa02ec00c
Build id: 3.10.0-1127.8.2.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffa017710e
Build id: 3.10.0-1127.8.2.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa0152749
Build id: 3.10.0-1127.8.2.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa0156453
Build id: 3.10.0-1127.8.2.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa01543d7
Build id: 3.10.0-1127.8.2.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffa0154680
Build id: 3.10.0-1127.8.2.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc03e9071
Build id: 3.10.0-1127.8.2.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc03e93af
Build id: 3.10.0-1127.8.2.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff9ffcb211
Build id: 3.10.0-1127.8.2.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffa007cc30
Build id: 3.10.0-1127.8.2.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffff9fec7745
Build id: 3.10.0-1127.8.2.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffffa007d7c4
Build id: 3.10.0-1127.8.2.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffa007dacf
Build id: 3.10.0-1127.8.2.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffa007dfb3
Build id: 3.10.0-1127.8.2.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffa0069c0c
Build id: 3.10.0-1127.8.2.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffa007ebab
Build id: 3.10.0-1127.8.2.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff9febe6bf
Build id: 3.10.0-1127.8.2.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff9febf7d6
Build id: 3.10.0-1127.8.2.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff9febf6b0
Build id: 3.10.0-1127.8.2.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff9fec6691
Build id: 3.10.0-1127.8.2.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff9fec65c0
Build id: 3.10.0-1127.8.2.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffa0592d1d
Build id: 3.10.0-1127.8.2.el7.x86_64
0x7
27
? insert_kthread_work
vmlinux 0xffffffff9fec65c0
Build id: 3.10.0-1127.8.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 52s! [kworker/u12:2:8807]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw vmw_balloon gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c
sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod sysimgblt fb_sys_fops crc_t10dif ttm crct10dif_generic ata_piix drm nfit libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 8807 Comm: kworker/u12:2 Kdump: loaded Tainted: G        W      ------------   3.10.0-1127.8.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffffa0b04a608000 ti: ffffa0b01104c000 task.ti: ffffa0b01104c000
RIP: 0010:[<ffffffffa05883c5>]  [<ffffffffa05883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffffa0b01104f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffa01b1433 RCX: ffffa0aebaecd060
RDX: ffffb4c801ba0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffffa0b01104f8f0 R08: 0000000000000000 R09: ffffa0addb297b00
R10: ffffa0b06e9f2800 R11: ffffa0b06e409000 R12: 0000000000000000
R13: 0000000000000020 R14: ffffa0ae9e243200 R15: ffffffffa02e9ca9
FS:  0000000000000000(0000) GS:ffffa0b07fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005574a65c2000 CR3: 000000032f016000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc0196632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffa02eaff0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffa02e28e0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa02ec00c>] scsi_request_fn+0x4ac/0x680
[<ffffffffa017710e>] ? deadline_add_request+0x6e/0x90
[<ffffffffa0152749>] __blk_run_queue+0x39/0x50
[<ffffffffa0156453>] blk_queue_bio+0x3b3/0x400
[<ffffffffa01543d7>] generic_make_request+0x147/0x380
[<ffffffffa0154680>] submit_bio+0x70/0x150
[<ffffffffc03e9071>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc03e93af>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9ffcb211>] do_writepages+0x21/0x50
[<ffffffffa007cc30>] __writeback_single_inode+0x40/0x260
[<ffffffff9fec7745>] ? wake_up_bit+0x25/0x30
[<ffffffffa007d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa007dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa007dfb3>] wb_writeback+0x263/0x2f0
[<ffffffffa0069c0c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa007ebab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9febe6bf>] process_one_work+0x17f/0x440
[<ffffffff9febf7d6>] worker_thread+0x126/0x3c0
[<ffffffff9febf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9fec6691>] kthread+0xd1/0xe0
[<ffffffff9fec65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa0592d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff9fec65c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #71235
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc019f3be
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02cd71e
Build id: 3.10.0-1160.15.2.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02f62e0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffbb8e3e70
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffbb8ed59c
Build id: 3.10.0-1160.15.2.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffbb77846e
Build id: 3.10.0-1160.15.2.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffbb753869
Build id: 3.10.0-1160.15.2.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffbb757673
Build id: 3.10.0-1160.15.2.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffbb7554f7
Build id: 3.10.0-1160.15.2.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffbb7557a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0407f31
Build id: 3.10.0-1160.15.2.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc040826f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffbb5caf51
Build id: 3.10.0-1160.15.2.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffbb67d8d0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffffbb4c6ec5
Build id: 3.10.0-1160.15.2.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffffbb67e454
Build id: 3.10.0-1160.15.2.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffbb67e75f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffbb67ec33
Build id: 3.10.0-1160.15.2.el7.x86_64
0x263
19
bdi_writeback_workfn
vmlinux 0xffffffffbb67f82b
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffffbb4bde3f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffffbb4bef56
Build id: 3.10.0-1160.15.2.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffffbb4bee30
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffffbb4c5e11
Build id: 3.10.0-1160.15.2.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffffbb4c5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffffbbb94df7
Build id: 3.10.0-1160.15.2.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffffbb4c5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u128:0:4107]
Modules linked in: binfmt_misc xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd pcspkr sg vmw_balloon joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common mptspi crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops scsi_transport_spi ttm serio_raw mptscsih drm e1000 ata_piix libata mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 4107 Comm: kworker/u128:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff89bd509d2100 ti: ffff89bd516a0000 task.ti: ffff89bd516a0000
RIP: 0010:[<ffffffffc019f3be>]  [<ffffffffc019f3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff89bd516a3888  EFLAGS: 00000246
RAX: ffffa9a940900000 RBX: ffff89bd480663d0 RCX: 0000000000000008
RDX: ffff89bded91f400 RSI: ffff89bdf7be5000 RDI: 000000000000000e
RBP: ffff89bd516a3898 R08: 0000000000000003 R09: 00000000b00e6fff
R10: ffff89bded509400 R11: ffff89bd5a31ebc0 R12: ffff89bdf0b79e00
R13: ffff89bd516a37fc R14: ffffffffbb75c28b R15: ffff89bd516a3840
FS:  0000000000000000(0000) GS:ffff89bdfa6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f31dcf26000 CR3: 0000000130dea000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc02cd71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02f62e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffbb8e3e70>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbb8ed59c>] scsi_request_fn+0x4ac/0x680
[<ffffffffbb77846e>] ? deadline_add_request+0x6e/0x90
[<ffffffffbb753869>] __blk_run_queue+0x39/0x50
[<ffffffffbb757673>] blk_queue_bio+0x3b3/0x400
[<ffffffffbb7554f7>] generic_make_request+0x147/0x380
[<ffffffffbb7557a0>] submit_bio+0x70/0x150
[<ffffffffc0407f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc040826f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffbb5caf51>] do_writepages+0x21/0x50
[<ffffffffbb67d8d0>] __writeback_single_inode+0x40/0x260
[<ffffffffbb4c6ec5>] ? wake_up_bit+0x25/0x30
[<ffffffffbb67e454>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffbb67e75f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbb67ec33>] wb_writeback+0x263/0x2f0
[<ffffffffbb67f82b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbb4bde3f>] process_one_work+0x17f/0x440
[<ffffffffbb4bef56>] worker_thread+0x126/0x3c0
[<ffffffffbb4bee30>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffbb4c5e11>] kthread+0xd1/0xe0
[<ffffffffbb4c5d40>] ? insert_kthread_work+0x40/0x40
[<ffffffffbbb94df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbb4c5d40>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 d3 1a 
Complete report #67633
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8cb89845
Build id: 3.10.0-1160.6.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc03c7632
Build id: 3.10.0-1160.6.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8c8ec0e0
Build id: 3.10.0-1160.6.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c8e39d0
Build id: 3.10.0-1160.6.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c8ed0fc
Build id: 3.10.0-1160.6.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8c777fce
Build id: 3.10.0-1160.6.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8c753519
Build id: 3.10.0-1160.6.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff8c757223
Build id: 3.10.0-1160.6.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff8c7551a7
Build id: 3.10.0-1160.6.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff8c755450
Build id: 3.10.0-1160.6.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc062af31
Build id: 3.10.0-1160.6.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc062b26f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff8c5cae21
Build id: 3.10.0-1160.6.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8c67d730
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffff8c4c6cd5
Build id: 3.10.0-1160.6.1.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffff8c67e2b4
Build id: 3.10.0-1160.6.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff8c67e5bf
Build id: 3.10.0-1160.6.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff8c67ea93
Build id: 3.10.0-1160.6.1.el7.x86_64
0x263
19
bdi_writeback_workfn
vmlinux 0xffffffff8c67f68b
Build id: 3.10.0-1160.6.1.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffff8c4bdc4f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff8c4bed66
Build id: 3.10.0-1160.6.1.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff8c4bec40
Build id: 3.10.0-1160.6.1.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff8c4c5c21
Build id: 3.10.0-1160.6.1.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff8c4c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff8cb94df7
Build id: 3.10.0-1160.6.1.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffff8c4c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 39s! [kworker/u256:0:66884]
Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm dns_resolver bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper ata_piix syscopyarea ahci sysfillrect sysimgblt fb_sys_fops ttm libahci crct10dif_pclmul crct10dif_common crc32c_intel drm libata nfit libnvdimm serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: CyProtectDrv]
CPU: 7 PID: 66884 Comm: kworker/u256:0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8e7ce6bbd280 ti: ffff8e7f07b98000 task.ti: ffff8e7f07b98000
RIP: 0010:[<ffffffff8cb89845>]  [<ffffffff8cb89845>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8e7f07b9b8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8c7b2363 RCX: ffff8e839e779060
RDX: ffffb85e83130000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8e7f07b9b8f0 R08: 0000000000000000 R09: ffff8e7ccc38be00
R10: ffff8e7cd8bfb000 R11: ffff8e7cd83de000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8e7fa6b0a280 R15: ffffffff8c8ead99
FS:  0000000000000000(0000) GS:ffff8e839d7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff471dac0a0 CR3: 0000000147bb0000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc03c7632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8c8ec0e0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8c8e39d0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c8ed0fc>] scsi_request_fn+0x4ac/0x680
[<ffffffff8c777fce>] ? deadline_add_request+0x6e/0x90
[<ffffffff8c753519>] __blk_run_queue+0x39/0x50
[<ffffffff8c757223>] blk_queue_bio+0x3b3/0x400
[<ffffffff8c7551a7>] generic_make_request+0x147/0x380
[<ffffffff8c755450>] submit_bio+0x70/0x150
[<ffffffffc062af31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc062b26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff8c5cae21>] do_writepages+0x21/0x50
[<ffffffff8c67d730>] __writeback_single_inode+0x40/0x260
[<ffffffff8c4c6cd5>] ? wake_up_bit+0x25/0x30
[<ffffffff8c67e2b4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8c67e5bf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8c67ea93>] wb_writeback+0x263/0x2f0
[<ffffffff8c67f68b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8c4bdc4f>] process_one_work+0x17f/0x440
[<ffffffff8c4bed66>] worker_thread+0x126/0x3c0
[<ffffffff8c4bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8c4c5c21>] kthread+0xd1/0xe0
[<ffffffff8c4c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8cb94df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8c4c5b50>] ? insert_kthread_work+0x40/0x40
Complete report #21090
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9d7883c5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc02a0632
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff9d4eb0a0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff9d4e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9d4ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9d3771ae
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9d3527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9d3564b3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9d354437
Build id: 3.10.0-1127.19.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff9d3546e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc04defb1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc04df2ef
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff9d1cb261
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff9d27cc40
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffff9d0c7745
Build id: 3.10.0-1127.19.1.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffff9d27d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff9d27dacf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff9d27dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffff9d269c5c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff9d27eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff9d0be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff9d0bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff9d0bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff9d0c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff9d0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff9d792d1d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7
27
? insert_kthread_work
vmlinux 0xffffffff9d0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [kworker/u4:0:7428]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink arc4 md4 nls_utf8 cifs ccm dns_resolver ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel sg lrw gf128mul glue_helper pcspkr joydev vmw_vmci ablk_helper cryptd i2c_piix4 parport_pc parport ip_tables xfs libcrc32c
sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod crc_t10dif sysimgblt fb_sys_fops crct10dif_generic ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ahci libahci libata vmxnet3 nfit vmw_pvscsi libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 7428 Comm: kworker/u4:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8b3dbb68d230 ti: ffff8b3ef2ca0000 task.ti: ffff8b3ef2ca0000
RIP: 0010:[<ffffffff9d7883c5>]  [<ffffffff9d7883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8b3ef2ca38f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff9d3b14b3 RCX: ffffffff9dc34e40
RDX: ffff9bb240eb0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8b3ef2ca38f0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8b3f346ab000 R11: ffff8b3db5f6b800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8b3e3b9f6600 R15: ffffffff9d4e9d59
FS:  0000000000000000(0000) GS:ffff8b3f3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000ec4f3000 CR3: 00000000ba4c4000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc02a0632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff9d4eb0a0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff9d4e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9d4ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff9d3771ae>] ? deadline_add_request+0x6e/0x90
[<ffffffff9d3527a9>] __blk_run_queue+0x39/0x50
[<ffffffff9d3564b3>] blk_queue_bio+0x3b3/0x400
[<ffffffff9d354437>] generic_make_request+0x147/0x380
[<ffffffff9d3546e0>] submit_bio+0x70/0x150
[<ffffffffc04defb1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04df2ef>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9d1cb261>] do_writepages+0x21/0x50
[<ffffffff9d27cc40>] __writeback_single_inode+0x40/0x260
[<ffffffff9d0c7745>] ? wake_up_bit+0x25/0x30
[<ffffffff9d27d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9d27dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9d27dfa3>] wb_writeback+0x263/0x2f0
[<ffffffff9d269c5c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff9d27eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9d0be6bf>] process_one_work+0x17f/0x440
[<ffffffff9d0bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff9d0bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9d0c6691>] kthread+0xd1/0xe0
[<ffffffff9d0c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9d792d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff9d0c65c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #17415
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff89583815
Build id: 3.10.0-1062.18.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0306632
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff892e72d0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff892decd0
Build id: 3.10.0-1062.18.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff892e833c
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff89173aae
Build id: 3.10.0-1062.18.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8914f289
Build id: 3.10.0-1062.18.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff89152fa3
Build id: 3.10.0-1062.18.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff89150f37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff891511e0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x70
11
? __switch_to
vmlinux 0xffffffff88e2b59e
Build id: 3.10.0-1062.18.1.el7.x86_64
0xce
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc056acd1
Build id: 3.10.0-1062.18.1.el7.x86_64
0x61
13
xfs_vm_writepages
xfs 0xffffffffc056b00f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x7f
14
do_writepages
vmlinux 0xffffffff88fca6a1
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
15
__writeback_single_inode
vmlinux 0xffffffff89079b00
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
16
? wake_up_bit
vmlinux 0xffffffff88ec73d5
Build id: 3.10.0-1062.18.1.el7.x86_64
0x25
17
writeback_sb_inodes
vmlinux 0xffffffff8907a694
Build id: 3.10.0-1062.18.1.el7.x86_64
0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffff8907a99f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x9f
19
wb_writeback
vmlinux 0xffffffff8907ae83
Build id: 3.10.0-1062.18.1.el7.x86_64
0x263
20
bdi_writeback_workfn
vmlinux 0xffffffff8907ba7b
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff88ebe34f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff88ebf466
Build id: 3.10.0-1062.18.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff88ebf340
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff88ec6321
Build id: 3.10.0-1062.18.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff88ec6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff8958dd37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffff88ec6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 95s! [kworker/u16:31:500]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea crc_t10dif sysfillrect crct10dif_generic
sysimgblt fb_sys_fops ttm ahci libahci ata_piix drm nfit libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 500 Comm: kworker/u16:31 Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/03/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8dc47b19b150 ti: ffff8dc47b1a8000 task.ti: ffff8dc47b1a8000
RIP: 0010:[<ffffffff89583815>]  [<ffffffff89583815>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8dc47b1ab8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff891adbd3 RCX: ffff8dc4f92cb060
RDX: ffffb08c007a0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8dc47b1ab8f0 R08: 0000000000000000 R09: ffff8dc458023300
R10: ffff8dc3f646e0d8 R11: ffff8dc4f5b85000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8dc471194a80 R15: ffffffff892e6029
FS:  0000000000000000(0000) GS:ffff8dc4ffd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f60f2dea000 CR3: 0000000035e40000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0306632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff892e72d0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff892decd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff892e833c>] scsi_request_fn+0x4cc/0x680
[<ffffffff89173aae>] ? deadline_add_request+0x6e/0x90
[<ffffffff8914f289>] __blk_run_queue+0x39/0x50
[<ffffffff89152fa3>] blk_queue_bio+0x3b3/0x400
[<ffffffff89150f37>] generic_make_request+0x147/0x380
[<ffffffff891511e0>] submit_bio+0x70/0x150
[<ffffffff88e2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc056acd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc056b00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff88fca6a1>] do_writepages+0x21/0x50
[<ffffffff89079b00>] __writeback_single_inode+0x40/0x260
[<ffffffff88ec73d5>] ? wake_up_bit+0x25/0x30
[<ffffffff8907a694>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8907a99f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8907ae83>] wb_writeback+0x263/0x2f0
[<ffffffff8907ba7b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff88ebe34f>] process_one_work+0x17f/0x440
[<ffffffff88ebf466>] worker_thread+0x126/0x3c0
[<ffffffff88ebf340>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff88ec6321>] kthread+0xd1/0xe0
[<ffffffff88ec6250>] ? insert_kthread_work+0x40/0x40
[<ffffffff8958dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff88ec6250>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #10421
Frame # Function Binary Source or offset Line
1
mptscsih_qcmd
mptscsih 0xffffffffc040f71e
Build id: 3.10.0-1062.18.1.el7.x86_64
0x24e
2
mptspi_qcmd
mptspi 0xffffffffc045b2e0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x50
3
scsi_dispatch_cmd
vmlinux 0xffffffffbb8decd0
Build id: 3.10.0-1062.18.1.el7.x86_64
0xb0
4
scsi_request_fn
vmlinux 0xffffffffbb8e833c
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4cc
5
? deadline_add_request
vmlinux 0xffffffffbb773aae
Build id: 3.10.0-1062.18.1.el7.x86_64
0x6e
6
__blk_run_queue
vmlinux 0xffffffffbb74f289
Build id: 3.10.0-1062.18.1.el7.x86_64
0x39
7
blk_queue_bio
vmlinux 0xffffffffbb752fa3
Build id: 3.10.0-1062.18.1.el7.x86_64
0x3b3
8
generic_make_request
vmlinux 0xffffffffbb750f37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x147
9
submit_bio
vmlinux 0xffffffffbb7511e0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x70
10
xfs_submit_ioend.isra.12
xfs 0xffffffffc0631cd1
Build id: 3.10.0-1062.18.1.el7.x86_64
0x61
11
xfs_vm_writepages
xfs 0xffffffffc063200f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x7f
12
do_writepages
vmlinux 0xffffffffbb5ca6a1
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
13
__writeback_single_inode
vmlinux 0xffffffffbb679b00
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
14
? wake_up_bit
vmlinux 0xffffffffbb4c73d5
Build id: 3.10.0-1062.18.1.el7.x86_64
0x25
15
writeback_sb_inodes
vmlinux 0xffffffffbb67a694
Build id: 3.10.0-1062.18.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffbb67a99f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffffbb67ae83
Build id: 3.10.0-1062.18.1.el7.x86_64
0x263
18
? get_nr_inodes
vmlinux 0xffffffffbb666d2c
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4c
19
bdi_writeback_workfn
vmlinux 0xffffffffbb67ba7b
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffffbb4be34f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffffbb4bf466
Build id: 3.10.0-1062.18.1.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffffbb4bf340
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffffbb4c6321
Build id: 3.10.0-1062.18.1.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffffbb4c6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffffbbb8dd37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffffbb4c6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 50s! [xfsaild/dm-0:584]
RBP: ffff9564e4f0f898 R08: 0000000000000003 R09: 00000001b20bafff
R10: ffff9565f165a0d8 R11: ffff9565f138f9c0 R12: ffff9564cc128480
R13: ffff9564e4f0f7fc R14: ffffffffbb757aeb R15: ffff9564e4f0f840
FS:  0000000000000000(0000) GS:ffff9565f6680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000847c8ea CR3: 000000010c11c000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc040f71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc045b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffbb8decd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbb8e833c>] scsi_request_fn+0x4cc/0x680
[<ffffffffbb773aae>] ? deadline_add_request+0x6e/0x90
[<ffffffffbb74f289>] __blk_run_queue+0x39/0x50
[<ffffffffbb752fa3>] blk_queue_bio+0x3b3/0x400
[<ffffffffbb750f37>] generic_make_request+0x147/0x380
[<ffffffffbb7511e0>] submit_bio+0x70/0x150
[<ffffffffc0631cd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc063200f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffbb5ca6a1>] do_writepages+0x21/0x50
[<ffffffffbb679b00>] __writeback_single_inode+0x40/0x260
[<ffffffffbb4c73d5>] ? wake_up_bit+0x25/0x30
[<ffffffffbb67a694>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffbb67a99f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbb67ae83>] wb_writeback+0x263/0x2f0
[<ffffffffbb666d2c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffbb67ba7b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbb4be34f>] process_one_work+0x17f/0x440
[<ffffffffbb4bf466>] worker_thread+0x126/0x3c0
[<ffffffffbb4bf340>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffbb4c6321>] kthread+0xd1/0xe0
[<ffffffffbb4c6250>] ? insert_kthread_work+0x40/0x40
[<ffffffffbbb8dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbb4c6250>] ? insert_kthread_work+0x40/0x40
Complete report #546
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01ea3be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc020c71e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02922e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffab0e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffab0ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffaaf771ae
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffaaf527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffaaf564b3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffaaf54437
Build id: 3.10.0-1127.19.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffaaf546e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0480fb1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc04812ef
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffaadcb261
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffaae7cc40
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
15
? wake_up_bit
vmlinux 0xffffffffaacc7745
Build id: 3.10.0-1127.19.1.el7.x86_64
0x25
16
writeback_sb_inodes
vmlinux 0xffffffffaae7d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffaae7dacf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffaae7dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffaae69c5c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffaae7eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffaacbe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffaacbf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffaacbf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffaacc6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffaacc65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffab392d24
Build id: 3.10.0-1127.19.1.el7.x86_64
0xe
27
? insert_kthread_work
vmlinux 0xffffffffaacc65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 32s! [kworker/u256:2:88713]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event edac_mce_amd crc32_pclmul ghash_clmulni_intel ext4 mbcache jbd2 aesni_intel ppdev snd_ens1371 lrw gf128mul glue_helper
snd_rawmidi ablk_helper vmw_balloon cryptd snd_ac97_codec ac97_bus snd_seq joydev pcspkr snd_seq_device snd_pcm snd_timer snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif ata_generic crct10dif_generic cdrom pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect ahci sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common nfit libahci ata_piix crc32c_intel libnvdimm drm mptspi libata serio_raw scsi_transport_spi pcnet32 mptscsih mptbase drm_panel_orientation_quirks mii
CPU: 2 PID: 88713 Comm: kworker/u256:2 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff9a7aebfb9070 ti: ffff9a7b4098c000 task.ti: ffff9a7b4098c000
RIP: 0010:[<ffffffffc01ea3be>]  [<ffffffffc01ea3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9a7b4098f888  EFLAGS: 00000246
RAX: ffffb27900880000 RBX: ffff9a7a5c5d7000 RCX: 000000000000001d
RDX: ffff9a7b72392400 RSI: ffff9a7a75c75000 RDI: 000000000000000e
RBP: ffff9a7b4098f898 R08: 0000000000000002 R09: 000000001087cfff
R10: ffff9a7b726df400 R11: ffff9a7adb188fc0 R12: ffff9a7aafb66280
R13: ffff9a7b4098f7fc R14: ffffffffaaf5b0cb R15: ffff9a7b4098f840
FS:  00007f5a9b2eb700(0000) GS:ffff9a7b79680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f7553a62000 CR3: 00000000ba0b8000 CR4: 00000000000407e0
Call Trace:
[<ffffffffc020c71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02922e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffab0e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffab0ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffffaaf771ae>] ? deadline_add_request+0x6e/0x90
[<ffffffffaaf527a9>] __blk_run_queue+0x39/0x50
[<ffffffffaaf564b3>] blk_queue_bio+0x3b3/0x400
[<ffffffffaaf54437>] generic_make_request+0x147/0x380
[<ffffffffaaf546e0>] submit_bio+0x70/0x150
[<ffffffffc0480fb1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04812ef>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffaadcb261>] do_writepages+0x21/0x50
[<ffffffffaae7cc40>] __writeback_single_inode+0x40/0x260
[<ffffffffaacc7745>] ? wake_up_bit+0x25/0x30
[<ffffffffaae7d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffaae7dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffaae7dfa3>] wb_writeback+0x263/0x2f0
[<ffffffffaae69c5c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffaae7eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffaacbe6bf>] process_one_work+0x17f/0x440
[<ffffffffaacbf7d6>] worker_thread+0x126/0x3c0
[<ffffffffaacbf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffaacc6691>] kthread+0xd1/0xe0
[<ffffffffaacc65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffab392d24>] ret_from_fork_nospec_begin+0xe/0x21
[<ffffffffaacc65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 83 1f 
Complete report #1070228
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03513be
Build id: 3.10.0-1160.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02ce71e
Build id: 3.10.0-1160.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03ed2e0
Build id: 3.10.0-1160.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9d2e3980
Build id: 3.10.0-1160.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9d2ed0ac
Build id: 3.10.0-1160.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9d177f8e
Build id: 3.10.0-1160.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9d1534d9
Build id: 3.10.0-1160.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9d1571e3
Build id: 3.10.0-1160.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9d155167
Build id: 3.10.0-1160.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff9d155410
Build id: 3.10.0-1160.el7.x86_64
0x70
11
? __switch_to
vmlinux 0xffffffff9ce2b59e
Build id: 3.10.0-1160.el7.x86_64
0xce
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc0544f31
Build id: 3.10.0-1160.el7.x86_64
0x61
13
xfs_vm_writepages
xfs 0xffffffffc054526f
Build id: 3.10.0-1160.el7.x86_64
0x7f
14
do_writepages
vmlinux 0xffffffff9cfcade1
Build id: 3.10.0-1160.el7.x86_64
0x21
15
__writeback_single_inode
vmlinux 0xffffffff9d07d6f0
Build id: 3.10.0-1160.el7.x86_64
0x40
16
writeback_sb_inodes
vmlinux 0xffffffff9d07e274
Build id: 3.10.0-1160.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff9d07e57f
Build id: 3.10.0-1160.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff9d07ea53
Build id: 3.10.0-1160.el7.x86_64
0x263
19
bdi_writeback_workfn
vmlinux 0xffffffff9d07f64b
Build id: 3.10.0-1160.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffff9cebdc4f
Build id: 3.10.0-1160.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff9cebed66
Build id: 3.10.0-1160.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff9cebec40
Build id: 3.10.0-1160.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff9cec5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff9cec5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff9d593df7
Build id: 3.10.0-1160.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffff9cec5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u256:1:46261]
Modules linked in: tcp_lp rfcomm xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock bnep sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul snd_ens1371 snd_rawmidi ghash_clmulni_intel snd_ac97_codec ppdev ac97_bus snd_seq aesni_intel btusb lrw btrtl gf128mul btbcm glue_helper ablk_helper btintel snd_seq_device vmw_balloon cryptd bluetooth snd_pcm joydev pcspkr rfkill snd_timer snd sg vmw_vmci soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod
cdrom ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops pata_acpi ttm mptspi crct10dif_pclmul scsi_transport_spi crct10dif_common mptscsih ata_piix nfit e1000 crc32c_intel mptbase serio_raw drm libnvdimm libata drm_panel_orientation_quirks fuse
CPU: 2 PID: 46261 Comm: kworker/u256:1 Kdump: loaded Tainted: G        W      ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff935c37fad280 ti: ffff935d0aed4000 task.ti: ffff935d0aed4000
RIP: 0010:[<ffffffffc03513be>]  [<ffffffffc03513be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff935d0aed7888  EFLAGS: 00010246
RAX: ffffa73800980000 RBX: ffff935cb2ef2000 RCX: 000000000000001d
RDX: ffff935d2c7b6a00 RSI: ffff935d3678e000 RDI: 000000000000000e
RBP: ffff935d0aed7898 R08: 0000000000000002 R09: 0000000037720fff
R10: ffff935c35e43400 R11: ffff935c35707b80 R12: ffff935c02a30c00
R13: ffff935d0aed77fc R14: ffffffff9d15bdfb R15: ffff935d0aed7840
FS:  0000000000000000(0000) GS:ffff935d39680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4f17a7f200 CR3: 00000000a07f8000 CR4: 0000000000360fe0
Call Trace:
[<ffffffffc02ce71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03ed2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9d2e3980>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9d2ed0ac>] scsi_request_fn+0x4ac/0x680
[<ffffffff9d177f8e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9d1534d9>] __blk_run_queue+0x39/0x50
[<ffffffff9d1571e3>] blk_queue_bio+0x3b3/0x400
[<ffffffff9d155167>] generic_make_request+0x147/0x380
[<ffffffff9d155410>] submit_bio+0x70/0x150
[<ffffffff9ce2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc0544f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc054526f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9cfcade1>] do_writepages+0x21/0x50
[<ffffffff9d07d6f0>] __writeback_single_inode+0x40/0x260
[<ffffffff9d07e274>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9d07e57f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9d07ea53>] wb_writeback+0x263/0x2f0
[<ffffffff9d07f64b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9cebdc4f>] process_one_work+0x17f/0x440
[<ffffffff9cebed66>] worker_thread+0x126/0x3c0
[<ffffffff9cebec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9cec5c21>] kthread+0xd1/0xe0
[<ffffffff9cec5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff9d593df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9cec5b50>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 f3 35 
Complete report #738582
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01223be
Build id: 3.10.0-1062.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc01c771e
Build id: 3.10.0-1062.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00e92e0
Build id: 3.10.0-1062.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff93edc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff93ee5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff93d716de
Build id: 3.10.0-1062.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff93d4cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff93d50c63
Build id: 3.10.0-1062.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff93d4ebf7
Build id: 3.10.0-1062.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff93d4eea0
Build id: 3.10.0-1062.el7.x86_64
0x70
11
? __schedule
vmlinux 0xffffffff9417eb31
Build id: 3.10.0-1062.el7.x86_64
0x351
12
? __switch_to
vmlinux 0xffffffff93a2b59e
Build id: 3.10.0-1062.el7.x86_64
0xce
13
xfs_submit_ioend.isra.12
xfs 0xffffffffc034ccd1
Build id: 3.10.0-1062.el7.x86_64
0x61
14
xfs_vm_writepages
xfs 0xffffffffc034d00f
Build id: 3.10.0-1062.el7.x86_64
0x7f
15
do_writepages
vmlinux 0xffffffff93bc8d31
Build id: 3.10.0-1062.el7.x86_64
0x21
16
__writeback_single_inode
vmlinux 0xffffffff93c776c0
Build id: 3.10.0-1062.el7.x86_64
0x40
17
writeback_sb_inodes
vmlinux 0xffffffff93c78254
Build id: 3.10.0-1062.el7.x86_64
0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffff93c7855f
Build id: 3.10.0-1062.el7.x86_64
0x9f
19
wb_writeback
vmlinux 0xffffffff93c78a43
Build id: 3.10.0-1062.el7.x86_64
0x263
20
bdi_writeback_workfn
vmlinux 0xffffffff93c7963b
Build id: 3.10.0-1062.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff93abd0ff
Build id: 3.10.0-1062.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff93abe216
Build id: 3.10.0-1062.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff93abe0f0
Build id: 3.10.0-1062.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff93ac50d1
Build id: 3.10.0-1062.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff93ac5000
Build id: 3.10.0-1062.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff9418bd1d
Build id: 3.10.0-1062.el7.x86_64
0x7
27
? insert_kthread_work
vmlinux 0xffffffff93ac5000
Build id: 3.10.0-1062.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 35s! [kworker/u256:1:110864]
Modules linked in: nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_vmci ac97_bus aesni_intel ppdev snd_seq lrw gf128mul glue_helper snd_seq_device vmw_balloon snd_pcm ablk_helper pcspkr sg cryptd snd_timer joydev i2c_piix4 snd soundcore parport_pc parport pcc_cpufreq ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih libata mptbase e1000 nfit libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 110864 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8e5f78f61070 ti: ffff8e5962ba4000 task.ti: ffff8e5962ba4000
RIP: 0010:[<ffffffffc01223be>]  [<ffffffffc01223be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8e5962ba7888  EFLAGS: 00000246
RAX: ffffb1ab062c0000 RBX: ffff8e639f7ee000 RCX: 0000000000000028
RDX: ffff8e63b80c7400 RSI: ffff8e63bf5a9000 RDI: 000000000000000e
RBP: ffff8e5962ba7898 R08: 0000000000000002 R09: 0000000f9b51dfff
R10: ffff8e63bf5a80d8 R11: ffff8e63b6762840 R12: ffff8e634bbc7c00
R13: ffff8e5962ba77fc R14: ffffffff93d557ab R15: ffff8e5962ba7840
FS:  0000000000000000(0000) GS:ffff8e63bd680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002bdc61b48000 CR3: 0000000fe1620000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffc01c771e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00e92e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff93edc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff93ee5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff93d716de>] ? deadline_add_request+0x6e/0x90
[<ffffffff93d4cf59>] __blk_run_queue+0x39/0x50
[<ffffffff93d50c63>] blk_queue_bio+0x3b3/0x400
[<ffffffff93d4ebf7>] generic_make_request+0x147/0x380
[<ffffffff93d4eea0>] submit_bio+0x70/0x150
[<ffffffff9417eb31>] ? __schedule+0x351/0x9c0
[<ffffffff93a2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc034ccd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc034d00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff93bc8d31>] do_writepages+0x21/0x50
[<ffffffff93c776c0>] __writeback_single_inode+0x40/0x260
[<ffffffff93c78254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff93c7855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff93c78a43>] wb_writeback+0x263/0x2f0
[<ffffffff93c7963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff93abd0ff>] process_one_work+0x17f/0x440
[<ffffffff93abe216>] worker_thread+0x126/0x3c0
[<ffffffff93abe0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff93ac50d1>] kthread+0xd1/0xe0
[<ffffffff93ac5000>] ? insert_kthread_work+0x40/0x40
[<ffffffff9418bd1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff93ac5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 13 
Complete report #651670
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03003be
Build id: 3.10.0-862.9.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02de71e
Build id: 3.10.0-862.9.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02b92e0
Build id: 3.10.0-862.9.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffbd09bb60
Build id: 3.10.0-862.9.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffbd0a50cc
Build id: 3.10.0-862.9.1.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffbcf3d28e
Build id: 3.10.0-862.9.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffbcf19519
Build id: 3.10.0-862.9.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffbcf1d1a3
Build id: 3.10.0-862.9.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffbcf1b0fb
Build id: 3.10.0-862.9.1.el7.x86_64
0x10b
10
submit_bio
vmlinux 0xffffffffbcf1b380
Build id: 3.10.0-862.9.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc05254b1
Build id: 3.10.0-862.9.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc0525671
Build id: 3.10.0-862.9.1.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffffbcda15e1
Build id: 3.10.0-862.9.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffbce49260
Build id: 3.10.0-862.9.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffffbce49cf4
Build id: 3.10.0-862.9.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffbce4a05f
Build id: 3.10.0-862.9.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffffbce4a893
Build id: 3.10.0-862.9.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffbce4b21b
Build id: 3.10.0-862.9.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffffbccb35ef
Build id: 3.10.0-862.9.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffbccb4686
Build id: 3.10.0-862.9.1.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffffbccb4560
Build id: 3.10.0-862.9.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffbccbb621
Build id: 3.10.0-862.9.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffbccbb550
Build id: 3.10.0-862.9.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffbd3205f7
Build id: 3.10.0-862.9.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffbccbb550
Build id: 3.10.0-862.9.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u256:1:80388]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel snd_ens1371 lrw snd_rawmidi gf128mul glue_helper ablk_helper cryptd snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm joydev pcspkr sg snd_timer nfit snd soundcore i2c_piix4 libnvdimm vmw_vmci shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom
crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops ttm ata_piix serio_raw drm libata mptspi scsi_transport_spi e1000 mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 80388 Comm: kworker/u256:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-862.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff8e433be50fd0 ti: ffff8e41b87ac000 task.ti: ffff8e41b87ac000
RIP: 0010:[<ffffffffc03003be>]  [<ffffffffc03003be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8e41b87af888  EFLAGS: 00000246
RAX: ffffae9e43200000 RBX: ffff8e4712f88000 RCX: 0000000000000014
RDX: ffff8e41b5cc5400 RSI: ffff8e48eef0a000 RDI: 000000000000000e
RBP: ffff8e41b87af898 R08: 0000000000000002 R09: 000000063339dfff
R10: ffff8e48f4f280d8 R11: ffff8e45a569fb80 R12: ffff8e465b880c00
R13: ffff8e41b87af7fc R14: ffffffffbcf21d0b R15: ffff8e41b87af840
FS:  0000000000000000(0000) GS:ffff8e48f3000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000cf5d360 CR3: 0000000771924000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc02de71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02b92e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffbd09bb60>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbd0a50cc>] scsi_request_fn+0x4cc/0x680
[<ffffffffbcf3d28e>] ? deadline_add_request+0x6e/0x90
[<ffffffffbcf19519>] __blk_run_queue+0x39/0x50
[<ffffffffbcf1d1a3>] blk_queue_bio+0x3b3/0x400
[<ffffffffbcf1b0fb>] generic_make_request+0x10b/0x320
[<ffffffffbcf1b380>] submit_bio+0x70/0x150
[<ffffffffc05254b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0525671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffffbcda15e1>] do_writepages+0x21/0x50
[<ffffffffbce49260>] __writeback_single_inode+0x40/0x260
[<ffffffffbce49cf4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbce4a05f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbce4a893>] wb_writeback+0x263/0x2f0
[<ffffffffbce4b21b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbccb35ef>] process_one_work+0x17f/0x440
[<ffffffffbccb4686>] worker_thread+0x126/0x3c0
[<ffffffffbccb4560>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffbccbb621>] kthread+0xd1/0xe0
[<ffffffffbccbb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd3205f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbccbb550>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 30 
Complete report #533271
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9ed8b795
Build id: 3.10.0-1160.41.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0098632
Build id: 3.10.0-1160.41.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff9eaecf50
Build id: 3.10.0-1160.41.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff9eae4840
Build id: 3.10.0-1160.41.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9eaedf6c
Build id: 3.10.0-1160.41.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9e978bce
Build id: 3.10.0-1160.41.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9e953f19
Build id: 3.10.0-1160.41.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9e957cc3
Build id: 3.10.0-1160.41.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9e955ba7
Build id: 3.10.0-1160.41.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff9e955e50
Build id: 3.10.0-1160.41.1.el7.x86_64
0x70
11
? pick_next_entity
vmlinux 0xffffffff9e6e22d9
Build id: 3.10.0-1160.41.1.el7.x86_64
0xa9
12
? set_next_entity
vmlinux 0xffffffff9e6e26ec
Build id: 3.10.0-1160.41.1.el7.x86_64
0x3c
13
xfs_submit_ioend.isra.12
xfs 0xffffffffc02ebf31
Build id: 3.10.0-1160.41.1.el7.x86_64
0x61
14
xfs_vm_writepages
xfs 0xffffffffc02ec26f
Build id: 3.10.0-1160.41.1.el7.x86_64
0x7f
15
do_writepages
vmlinux 0xffffffff9e7cb9b1
Build id: 3.10.0-1160.41.1.el7.x86_64
0x21
16
__writeback_single_inode
vmlinux 0xffffffff9e87def0
Build id: 3.10.0-1160.41.1.el7.x86_64
0x40
17
writeback_sb_inodes
vmlinux 0xffffffff9e87ea74
Build id: 3.10.0-1160.41.1.el7.x86_64
0x1c4
18
wb_writeback
vmlinux 0xffffffff9e87f0ef
Build id: 3.10.0-1160.41.1.el7.x86_64
0xff
19
bdi_writeback_workfn
vmlinux 0xffffffff9e87fc95
Build id: 3.10.0-1160.41.1.el7.x86_64
0x115
20
process_one_work
vmlinux 0xffffffff9e6bde8f
Build id: 3.10.0-1160.41.1.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff9e6befa6
Build id: 3.10.0-1160.41.1.el7.x86_64
0x126
22
? manage_workers.isra.26
vmlinux 0xffffffff9e6bee80
Build id: 3.10.0-1160.41.1.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff9e6c5e61
Build id: 3.10.0-1160.41.1.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff9e6c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff9ed95df7
Build id: 3.10.0-1160.41.1.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffff9e6c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [kworker/u20:0:22646]
Modules linked in: binfmt_misc xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg vmw_balloon pcspkr joydev vmw_vmci parport_pc parport i2c_piix4
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops nfit ttm sd_mod libata libnvdimm crc_t10dif crct10dif_generic drm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 7 PID: 22646 Comm: kworker/u20:0 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8c948feb8000 ti: ffff8c9f260bc000 task.ti: ffff8c9f260bc000
RIP: 0010:[<ffffffff9ed8b795>]  [<ffffffff9ed8b795>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8c9f260bf940  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff9e9b2f63 RCX: ffffffff9f234f40
RDX: ffffa257cce40000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8c9f260bf940 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8ca31a31cc00 R11: ffff8ca319d82000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8c9952b06180 R15: ffffffff9eaebc09
FS:  0000000000000000(0000) GS:ffff8ca31f380000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fff1a662000 CR3: 0000001718c68000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0098632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff9eaecf50>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff9eae4840>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9eaedf6c>] scsi_request_fn+0x4ac/0x680
[<ffffffff9e978bce>] ? deadline_add_request+0x6e/0x90
[<ffffffff9e953f19>] __blk_run_queue+0x39/0x50
[<ffffffff9e957cc3>] blk_queue_bio+0x3b3/0x400
[<ffffffff9e955ba7>] generic_make_request+0x147/0x380
[<ffffffff9e955e50>] submit_bio+0x70/0x150
[<ffffffff9e6e22d9>] ? pick_next_entity+0xa9/0x190
[<ffffffff9e6e26ec>] ? set_next_entity+0x3c/0xe0
[<ffffffffc02ebf31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc02ec26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9e7cb9b1>] do_writepages+0x21/0x50
[<ffffffff9e87def0>] __writeback_single_inode+0x40/0x260
[<ffffffff9e87ea74>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9e87f0ef>] wb_writeback+0xff/0x2f0
[<ffffffff9e87fc95>] bdi_writeback_workfn+0x115/0x460
[<ffffffff9e6bde8f>] process_one_work+0x17f/0x440
[<ffffffff9e6befa6>] worker_thread+0x126/0x3c0
[<ffffffff9e6bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9e6c5e61>] kthread+0xd1/0xe0
[<ffffffff9e6c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffff9ed95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9e6c5d90>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #532558
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa6d8b795
Build id: 3.10.0-1160.45.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc023e632
Build id: 3.10.0-1160.45.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffa6aecf70
Build id: 3.10.0-1160.45.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffa6ae4860
Build id: 3.10.0-1160.45.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa6aedf8c
Build id: 3.10.0-1160.45.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffa6978bee
Build id: 3.10.0-1160.45.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa6953f39
Build id: 3.10.0-1160.45.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa6957ce3
Build id: 3.10.0-1160.45.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa6955bc7
Build id: 3.10.0-1160.45.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffa6955e70
Build id: 3.10.0-1160.45.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc04b6f31
Build id: 3.10.0-1160.45.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc04b726f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffa67cb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffa687df10
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffffa687ea94
Build id: 3.10.0-1160.45.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffa687ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffffa687f273
Build id: 3.10.0-1160.45.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffa687fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffffa66bde8f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffa66befa6
Build id: 3.10.0-1160.45.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffffa66bee80
Build id: 3.10.0-1160.45.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffa66c5e61
Build id: 3.10.0-1160.45.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffa66c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa6d95df7
Build id: 3.10.0-1160.45.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffa66c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [kworker/u8:1:9584]
Modules linked in: eset_rtp(OE) rpcsec_gss_krb5 vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_vmci pcspkr joydev sg vmw_balloon i2c_piix4 parport_pc parport dm_multipath nfsd nfs_acl lockd binfmt_misc grace auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif ahci crct10dif_generic drm_kms_helper ata_piix libahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata crct10dif_pclmul crct10dif_common nfit drm crc32c_intel libnvdimm serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: eset_rtp]
CPU: 3 PID: 9584 Comm: kworker/u8:1 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff89dc4992b180 ti: ffff89dcf865c000 task.ti: ffff89dcf865c000
RIP: 0010:[<ffffffffa6d8b795>]  [<ffffffffa6d8b795>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff89dcf865f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffa69b2f83 RCX: ffff89ddbab21060
RDX: ffffad2780e90000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff89dcf865f8f0 R08: 0000000000000000 R09: ffff89dc4514a400
R10: ffff89de73ac3c00 R11: ffff89de72850800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff89de754ac180 R15: ffffffffa6aebc29
FS:  0000000000000000(0000) GS:ffff89de7fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe798436000 CR3: 000000007b210000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc023e632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffa6aecf70>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffa6ae4860>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa6aedf8c>] scsi_request_fn+0x4ac/0x680
[<ffffffffa6978bee>] ? deadline_add_request+0x6e/0x90
[<ffffffffa6953f39>] __blk_run_queue+0x39/0x50
[<ffffffffa6957ce3>] blk_queue_bio+0x3b3/0x400
[<ffffffffa6955bc7>] generic_make_request+0x147/0x380
[<ffffffffa6955e70>] submit_bio+0x70/0x150
[<ffffffffc04b6f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04b726f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffa67cb9b1>] do_writepages+0x21/0x50
[<ffffffffa687df10>] __writeback_single_inode+0x40/0x260
[<ffffffffa687ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa687ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa687f273>] wb_writeback+0x263/0x2f0
[<ffffffffa687fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa66bde8f>] process_one_work+0x17f/0x440
[<ffffffffa66befa6>] worker_thread+0x126/0x3c0
[<ffffffffa66bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa66c5e61>] kthread+0xd1/0xe0
[<ffffffffa66c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffa6d95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa66c5d90>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #527816
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc037a3be
Build id: 3.10.0-1062.1.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc046571e
Build id: 3.10.0-1062.1.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc047e2e0
Build id: 3.10.0-1062.1.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff97adc890
Build id: 3.10.0-1062.1.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff97ae5efc
Build id: 3.10.0-1062.1.1.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff979716de
Build id: 3.10.0-1062.1.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9794cf59
Build id: 3.10.0-1062.1.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff97950c63
Build id: 3.10.0-1062.1.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9794ebf7
Build id: 3.10.0-1062.1.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff9794eea0
Build id: 3.10.0-1062.1.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc060ccd1
Build id: 3.10.0-1062.1.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc060d00f
Build id: 3.10.0-1062.1.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff977c8d31
Build id: 3.10.0-1062.1.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff978776c0
Build id: 3.10.0-1062.1.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff97878254
Build id: 3.10.0-1062.1.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff9787855f
Build id: 3.10.0-1062.1.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff97878a43
Build id: 3.10.0-1062.1.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9787963b
Build id: 3.10.0-1062.1.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff976bd0ff
Build id: 3.10.0-1062.1.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff976be216
Build id: 3.10.0-1062.1.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff976be0f0
Build id: 3.10.0-1062.1.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff976c50d1
Build id: 3.10.0-1062.1.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff976c5000
Build id: 3.10.0-1062.1.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff97d8cd1d
Build id: 3.10.0-1062.1.1.el7.x86_64
0x7
25
? insert_kthread_work
vmlinux 0xffffffff976c5000
Build id: 3.10.0-1062.1.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [kworker/u256:2:4140]
Modules linked in: resguard_linux(OE) secmodel_linux(OE) syshook_linux(OE) secmodel2(OE) tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr sg i2c_piix4 vmw_vmci parport_pc parport
ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul mptspi crct10dif_common ata_piix crc32c_intel scsi_transport_spi mptscsih libata nfit e1000 libnvdimm serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 4140 Comm: kworker/u256:2 Tainted: G        W  OEL ------------   3.10.0-1062.1.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff94be05e320e0 ti: ffff94be277a8000 task.ti: ffff94be277a8000
RIP: 0010:[<ffffffffc037a3be>]  [<ffffffffc037a3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff94be277ab888  EFLAGS: 00000246
RAX: ffffadaa803c0000 RBX: ffff94be36114000 RCX: 0000000000000005
RDX: ffff94be362b2c00 RSI: ffff94be35dda000 RDI: 000000000000000e
RBP: ffff94be277ab898 R08: 0000000000000002 R09: 0000000049773fff
R10: ffff94be35d5e0d8 R11: ffff94be595d0540 R12: ffff94be0c9e0000
R13: ffff94be277ab7fc R14: ffffffff979557ab R15: ffff94be277ab840
FS:  0000000000000000(0000) GS:ffff94be7b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006373d0 CR3: 0000000057de0000 CR4: 00000000007607f0
PKRU: 00000000
Call Trace:
[<ffffffffc046571e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc047e2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff97adc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff97ae5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff979716de>] ? deadline_add_request+0x6e/0x90
[<ffffffff9794cf59>] __blk_run_queue+0x39/0x50
[<ffffffff97950c63>] blk_queue_bio+0x3b3/0x400
[<ffffffff9794ebf7>] generic_make_request+0x147/0x380
[<ffffffff9794eea0>] submit_bio+0x70/0x150
[<ffffffffc060ccd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc060d00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff977c8d31>] do_writepages+0x21/0x50
[<ffffffff978776c0>] __writeback_single_inode+0x40/0x260
[<ffffffff97878254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9787855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff97878a43>] wb_writeback+0x263/0x2f0
[<ffffffff9787963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff976bd0ff>] process_one_work+0x17f/0x440
[<ffffffff976be216>] worker_thread+0x126/0x3c0
[<ffffffff976be0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff976c50d1>] kthread+0xd1/0xe0
[<ffffffff976c5000>] ? insert_kthread_work+0x40/0x40
[<ffffffff97d8cd1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff976c5000>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 83 38 
Complete report #465935
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffad51b2c5
Build id: 3.10.0-862.11.6.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0125622
Build id: 3.10.0-862.11.6.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffad2a8500
Build id: 3.10.0-862.11.6.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffad2a0010
Build id: 3.10.0-862.11.6.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffad2a957c
Build id: 3.10.0-862.11.6.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffad14122e
Build id: 3.10.0-862.11.6.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffad11d4e9
Build id: 3.10.0-862.11.6.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffad121173
Build id: 3.10.0-862.11.6.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffad11f0cb
Build id: 3.10.0-862.11.6.el7.x86_64
0x10b
10
submit_bio
vmlinux 0xffffffffad11f350
Build id: 3.10.0-862.11.6.el7.x86_64
0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc03634b1
Build id: 3.10.0-862.11.6.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc0363671
Build id: 3.10.0-862.11.6.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffffacfa3ac1
Build id: 3.10.0-862.11.6.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffad04cf00
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffffad04d994
Build id: 3.10.0-862.11.6.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffad04dcff
Build id: 3.10.0-862.11.6.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffffad04e533
Build id: 3.10.0-862.11.6.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffad04edbc
Build id: 3.10.0-862.11.6.el7.x86_64
0x1cc
19
process_one_work
vmlinux 0xffffffffaceb613f
Build id: 3.10.0-862.11.6.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffaceb71d6
Build id: 3.10.0-862.11.6.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffffaceb70b0
Build id: 3.10.0-862.11.6.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffacebdf21
Build id: 3.10.0-862.11.6.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffacebde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffad5255dd
Build id: 3.10.0-862.11.6.el7.x86_64
0x7
25
? insert_kthread_work
vmlinux 0xffffffffacebde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [kworker/u21:0:26266]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_2_4_933667(OE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon ppdev
pcspkr nfit sg joydev libnvdimm parport_pc parport vmw_vmci shpchp i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul libata crct10dif_common crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 26266 Comm: kworker/u21:0 Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9c42b6f48000 ti: ffff9c3d53bdc000 task.ti: ffff9c3d53bdc000
RIP: 0010:[<ffffffffad51b2c5>]  [<ffffffffad51b2c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9c3d53bdf8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffad1795b3 RCX: ffffffffada30880
RDX: ffffab220ce20000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9c3d53bdf8f0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff9c537ff008d8 R11: ffff9c46cb5bc800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9c44f9649c80 R15: ffffffffad2a7259
FS:  0000000000000000(0000) GS:ffff9c46cda00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001c77a50 CR3: 000000190637c000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc0125622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffad2a8500>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffad2a0010>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffad2a957c>] scsi_request_fn+0x4cc/0x680
[<ffffffffad14122e>] ? deadline_add_request+0x6e/0x90
[<ffffffffad11d4e9>] __blk_run_queue+0x39/0x50
[<ffffffffad121173>] blk_queue_bio+0x3b3/0x400
[<ffffffffad11f0cb>] generic_make_request+0x10b/0x320
[<ffffffffad11f350>] submit_bio+0x70/0x150
[<ffffffffc03634b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0363671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffffacfa3ac1>] do_writepages+0x21/0x50
[<ffffffffad04cf00>] __writeback_single_inode+0x40/0x260
[<ffffffffad04d994>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffad04dcff>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffad04e533>] wb_writeback+0x263/0x2f0
[<ffffffffad04edbc>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffaceb613f>] process_one_work+0x17f/0x440
[<ffffffffaceb71d6>] worker_thread+0x126/0x3c0
[<ffffffffaceb70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffacebdf21>] kthread+0xd1/0xe0
[<ffffffffacebde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffad5255dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffacebde50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #409469
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffbd36a0f5
Build id: 3.10.0-957.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc03ba622
Build id: 3.10.0-957.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffbd0d8e90
Build id: 3.10.0-957.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffbd0d0960
Build id: 3.10.0-957.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffbd0d9efc
Build id: 3.10.0-957.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffbcf675fe
Build id: 3.10.0-957.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffbcf42ff9
Build id: 3.10.0-957.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffbcf46cf3
Build id: 3.10.0-957.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffbcf44c87
Build id: 3.10.0-957.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffbcf44f30
Build id: 3.10.0-957.el7.x86_64
0x70
11
? __switch_to
vmlinux 0xffffffffbcc2a59e
Build id: 3.10.0-957.el7.x86_64
0xce
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc04e5251
Build id: 3.10.0-957.el7.x86_64
0x61
13
xfs_vm_writepages
xfs 0xffffffffc04e53bf
Build id: 3.10.0-957.el7.x86_64
0x7f
14
do_writepages
vmlinux 0xffffffffbcdc3181
Build id: 3.10.0-957.el7.x86_64
0x21
15
__writeback_single_inode
vmlinux 0xffffffffbce6f930
Build id: 3.10.0-957.el7.x86_64
0x40
16
writeback_sb_inodes
vmlinux 0xffffffffbce703c4
Build id: 3.10.0-957.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffbce7072f
Build id: 3.10.0-957.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffbce70f63
Build id: 3.10.0-957.el7.x86_64
0x263
19
bdi_writeback_workfn
vmlinux 0xffffffffbce718eb
Build id: 3.10.0-957.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffffbccb9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffffbccbade6
Build id: 3.10.0-957.el7.x86_64
0x126
22
? manage_workers.isra.25
vmlinux 0xffffffffbccbacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffffbccc1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffffbccc1b60
Build id: 3.10.0-957.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffffbd374c1d
Build id: 3.10.0-957.el7.x86_64
0x7
26
? insert_kthread_work
vmlinux 0xffffffffbccc1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [kworker/u16:2:17936]
Modules linked in: tcp_diag inet_diag rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_set nf_tables(T) ip6t_MASQUERADE nf_nat_masquerade_ipv6 xt_comment xt_mark ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack_netlink xt_addrtype xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 nf_log_ipv4 ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common xt_LOG xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables veth vxlan ip6_udp_tunnel udp_tunnel
ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport dummy br_netfilter tun devlink binfmt_misc overlay(T) ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr joydev vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper vmxnet3 ahci syscopyarea sysfillrect sysimgblt fb_sys_fops vmw_pvscsi libahci ttm nfit libnvdimm ata_piix drm libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 3 PID: 17936 Comm: kworker/u16:2 Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff900484040000 ti: ffff900485de4000 task.ti: ffff900485de4000
RIP: 0010:[<ffffffffbd36a0f5>]  [<ffffffffbd36a0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff900485de78f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffbcfa10f3 RCX: ffff90063f763060
RDX: ffffb1a780ea0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff900485de78f0 R08: 0000000000000000 R09: ffff9005810cc700
R10: ffff90063fd6c8d8 R11: ffff900435cea800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9004b72c5200 R15: ffffffffbd0d7be9
FS:  0000000000000000(0000) GS:ffff90063fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007faf75f8b950 CR3: 000000014d7b8000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc03ba622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffbd0d8e90>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffbd0d0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbd0d9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffbcf675fe>] ? deadline_add_request+0x6e/0x90
[<ffffffffbcf42ff9>] __blk_run_queue+0x39/0x50
[<ffffffffbcf46cf3>] blk_queue_bio+0x3b3/0x400
[<ffffffffbcf44c87>] generic_make_request+0x147/0x380
[<ffffffffbcf44f30>] submit_bio+0x70/0x150
[<ffffffffbcc2a59e>] ? __switch_to+0xce/0x580
[<ffffffffc04e5251>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04e53bf>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffbcdc3181>] do_writepages+0x21/0x50
[<ffffffffbce6f930>] __writeback_single_inode+0x40/0x260
[<ffffffffbce703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbce7072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbce70f63>] wb_writeback+0x263/0x2f0
[<ffffffffbce718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbccb9d4f>] process_one_work+0x17f/0x440
[<ffffffffbccbade6>] worker_thread+0x126/0x3c0
[<ffffffffbccbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffbccc1c31>] kthread+0xd1/0xe0
[<ffffffffbccc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd374c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffbccc1b60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #372660
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00823be
Build id: 3.10.0-693.21.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc005e72e
Build id: 3.10.0-693.21.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00692b0
Build id: 3.10.0-693.21.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146d8f0
Build id: 3.10.0-693.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8147678f
Build id: 3.10.0-693.21.1.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812fd1d9
Build id: 3.10.0-693.21.1.el7.x86_64
0x39
7
blk_queue_bio
vmlinux 0xffffffff81300b33
Build id: 3.10.0-693.21.1.el7.x86_64
0x363
8
generic_make_request
vmlinux 0xffffffff812feb7b
Build id: 3.10.0-693.21.1.el7.x86_64
0x10b
9
submit_bio
vmlinux 0xffffffff812fee00
Build id: 3.10.0-693.21.1.el7.x86_64
0x70
10
? __switch_to
vmlinux 0xffffffff8102954d
Build id: 3.10.0-693.21.1.el7.x86_64
0xcd
11
xfs_submit_ioend.isra.16
xfs 0xffffffffc0283071
Build id: 3.10.0-693.21.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc0283231
Build id: 3.10.0-693.21.1.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffff81193531
Build id: 3.10.0-693.21.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff812326d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff81233344
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff812336af
Build id: 3.10.0-693.21.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff81233ee3
Build id: 3.10.0-693.21.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8123438b
Build id: 3.10.0-693.21.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff810abe2f
Build id: 3.10.0-693.21.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff810acaf6
Build id: 3.10.0-693.21.1.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff810ac9d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff810b4031
Build id: 3.10.0-693.21.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
24
ret_from_fork
vmlinux 0xffffffff816c0577
Build id: 3.10.0-693.21.1.el7.x86_64
0x77
25
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/u256:2:84064]
Modules linked in: nls_utf8 isofs fuse vmw_vsock_vmci_transport vsock ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter snd_seq_midi snd_seq_midi_event sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper snd_ens1371 snd_rawmidi snd_ac97_codec ablk_helper cryptd ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore ppdev vmw_balloon joydev sg pcspkr i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata e1000 mptspi scsi_transport_spi serio_raw mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 84064 Comm: kworker/u256:2 Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff8803e4791fa0 ti: ffff8800a3b54000 task.ti: ffff8800a3b54000
RIP: 0010:[<ffffffffc00823be>]  [<ffffffffc00823be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8800a3b57890  EFLAGS: 00000246
RAX: ffffc90001940000 RBX: ffff88041bbe2f00 RCX: 0000000000000002
RDX: ffff8804270b7e00 RSI: ffff88042f56e000 RDI: 000000000000000e
RBP: ffff8800a3b578a0 R08: 0000000000000002 R09: ffff88042e27f0d8
R10: ffff8804276d8a80 R11: ffff8804276d8a80 R12: 0000000100000001
R13: ffff8800a3a0ad00 R14: ffff8800a3b57804 R15: ffffffff813051fb
FS:  0000000000000000(0000) GS:ffff88042d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2dce056480 CR3: 000000041f3a4000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc005e72e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00692b0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8146d8f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8147678f>] scsi_request_fn+0x4df/0x680
[<ffffffff812fd1d9>] __blk_run_queue+0x39/0x50
[<ffffffff81300b33>] blk_queue_bio+0x363/0x3a0
[<ffffffff812feb7b>] generic_make_request+0x10b/0x320
[<ffffffff812fee00>] submit_bio+0x70/0x150
[<ffffffff8102954d>] ? __switch_to+0xcd/0x500
[<ffffffffc0283071>] xfs_submit_ioend.isra.16+0x61/0xe0 [xfs]
[<ffffffffc0283231>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff81193531>] do_writepages+0x21/0x50
[<ffffffff812326d0>] __writeback_single_inode+0x40/0x250
[<ffffffff81233344>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff812336af>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff81233ee3>] wb_writeback+0x263/0x2f0
[<ffffffff8123438b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810abe2f>] process_one_work+0x17f/0x440
[<ffffffff810acaf6>] worker_thread+0x126/0x3c0
[<ffffffff810ac9d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b4031>] kthread+0xd1/0xe0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 09 
Complete report #328157
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01753be
Build id: 3.10.0-1160.45.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc01ce71e
Build id: 3.10.0-1160.45.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02202e0
Build id: 3.10.0-1160.45.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffac4e4860
Build id: 3.10.0-1160.45.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffac4edf8c
Build id: 3.10.0-1160.45.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffac378bee
Build id: 3.10.0-1160.45.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffac353f39
Build id: 3.10.0-1160.45.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffac357ce3
Build id: 3.10.0-1160.45.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffac355bc7
Build id: 3.10.0-1160.45.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffac355e70
Build id: 3.10.0-1160.45.1.el7.x86_64
0x70
11
? __enqueue_entity
vmlinux 0xffffffffac0dfbd8
Build id: 3.10.0-1160.45.1.el7.x86_64
0x78
12
? put_prev_entity
vmlinux 0xffffffffac0e637b
Build id: 3.10.0-1160.45.1.el7.x86_64
0x8b
13
xfs_submit_ioend.isra.12
xfs 0xffffffffc0410f31
Build id: 3.10.0-1160.45.1.el7.x86_64
0x61
14
xfs_vm_writepages
xfs 0xffffffffc041126f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x7f
15
do_writepages
vmlinux 0xffffffffac1cb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64
0x21
16
__writeback_single_inode
vmlinux 0xffffffffac27df10
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
17
writeback_sb_inodes
vmlinux 0xffffffffac27ea94
Build id: 3.10.0-1160.45.1.el7.x86_64
0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffffac27ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x9f
19
wb_writeback
vmlinux 0xffffffffac27f273
Build id: 3.10.0-1160.45.1.el7.x86_64
0x263
20
bdi_writeback_workfn
vmlinux 0xffffffffac27fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffac0bde8f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffac0befa6
Build id: 3.10.0-1160.45.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffac0bee80
Build id: 3.10.0-1160.45.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffac0c5e61
Build id: 3.10.0-1160.45.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffac0c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffac795df7
Build id: 3.10.0-1160.45.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffffac0c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [kworker/u256:2:19367]
Modules linked in: rfcomm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat talpa_vfshook(OE) iptable_mangle talpa_pedconnector(OE) iptable_security talpa_pedevice(OE) iptable_raw talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) nf_conntrack talpa_syscallhook(OE) ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi snd_ens1371 crc32_pclmul ghash_clmulni_intel aesni_intel snd_rawmidi snd_ac97_codec lrw ac97_bus gf128mul btusb snd_seq btrtl snd_seq_device
btbcm btintel glue_helper snd_pcm bluetooth ablk_helper cryptd rfkill snd_timer snd soundcore joydev ppdev sg pcspkr vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi nfit ata_piix drm libnvdimm crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel libata mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 19367 Comm: kworker/u256:2 Kdump: loaded Tainted: P           OEL ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff948cfc032100 ti: ffff948d7aa7c000 task.ti: ffff948d7aa7c000
RIP: 0010:[<ffffffffc01753be>]  [<ffffffffc01753be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff948d7aa7f888  EFLAGS: 00000246
RAX: ffffaf34c1940000 RBX: ffff948ceacac000 RCX: 0000000000000013
RDX: ffff948e0cb80e00 RSI: ffff948e0d3d7000 RDI: 000000000000000e
RBP: ffff948d7aa7f898 R08: 0000000000000002 R09: 00000001d9547fff
R10: ffff948e0bc6d400 R11: ffff948d2a7101c0 R12: ffff948ba2285c80
R13: ffff948d7aa7f7fc R14: ffffffffac35c8fb R15: ffff948d7aa7f840
FS:  0000000000000000(0000) GS:ffff948e11b00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa9a01ce000 CR3: 00000002d096c000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc01ce71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02202e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffac4e4860>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffac4edf8c>] scsi_request_fn+0x4ac/0x680
[<ffffffffac378bee>] ? deadline_add_request+0x6e/0x90
[<ffffffffac353f39>] __blk_run_queue+0x39/0x50
[<ffffffffac357ce3>] blk_queue_bio+0x3b3/0x400
[<ffffffffac355bc7>] generic_make_request+0x147/0x380
[<ffffffffac355e70>] submit_bio+0x70/0x150
[<ffffffffac0dfbd8>] ? __enqueue_entity+0x78/0x80
[<ffffffffac0e637b>] ? put_prev_entity+0x8b/0x400
[<ffffffffc0410f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc041126f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffac1cb9b1>] do_writepages+0x21/0x50
[<ffffffffac27df10>] __writeback_single_inode+0x40/0x260
[<ffffffffac27ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffac27ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffac27f273>] wb_writeback+0x263/0x2f0
[<ffffffffac27fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffac0bde8f>] process_one_work+0x17f/0x440
[<ffffffffac0befa6>] worker_thread+0x126/0x3c0
[<ffffffffac0bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffac0c5e61>] kthread+0xd1/0xe0
[<ffffffffac0c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffac795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffac0c5d90>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 33 18 
Complete report #264372
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8d98b4d5
Build id: 3.10.0-1160.24.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc03e5632
Build id: 3.10.0-1160.24.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8d6ec6b0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8d6e3fa0
Build id: 3.10.0-1160.24.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8d6ed6cc
Build id: 3.10.0-1160.24.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8d57853e
Build id: 3.10.0-1160.24.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8d553939
Build id: 3.10.0-1160.24.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff8d557743
Build id: 3.10.0-1160.24.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff8d5555c7
Build id: 3.10.0-1160.24.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff8d555870
Build id: 3.10.0-1160.24.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0626f31
Build id: 3.10.0-1160.24.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc062726f
Build id: 3.10.0-1160.24.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff8d3cb001
Build id: 3.10.0-1160.24.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8d47d9a0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8d47e524
Build id: 3.10.0-1160.24.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8d47e82f
Build id: 3.10.0-1160.24.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff8d47ed03
Build id: 3.10.0-1160.24.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8d47f8fb
Build id: 3.10.0-1160.24.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff8d2bddcf
Build id: 3.10.0-1160.24.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff8d2beee6
Build id: 3.10.0-1160.24.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff8d2bedc0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff8d2c5da1
Build id: 3.10.0-1160.24.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8d2c5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8d995df7
Build id: 3.10.0-1160.24.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff8d2c5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 30s! [kworker/u4:30:390]
Modules linked in: talpa_vfshook(OE) talpa_pedconnector(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) tcp_diag inet_diag talpa_pedevice(OE) vmw_vsock_vmci_transport vsock nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack xt_multiport iptable_filter sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper sg ablk_helper vmw_balloon cryptd pcspkr joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx ahci libahci ata_piix crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel serio_raw syscopyarea sysfillrect libata sysimgblt fb_sys_fops ttm vmxnet3 drm vmw_pvscsi nfit drm_panel_orientation_quirks libnvdimm
dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: talpa_syscallhook]
CPU: 1 PID: 390 Comm: kworker/u4:30 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8afcf5801080 ti: ffff8afcf580c000 task.ti: ffff8afcf580c000
RIP: 0010:[<ffffffff8d98b4d5>]  [<ffffffff8d98b4d5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8afcf580f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8d5b28d3 RCX: ffff8afeffcb9060
RDX: fffface680f40000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8afcf580f8f0 R08: 0000000000000000 R09: ffff8afccf7fbb00
R10: ffff8afef5dab000 R11: ffff8afcf660e800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8afcf59a2600 R15: ffffffff8d6eb369
FS:  0000000000000000(0000) GS:ffff8afeffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb8fb775000 CR3: 000000017fef6000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc03e5632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8d6ec6b0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8d6e3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8d6ed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffff8d57853e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8d553939>] __blk_run_queue+0x39/0x50
[<ffffffff8d557743>] blk_queue_bio+0x3b3/0x400
[<ffffffff8d5555c7>] generic_make_request+0x147/0x380
[<ffffffff8d555870>] submit_bio+0x70/0x150
[<ffffffffc0626f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc062726f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff8d3cb001>] do_writepages+0x21/0x50
[<ffffffff8d47d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffff8d47e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8d47e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8d47ed03>] wb_writeback+0x263/0x2f0
[<ffffffff8d47f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8d2bddcf>] process_one_work+0x17f/0x440
[<ffffffff8d2beee6>] worker_thread+0x126/0x3c0
[<ffffffff8d2bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8d2c5da1>] kthread+0xd1/0xe0
[<ffffffff8d2c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8d995df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8d2c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #229782
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04873be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc04d671e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03e72e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9fe9b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9fea45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff9fd3c84e
Build id: 3.10.0-862.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9fd18ad9
Build id: 3.10.0-862.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9fd1c763
Build id: 3.10.0-862.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9fd1a6bb
Build id: 3.10.0-862.el7.x86_64
0x10b
10
submit_bio
vmlinux 0xffffffff9fd1a940
Build id: 3.10.0-862.el7.x86_64
0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc06784b1
Build id: 3.10.0-862.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc0678671
Build id: 3.10.0-862.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffff9fba0e21
Build id: 3.10.0-862.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff9fc48af0
Build id: 3.10.0-862.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff9fc49584
Build id: 3.10.0-862.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff9fc498ef
Build id: 3.10.0-862.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff9fc4a123
Build id: 3.10.0-862.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9fc4a9ac
Build id: 3.10.0-862.el7.x86_64
0x1cc
19
process_one_work
vmlinux 0xffffffff9fab2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff9fab3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff9fab39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff9fabae31
Build id: 3.10.0-862.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa011f637
Build id: 3.10.0-862.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 66s! [kworker/u128:0:2698]
Modules linked in: esn_cfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul
syscopyarea crct10dif_common sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2698 Comm: kworker/u128:0 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff88709bea5ee0 ti: ffff887064dec000 task.ti: ffff887064dec000
RIP: 0010:[<ffffffffc04873be>]  [<ffffffffc04873be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff887064def888  EFLAGS: 00000246
RAX: ffffb42600380000 RBX: ffff8870b5afe000 RCX: 0000000000000017
RDX: ffff887075b7a200 RSI: ffff887075b88000 RDI: 000000000000000e
RBP: ffff887064def898 R08: 0000000000000002 R09: 000000001d15efff
R10: ffff887075df10d8 R11: ffff887075b7d880 R12: ffff8870614c0000
R13: ffff887064def7fc R14: ffffffff9fd212cb R15: ffff887064def840
FS:  0000000000000000(0000) GS:ffff8870bc6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b67eed090 CR3: 00000000785fa000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc04d671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03e72e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9fe9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9fea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff9fd3c84e>] ? deadline_add_request+0x6e/0x90
[<ffffffff9fd18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff9fd1c763>] blk_queue_bio+0x3b3/0x400
[<ffffffff9fd1a6bb>] generic_make_request+0x10b/0x320
[<ffffffff9fd1a940>] submit_bio+0x70/0x150
[<ffffffffc06784b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0678671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff9fba0e21>] do_writepages+0x21/0x50
[<ffffffff9fc48af0>] __writeback_single_inode+0x40/0x260
[<ffffffff9fc49584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff9fc498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9fc4a123>] wb_writeback+0x263/0x2f0
[<ffffffff9fc4a9ac>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff9fab2dff>] process_one_work+0x17f/0x440
[<ffffffff9fab3ac6>] worker_thread+0x126/0x3c0
[<ffffffff9fab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff9fabae31>] kthread+0xd1/0xe0
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa011f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 49 
Complete report #187898
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8a78b495
Build id: 3.10.0-1160.21.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0179632
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8a4ec580
Build id: 3.10.0-1160.21.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8a4e3e70
Build id: 3.10.0-1160.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8a4ed59c
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8a37840e
Build id: 3.10.0-1160.21.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8a353809
Build id: 3.10.0-1160.21.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff8a357613
Build id: 3.10.0-1160.21.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff8a355497
Build id: 3.10.0-1160.21.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff8a355740
Build id: 3.10.0-1160.21.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc03d5f31
Build id: 3.10.0-1160.21.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc03d626f
Build id: 3.10.0-1160.21.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff8a1caef1
Build id: 3.10.0-1160.21.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8a27d870
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8a27e3f4
Build id: 3.10.0-1160.21.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8a27e6ff
Build id: 3.10.0-1160.21.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff8a27ebd3
Build id: 3.10.0-1160.21.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8a27f7cb
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff8a0bddcf
Build id: 3.10.0-1160.21.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff8a0beee6
Build id: 3.10.0-1160.21.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff8a0bedc0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff8a0c5da1
Build id: 3.10.0-1160.21.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8a0c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a795df7
Build id: 3.10.0-1160.21.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff8a0c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 28s! [kworker/u25:0:14007]
Modules linked in: vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr sg joydev vmw_balloon parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect sysimgblt crc_t10dif fb_sys_fops crct10dif_generic ttm drm ata_piix crct10dif_pclmul crct10dif_common nfit libata crc32c_intel serio_raw libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 14007 Comm: kworker/u25:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8edff95b4200 ti: ffff8edff0acc000 task.ti: ffff8edff0acc000
RIP: 0010:[<ffffffff8a78b495>]  [<ffffffff8a78b495>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8edff0acf8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8a3b27a3 RCX: ffffffff8ac34f40
RDX: ffffb0a081bd0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8edff0acf8f0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8edff7ea9800 R11: ffff8edff9d3b000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8edfbfda8900 R15: ffffffff8a4eb239
FS:  0000000000000000(0000) GS:ffff8edff9d40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fce6b2ca4b1 CR3: 000000031fa10000 CR4: 00000000000407e0
Call Trace:
[<ffffffffc0179632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8a4ec580>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8a4e3e70>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8a4ed59c>] scsi_request_fn+0x4ac/0x680
[<ffffffff8a37840e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8a353809>] __blk_run_queue+0x39/0x50
[<ffffffff8a357613>] blk_queue_bio+0x3b3/0x400
[<ffffffff8a355497>] generic_make_request+0x147/0x380
[<ffffffff8a355740>] submit_bio+0x70/0x150
[<ffffffffc03d5f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc03d626f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff8a1caef1>] do_writepages+0x21/0x50
[<ffffffff8a27d870>] __writeback_single_inode+0x40/0x260
[<ffffffff8a27e3f4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8a27e6ff>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8a27ebd3>] wb_writeback+0x263/0x2f0
[<ffffffff8a27f7cb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8a0bddcf>] process_one_work+0x17f/0x440
[<ffffffff8a0beee6>] worker_thread+0x126/0x3c0
[<ffffffff8a0bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8a0c5da1>] kthread+0xd1/0xe0
[<ffffffff8a0c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8a0c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #71185
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa196b455
Build id: 3.10.0-957.21.3.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0333622
Build id: 3.10.0-957.21.3.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffa16d9c30
Build id: 3.10.0-957.21.3.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffa16d1700
Build id: 3.10.0-957.21.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa16dac9c
Build id: 3.10.0-957.21.3.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffa1567e6e
Build id: 3.10.0-957.21.3.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa1543849
Build id: 3.10.0-957.21.3.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa1547553
Build id: 3.10.0-957.21.3.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa15454e7
Build id: 3.10.0-957.21.3.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffa1545790
Build id: 3.10.0-957.21.3.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc056f471
Build id: 3.10.0-957.21.3.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc056f5df
Build id: 3.10.0-957.21.3.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffa13c36d1
Build id: 3.10.0-957.21.3.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffa14701b0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffffa1470c44
Build id: 3.10.0-957.21.3.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffa1470faf
Build id: 3.10.0-957.21.3.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffffa14717e3
Build id: 3.10.0-957.21.3.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffa147206c
Build id: 3.10.0-957.21.3.el7.x86_64
0x1cc
19
process_one_work
vmlinux 0xffffffffa12b9ebf
Build id: 3.10.0-957.21.3.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffa12baf56
Build id: 3.10.0-957.21.3.el7.x86_64
0x126
21
? manage_workers.isra.25
vmlinux 0xffffffffa12bae30
Build id: 3.10.0-957.21.3.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffa12c1da1
Build id: 3.10.0-957.21.3.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffa12c1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa1975c1d
Build id: 3.10.0-957.21.3.el7.x86_64
0x7
25
? insert_kthread_work
vmlinux 0xffffffffa12c1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/u4:3:30439]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_11110(E) falcon_lsm_pinned_10807(E) binfmt_misc falcon_lsm_pinned_10703(E) falcon_lsm_pinned_10404(E) falcon_lsm_pinned_9918(E) falcon_lsm_pinned_9808(E) falcon_lsm_pinned_9705(E) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock falcon_lsm_pinned_9607(E) sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables
xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper ahci syscopyarea sysfillrect sysimgblt fb_sys_fops libahci ttm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm libata vmxnet3 nfit vmw_pvscsi drm_panel_orientation_quirks libnvdimm floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_lsm_serviceable]
CPU: 1 PID: 30439 Comm: kworker/u4:3 Tainted: P            E  ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff89049de35140 ti: ffff89048a124000 task.ti: ffff89048a124000
RIP: 0010:[<ffffffffa196b455>]  [<ffffffffa196b455>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff89048a1278f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffa16d89fe RCX: 0000000000001000
RDX: ffffb456803f0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff89048a1278f0 R08: ffffffffffffffff R09: ffff8904895a6a00
R10: ffff8904b5ce40d8 R11: ffff8904b5a59800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8904a1f62600 R15: ffffffffa16d8989
FS:  0000000000000000(0000) GS:ffff8904ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5b48df90e0 CR3: 0000000008082000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc0333622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffa16d9c30>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffa16d1700>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa16dac9c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa1567e6e>] ? deadline_add_request+0x6e/0x90
[<ffffffffa1543849>] __blk_run_queue+0x39/0x50
[<ffffffffa1547553>] blk_queue_bio+0x3b3/0x400
[<ffffffffa15454e7>] generic_make_request+0x147/0x380
[<ffffffffa1545790>] submit_bio+0x70/0x150
[<ffffffffc056f471>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc056f5df>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffa13c36d1>] do_writepages+0x21/0x50
[<ffffffffa14701b0>] __writeback_single_inode+0x40/0x260
[<ffffffffa1470c44>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa1470faf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa14717e3>] wb_writeback+0x263/0x2f0
[<ffffffffa147206c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffa12b9ebf>] process_one_work+0x17f/0x440
[<ffffffffa12baf56>] worker_thread+0x126/0x3c0
[<ffffffffa12bae30>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa12c1da1>] kthread+0xd1/0xe0
[<ffffffffa12c1cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa1975c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa12c1cd0>] ? insert_kthread_work+0x40/0x40
Complete report #58249
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc040e3be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc048f71e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04862e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c4e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c4ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff8c3771ae
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8c3527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff8c3564b3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff8c354437
Build id: 3.10.0-1127.19.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff8c3546e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0626fb1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc06272ef
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff8c1cb261
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8c27cc40
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8c27d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8c27dacf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff8c27dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8c27eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff8c0be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff8c0bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff8c0bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff8c0c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8c0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8c792d1d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7
25
? insert_kthread_work
vmlinux 0xffffffff8c0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/u256:0:4298]
Modules linked in: xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmhgfs(OE) vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit sg libnvdimm pcspkr joydev vmw_balloon parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm ata_piix mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi mptscsih libata mptbase crc32c_intel e1000 drm_panel_orientation_quirks serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 4298 Comm: kworker/u256:0 Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff90d130de1070 ti: ffff90d176864000 task.ti: ffff90d176864000
RIP: 0010:[<ffffffffc040e3be>]  [<ffffffffc040e3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff90d176867888  EFLAGS: 00000246
RAX: ffffb780c0700000 RBX: ffff90d176fed000 RCX: 000000000000000d
RDX: ffff90d0f58dda00 RSI: ffff90d0f5d22000 RDI: 000000000000000e
RBP: ffff90d176867898 R08: 0000000000000002 R09: 000000005880dfff
R10: ffff90d1f1b60c00 R11: ffff90d0f5d281c0 R12: ffff90d13e94d200
R13: ffff90d1768677fc R14: ffffffff8c35b0cb R15: ffff90d176867840
FS:  0000000000000000(0000) GS:ffff90d1f9600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f49e99c2118 CR3: 0000000080090000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc048f71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04862e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8c4e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c4ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff8c3771ae>] ? deadline_add_request+0x6e/0x90
[<ffffffff8c3527a9>] __blk_run_queue+0x39/0x50
[<ffffffff8c3564b3>] blk_queue_bio+0x3b3/0x400
[<ffffffff8c354437>] generic_make_request+0x147/0x380
[<ffffffff8c3546e0>] submit_bio+0x70/0x150
[<ffffffffc0626fb1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc06272ef>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff8c1cb261>] do_writepages+0x21/0x50
[<ffffffff8c27cc40>] __writeback_single_inode+0x40/0x260
[<ffffffff8c27d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8c27dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8c27dfa3>] wb_writeback+0x263/0x2f0
[<ffffffff8c27eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8c0be6bf>] process_one_work+0x17f/0x440
[<ffffffff8c0bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff8c0bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8c0c6691>] kthread+0xd1/0xe0
[<ffffffff8c0c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8c792d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff8c0c65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 c3 41 
Complete report #20754
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8c916475
Build id: 3.10.0-862.3.2.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc02ab622
Build id: 3.10.0-862.3.2.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8c6a39e0
Build id: 3.10.0-862.3.2.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c69b4f0
Build id: 3.10.0-862.3.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c6a4a5c
Build id: 3.10.0-862.3.2.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff8c53cc8e
Build id: 3.10.0-862.3.2.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff8c518f19
Build id: 3.10.0-862.3.2.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff8c51cba3
Build id: 3.10.0-862.3.2.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff8c51aafb
Build id: 3.10.0-862.3.2.el7.x86_64
0x10b
10
submit_bio
vmlinux 0xffffffff8c51ad80
Build id: 3.10.0-862.3.2.el7.x86_64
0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc04df4b1
Build id: 3.10.0-862.3.2.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc04df671
Build id: 3.10.0-862.3.2.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffff8c3a1161
Build id: 3.10.0-862.3.2.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8c448e30
Build id: 3.10.0-862.3.2.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff8c4498c4
Build id: 3.10.0-862.3.2.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8c449c2f
Build id: 3.10.0-862.3.2.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff8c44a463
Build id: 3.10.0-862.3.2.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff8c44adeb
Build id: 3.10.0-862.3.2.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff8c2b312f
Build id: 3.10.0-862.3.2.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff8c2b3df6
Build id: 3.10.0-862.3.2.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff8c2b3cd0
Build id: 3.10.0-862.3.2.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff8c2bb161
Build id: 3.10.0-862.3.2.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8c2bb090
Build id: 3.10.0-862.3.2.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8c920677
Build id: 3.10.0-862.3.2.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff8c2bb090
Build id: 3.10.0-862.3.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 50s! [kworker/u2:0:24987]
Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache binfmt_misc vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev nfit sg libnvdimm parport_pc shpchp parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic drm crct10dif_pclmul ata_piix crct10dif_common crc32c_intel serio_raw libata vmw_pvscsi i2c_core vmxnet3 floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 24987 Comm: kworker/u2:0 Kdump: loaded Tainted: G             L ------------   3.10.0-862.3.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff94a0d1aa1fa0 ti: ffff94a107f6c000 task.ti: ffff94a107f6c000
RIP: 0010:[<ffffffff8c916475>]  [<ffffffff8c916475>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff94a107f6f8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8c574e63 RCX: ffff94a0cc09a060
RDX: ffffb3d841b70000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff94a107f6f8f0 R08: 0000000000000000 R09: ffff94a1276ec700
R10: ffff94a12ba0e8d8 R11: ffff94a12ba0f000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff94a12a238480 R15: ffffffff8c6a2739
FS:  0000000000000000(0000) GS:ffff94a13fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9e2cd76270 CR3: 00000000b2efc000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc02ab622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8c6a39e0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8c69b4f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c6a4a5c>] scsi_request_fn+0x4cc/0x680
[<ffffffff8c53cc8e>] ? deadline_add_request+0x6e/0x90
[<ffffffff8c518f19>] __blk_run_queue+0x39/0x50
[<ffffffff8c51cba3>] blk_queue_bio+0x3b3/0x400
[<ffffffff8c51aafb>] generic_make_request+0x10b/0x320
[<ffffffff8c51ad80>] submit_bio+0x70/0x150
[<ffffffffc04df4b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc04df671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff8c3a1161>] do_writepages+0x21/0x50
[<ffffffff8c448e30>] __writeback_single_inode+0x40/0x260
[<ffffffff8c4498c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8c449c2f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8c44a463>] wb_writeback+0x263/0x2f0
[<ffffffff8c44adeb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8c2b312f>] process_one_work+0x17f/0x440
[<ffffffff8c2b3df6>] worker_thread+0x126/0x3c0
[<ffffffff8c2b3cd0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff8c2bb161>] kthread+0xd1/0xe0
[<ffffffff8c2bb090>] ? insert_kthread_work+0x40/0x40
[<ffffffff8c920677>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8c2bb090>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #18678
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00743be
Build id: 3.10.0-693.21.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc00f672e
Build id: 3.10.0-693.21.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00de2b0
Build id: 3.10.0-693.21.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146d8f0
Build id: 3.10.0-693.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8147678f
Build id: 3.10.0-693.21.1.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812fd1d9
Build id: 3.10.0-693.21.1.el7.x86_64
0x39
7
blk_queue_bio
vmlinux 0xffffffff81300b33
Build id: 3.10.0-693.21.1.el7.x86_64
0x363
8
generic_make_request
vmlinux 0xffffffff812feb7b
Build id: 3.10.0-693.21.1.el7.x86_64
0x10b
9
submit_bio
vmlinux 0xffffffff812fee00
Build id: 3.10.0-693.21.1.el7.x86_64
0x70
10
xfs_submit_ioend.isra.16
xfs 0xffffffffc0276071
Build id: 3.10.0-693.21.1.el7.x86_64
0x61
11
xfs_vm_writepages
xfs 0xffffffffc0276231
Build id: 3.10.0-693.21.1.el7.x86_64
0xd1
12
do_writepages
vmlinux 0xffffffff81193531
Build id: 3.10.0-693.21.1.el7.x86_64
0x21
13
__writeback_single_inode
vmlinux 0xffffffff812326d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
14
writeback_sb_inodes
vmlinux 0xffffffff81233344
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c4
15
__writeback_inodes_wb
vmlinux 0xffffffff812336af
Build id: 3.10.0-693.21.1.el7.x86_64
0x9f
16
wb_writeback
vmlinux 0xffffffff81233ee3
Build id: 3.10.0-693.21.1.el7.x86_64
0x263
17
bdi_writeback_workfn
vmlinux 0xffffffff8123438b
Build id: 3.10.0-693.21.1.el7.x86_64
0x2cb
18
process_one_work
vmlinux 0xffffffff810abe2f
Build id: 3.10.0-693.21.1.el7.x86_64
0x17f
19
worker_thread
vmlinux 0xffffffff810acaf6
Build id: 3.10.0-693.21.1.el7.x86_64
0x126
20
? manage_workers.isra.24
vmlinux 0xffffffff810ac9d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x2a0
21
kthread
vmlinux 0xffffffff810b4031
Build id: 3.10.0-693.21.1.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
23
ret_from_fork
vmlinux 0xffffffff816c0577
Build id: 3.10.0-693.21.1.el7.x86_64
0x77
24
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 79s! [kworker/u4:0:24852]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif
sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix scsi_transport_spi drm crct10dif_pclmul mptscsih crct10dif_common libata crc32c_intel mptbase serio_raw vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 24852 Comm: kworker/u4:0 Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 10/22/2013
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff88005b662f70 ti: ffff880049c7c000 task.ti: ffff880049c7c000
RIP: 0010:[<ffffffffc00743be>]  [<ffffffffc00743be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880049c7f890  EFLAGS: 00000246
RAX: ffffc90000440000 RBX: ffff880077ea7900 RCX: 000000000000001d
RDX: ffff880035e9fa00 RSI: ffff880035019000 RDI: 000000000000000e
RBP: ffff880049c7f8a0 R08: 0000000000000002 R09: 0000000073658fff
R10: ffff880005c9a700 R11: ffff880005c9a700 R12: 0000000100000001
R13: ffff880029c9e180 R14: ffff880049c7f804 R15: ffffffff813051fb
FS:  0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f0478635000 CR3: 000000005365e000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc00f672e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00de2b0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8146d8f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8147678f>] scsi_request_fn+0x4df/0x680
[<ffffffff812fd1d9>] __blk_run_queue+0x39/0x50
[<ffffffff81300b33>] blk_queue_bio+0x363/0x3a0
[<ffffffff812feb7b>] generic_make_request+0x10b/0x320
[<ffffffff812fee00>] submit_bio+0x70/0x150
[<ffffffffc0276071>] xfs_submit_ioend.isra.16+0x61/0xe0 [xfs]
[<ffffffffc0276231>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff81193531>] do_writepages+0x21/0x50
[<ffffffff812326d0>] __writeback_single_inode+0x40/0x250
[<ffffffff81233344>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff812336af>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff81233ee3>] wb_writeback+0x263/0x2f0
[<ffffffff8123438b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810abe2f>] process_one_work+0x17f/0x440
[<ffffffff810acaf6>] worker_thread+0x126/0x3c0
[<ffffffff810ac9d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b4031>] kthread+0xd1/0xe0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 08 
Complete report #4725
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02b83be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02ef71e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc032b2e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff85ee2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff85eec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff85d771ae
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff85d527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff85d564b3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff85d54437
Build id: 3.10.0-1127.19.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff85d546e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc052efb1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc052f2ef
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff85bcb261
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff85c7cc40
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff85c7d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff85c7dacf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff85c7dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff85c7eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff85abe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff85abf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff85abf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff85ac6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff86192d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [kworker/u64:0:15179]
Modules linked in: arc4 md4 nls_utf8 cifs ccm dns_resolver fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel ata_piix serio_raw drm libata mptspi scsi_transport_spi nfit libnvdimm mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmfsrtx_bridge_helper]
CPU: 0 PID: 15179 Comm: kworker/u64:0 Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff92a577b15230 ti: ffff92a55028c000 task.ti: ffff92a55028c000
RIP: 0010:[<ffffffffc02b83be>]  [<ffffffffc02b83be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff92a55028f888  EFLAGS: 00000246
RAX: ffffae5d407c0000 RBX: ffff92a5754093b0 RCX: 000000000000001d
RDX: ffff92a475499400 RSI: ffff92a573aba000 RDI: 000000000000000e
RBP: ffff92a55028f898 R08: 0000000000000003 R09: 000000008a36cfff
R10: ffff92a4754a1400 R11: ffff92a4b9822a00 R12: ffff92a4a3fe0300
R13: ffff92a55028f7fc R14: ffffffff85d5b0cb R15: ffff92a55028f840
FS:  0000000000000000(0000) GS:ffff92a57ae00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc7d39a6000 CR3: 0000000079b58000 CR4: 00000000000007f0
Call Trace:
[<ffffffffc02ef71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc032b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff85ee2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff85eec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff85d771ae>] ? deadline_add_request+0x6e/0x90
[<ffffffff85d527a9>] __blk_run_queue+0x39/0x50
[<ffffffff85d564b3>] blk_queue_bio+0x3b3/0x400
[<ffffffff85d54437>] generic_make_request+0x147/0x380
[<ffffffff85d546e0>] submit_bio+0x70/0x150
[<ffffffffc052efb1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc052f2ef>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff85bcb261>] do_writepages+0x21/0x50
[<ffffffff85c7cc40>] __writeback_single_inode+0x40/0x260
[<ffffffff85c7d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff85c7dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff85c7dfa3>] wb_writeback+0x263/0x2f0
[<ffffffff85c7eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff85abe6bf>] process_one_work+0x17f/0x440
[<ffffffff85abf7d6>] worker_thread+0x126/0x3c0
[<ffffffff85abf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff85ac6691>] kthread+0xd1/0xe0
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff86192d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 2c 
Complete report #4154
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03c43be
Build id: 3.10.0-957.27.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03e671e
Build id: 3.10.0-957.27.2.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03b42e0
Build id: 3.10.0-957.27.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9fed2ee0
Build id: 3.10.0-957.27.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9fedc47c
Build id: 3.10.0-957.27.2.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff9fd694de
Build id: 3.10.0-957.27.2.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9fd44e59
Build id: 3.10.0-957.27.2.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff9fd48b63
Build id: 3.10.0-957.27.2.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9fd46af7
Build id: 3.10.0-957.27.2.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff9fd46da0
Build id: 3.10.0-957.27.2.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc064c471
Build id: 3.10.0-957.27.2.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc064c5df
Build id: 3.10.0-957.27.2.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff9fbc47a1
Build id: 3.10.0-957.27.2.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff9fc71020
Build id: 3.10.0-957.27.2.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff9fc71ab4
Build id: 3.10.0-957.27.2.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff9fc71e1f
Build id: 3.10.0-957.27.2.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff9fc72653
Build id: 3.10.0-957.27.2.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffff9fc72fdb
Build id: 3.10.0-957.27.2.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff9fabaf9f
Build id: 3.10.0-957.27.2.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff9fabc036
Build id: 3.10.0-957.27.2.el7.x86_64
0x126
21
? manage_workers.isra.25
vmlinux 0xffffffff9fabbf10
Build id: 3.10.0-957.27.2.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff9fac2e81
Build id: 3.10.0-957.27.2.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9fac2db0
Build id: 3.10.0-957.27.2.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa0176c37
Build id: 3.10.0-957.27.2.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff9fac2db0
Build id: 3.10.0-957.27.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 36s! [kworker/u128:2:55375]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_hda_codec_generic sb_edac snd_hda_intel coretemp snd_hda_codec iosf_mbi crc32_pclmul snd_hda_core ghash_clmulni_intel snd_hwdep snd_seq ppdev aesni_intel snd_seq_device vmw_balloon lrw gf128mul
glue_helper ablk_helper snd_pcm cryptd joydev snd_timer pcspkr sg snd soundcore vmw_vmci i2c_piix4 parport_pc parport xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul e1000 crct10dif_common libata crc32c_intel mptspi scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks floppy [last unloaded: ip_tables]
CPU: 7 PID: 55375 Comm: kworker/u128:2 Tainted: G        W    L ------------   3.10.0-957.27.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff94da2449c100 ti: ffff94da66624000 task.ti: ffff94da66624000
RIP: 0010:[<ffffffffc03c43be>]  [<ffffffffc03c43be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff94da66627888  EFLAGS: 00000246
RAX: ffffb330c0740000 RBX: ffff94da5e235020 RCX: 000000000000001c
RDX: ffff94d9f594b800 RSI: ffff94d9f5d55000 RDI: 000000000000000e
RBP: ffff94da66627898 R08: 0000000000000003 R09: 0000000062fb3fff
R10: ffff94da77e790d8 R11: ffff94da461a2300 R12: ffff94da2433ad00
R13: ffff94da666277fc R14: ffffffff9fd4d6ab R15: ffff94da66627840
FS:  0000000000000000(0000) GS:ffff94da7b5c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f31fe0a3000 CR3: 00000000ba8fa000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc03e671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03b42e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9fed2ee0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9fedc47c>] scsi_request_fn+0x4cc/0x680
[<ffffffff9fd694de>] ? deadline_add_request+0x6e/0x90
[<ffffffff9fd44e59>] __blk_run_queue+0x39/0x50
[<ffffffff9fd48b63>] blk_queue_bio+0x3b3/0x400
[<ffffffff9fd46af7>] generic_make_request+0x147/0x380
[<ffffffff9fd46da0>] submit_bio+0x70/0x150
[<ffffffffc064c471>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc064c5df>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff9fbc47a1>] do_writepages+0x21/0x50
[<ffffffff9fc71020>] __writeback_single_inode+0x40/0x260
[<ffffffff9fc71ab4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff9fc71e1f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9fc72653>] wb_writeback+0x263/0x2f0
[<ffffffff9fc72fdb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9fabaf9f>] process_one_work+0x17f/0x440
[<ffffffff9fabc036>] worker_thread+0x126/0x3c0
[<ffffffff9fabbf10>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff9fac2e81>] kthread+0xd1/0xe0
[<ffffffff9fac2db0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa0176c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fac2db0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 3d 
Complete report #910
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffbd56a175
Build id: 3.10.0-957.1.3.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc00b3622
Build id: 3.10.0-957.1.3.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffbd2d9100
Build id: 3.10.0-957.1.3.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffbd2d0bd0
Build id: 3.10.0-957.1.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffbd2da16c
Build id: 3.10.0-957.1.3.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffbd16786e
Build id: 3.10.0-957.1.3.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffbd143269
Build id: 3.10.0-957.1.3.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffbd146f63
Build id: 3.10.0-957.1.3.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffbd144ef7
Build id: 3.10.0-957.1.3.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffbd1451a0
Build id: 3.10.0-957.1.3.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0334251
Build id: 3.10.0-957.1.3.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc03343bf
Build id: 3.10.0-957.1.3.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffbcfc33f1
Build id: 3.10.0-957.1.3.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffbd06fba0
Build id: 3.10.0-957.1.3.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffffbd070634
Build id: 3.10.0-957.1.3.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffffbd07099f
Build id: 3.10.0-957.1.3.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffffbd0711d3
Build id: 3.10.0-957.1.3.el7.x86_64
0x263
18
bdi_writeback_workfn
vmlinux 0xffffffffbd071b5b
Build id: 3.10.0-957.1.3.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffffbceb9d4f
Build id: 3.10.0-957.1.3.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffbcebade6
Build id: 3.10.0-957.1.3.el7.x86_64
0x126
21
? manage_workers.isra.25
vmlinux 0xffffffffbcebacc0
Build id: 3.10.0-957.1.3.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffbcec1c31
Build id: 3.10.0-957.1.3.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffbcec1b60
Build id: 3.10.0-957.1.3.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffbd574c37
Build id: 3.10.0-957.1.3.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffbcec1b60
Build id: 3.10.0-957.1.3.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u4:0:19800]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc ppdev sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw e1000e ptp pps_core ata_generic pata_acpi vmwgfx drm_kms_helper nfit ahci syscopyarea
libnvdimm sysfillrect sysimgblt fb_sys_fops ttm libahci drm ata_piix libata vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 19800 Comm: kworker/u4:0 Kdump: loaded Not tainted 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8da59ad88000 ti: ffff8da92adfc000 task.ti: ffff8da92adfc000
RIP: 0010:[<ffffffffbd56a175>]  [<ffffffffbd56a175>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8da92adff8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffbd1a1363 RCX: ffff8da9bfcbe060
RDX: ffffba0ec1be0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8da92adff8f0 R08: 0000000000000000 R09: ffff8da5bf97eb00
R10: ffff8da9282f80d8 R11: ffff8da5b5c68000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8da915fc4a80 R15: ffffffffbd2d7e59
FS:  0000000000000000(0000) GS:ffff8da9bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f268819e6f8 CR3: 00000003bdb16000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc00b3622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffbd2d9100>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffbd2d0bd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbd2da16c>] scsi_request_fn+0x4cc/0x680
[<ffffffffbd16786e>] ? deadline_add_request+0x6e/0x90
[<ffffffffbd143269>] __blk_run_queue+0x39/0x50
[<ffffffffbd146f63>] blk_queue_bio+0x3b3/0x400
[<ffffffffbd144ef7>] generic_make_request+0x147/0x380
[<ffffffffbd1451a0>] submit_bio+0x70/0x150
[<ffffffffc0334251>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc03343bf>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffbcfc33f1>] do_writepages+0x21/0x50
[<ffffffffbd06fba0>] __writeback_single_inode+0x40/0x260
[<ffffffffbd070634>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbd07099f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbd0711d3>] wb_writeback+0x263/0x2f0
[<ffffffffbd071b5b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbceb9d4f>] process_one_work+0x17f/0x440
[<ffffffffbcebade6>] worker_thread+0x126/0x3c0
[<ffffffffbcebacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffbcec1c31>] kthread+0xd1/0xe0
[<ffffffffbcec1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd574c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbcec1b60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #1109923
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00783be
Build id: 3.10.0-693.21.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc009a72e
Build id: 3.10.0-693.21.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00432b0
Build id: 3.10.0-693.21.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146d8f0
Build id: 3.10.0-693.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8147678f
Build id: 3.10.0-693.21.1.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812fd1d9
Build id: 3.10.0-693.21.1.el7.x86_64
0x39
7
blk_queue_bio
vmlinux 0xffffffff81300b33
Build id: 3.10.0-693.21.1.el7.x86_64
0x363
8
generic_make_request
vmlinux 0xffffffff812feb7b
Build id: 3.10.0-693.21.1.el7.x86_64
0x10b
9
submit_bio
vmlinux 0xffffffff812fee00
Build id: 3.10.0-693.21.1.el7.x86_64
0x70
10
xfs_submit_ioend.isra.16
xfs 0xffffffffc028d071
Build id: 3.10.0-693.21.1.el7.x86_64
0x61
11
xfs_vm_writepages
xfs 0xffffffffc028d231
Build id: 3.10.0-693.21.1.el7.x86_64
0xd1
12
do_writepages
vmlinux 0xffffffff81193531
Build id: 3.10.0-693.21.1.el7.x86_64
0x21
13
__writeback_single_inode
vmlinux 0xffffffff812326d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
14
writeback_sb_inodes
vmlinux 0xffffffff81233344
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c4
15
__writeback_inodes_wb
vmlinux 0xffffffff812336af
Build id: 3.10.0-693.21.1.el7.x86_64
0x9f
16
bdi_writeback_workfn
vmlinux 0xffffffff81234496
Build id: 3.10.0-693.21.1.el7.x86_64
0x3d6
17
process_one_work
vmlinux 0xffffffff810abe2f
Build id: 3.10.0-693.21.1.el7.x86_64
0x17f
18
rescuer_thread
vmlinux 0xffffffff810ac384
Build id: 3.10.0-693.21.1.el7.x86_64
0x294
19
? process_one_work
vmlinux 0xffffffff810ac0f0
Build id: 3.10.0-693.21.1.el7.x86_64
0x440
20
kthread
vmlinux 0xffffffff810b4031
Build id: 3.10.0-693.21.1.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
22
ret_from_fork
vmlinux 0xffffffff816c0577
Build id: 3.10.0-693.21.1.el7.x86_64
0x77
23
? insert_kthread_work
vmlinux 0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writeback:50]
Modules linked in: nfsv3 nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg ppdev joydev vmw_balloon parport_pc pcspkr nfit libnvdimm parport shpchp i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc fuse ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common
drm crc32c_intel serio_raw ata_piix mptspi libata scsi_transport_spi mptscsih mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 50 Comm: writeback Not tainted 3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff880179d52f70 ti: ffff880179d90000 task.ti: ffff880179d90000
RIP: 0010:[<ffffffffc00783be>]  [<ffffffffc00783be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff880179d938c0  EFLAGS: 00000246
RAX: ffffc90001bc0000 RBX: 0000000000000003 RCX: 0000000000000003
RDX: ffff880035dcf200 RSI: ffff88042b157000 RDI: 000000000000000e
RBP: ffff880179d938d0 R08: 0000000000000003 R09: ffff880035f700d8
R10: ffff880006eaf640 R11: ffff880006eaf640 R12: ffff880429142920
R13: ffff880035f700d8 R14: ffff880006eaf640 R15: ffff880006eaf640
FS:  0000000000000000(0000) GS:ffff88043fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f5020bf1a00 CR3: 0000000297ece000 CR4: 00000000000607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc009a72e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00432b0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8146d8f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8147678f>] scsi_request_fn+0x4df/0x680
[<ffffffff812fd1d9>] __blk_run_queue+0x39/0x50
[<ffffffff81300b33>] blk_queue_bio+0x363/0x3a0
[<ffffffff812feb7b>] generic_make_request+0x10b/0x320
[<ffffffff812fee00>] submit_bio+0x70/0x150
[<ffffffffc028d071>] xfs_submit_ioend.isra.16+0x61/0xe0 [xfs]
[<ffffffffc028d231>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff81193531>] do_writepages+0x21/0x50
[<ffffffff812326d0>] __writeback_single_inode+0x40/0x250
[<ffffffff81233344>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff812336af>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff81234496>] bdi_writeback_workfn+0x3d6/0x460
[<ffffffff810abe2f>] process_one_work+0x17f/0x440
[<ffffffff810ac384>] rescuer_thread+0x294/0x3c0
[<ffffffff810ac0f0>] ? process_one_work+0x440/0x440
[<ffffffff810b4031>] kthread+0xd1/0xe0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 08 
Complete report #709406
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff987817c5
Build id: 3.10.0-1062.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc026c632
Build id: 3.10.0-1062.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff984e4e90
Build id: 3.10.0-1062.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff984dc890
Build id: 3.10.0-1062.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff984e5efc
Build id: 3.10.0-1062.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff983716de
Build id: 3.10.0-1062.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff9834cf59
Build id: 3.10.0-1062.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff98350c63
Build id: 3.10.0-1062.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff9834ebf7
Build id: 3.10.0-1062.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff9834eea0
Build id: 3.10.0-1062.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc08dbcd1
Build id: 3.10.0-1062.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc08dc00f
Build id: 3.10.0-1062.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff981c8d31
Build id: 3.10.0-1062.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff982776c0
Build id: 3.10.0-1062.el7.x86_64
0x40
15
? __schedule
vmlinux 0xffffffff9877ec28
Build id: 3.10.0-1062.el7.x86_64
0x448
16
writeback_sb_inodes
vmlinux 0xffffffff98278254
Build id: 3.10.0-1062.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff9827855f
Build id: 3.10.0-1062.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff98278a43
Build id: 3.10.0-1062.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffff9826490c
Build id: 3.10.0-1062.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff9827963b
Build id: 3.10.0-1062.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff980bd0ff
Build id: 3.10.0-1062.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff980be216
Build id: 3.10.0-1062.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff980be0f0
Build id: 3.10.0-1062.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff980c50d1
Build id: 3.10.0-1062.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff980c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff9878bd37
Build id: 3.10.0-1062.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffff980c5000
Build id: 3.10.0-1062.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 31s! [kworker/u8:0:9620]
Modules linked in: vfat fat isofs ext4 mbcache jbd2 nfsv3 nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_vs nf_conntrack iscsi_target_mod target_core_mod vmw_vsock_vmci_transport vsock sb_edac coretemp ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c zfs(POE) zunicode(POE) zlua(POE) sr_mod cdrom ata_generic pata_acpi
sd_mod crc_t10dif crct10dif_generic vmwgfx ata_piix crct10dif_pclmul crct10dif_common drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel serio_raw drm vmxnet3 libata vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod zcommon(POE) znvpair(POE) zavl(POE) icp(POE) spl(OE)
CPU: 1 PID: 9620 Comm: kworker/u8:0 Kdump: loaded Tainted: P           OEL ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9f79c4d5a0e0 ti: ffff9f79ae370000 task.ti: ffff9f79ae370000
RIP: 0010:[<ffffffff987817c5>]  [<ffffffff987817c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9f79ae3738f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff983ab803 RCX: ffff9f7a791a8060
RDX: ffffb718407f0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9f79ae3738f0 R08: 0000000000000000 R09: ffff9f79eccc2b00
R10: ffff9f7a7401e8d8 R11: ffff9f7974541800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9f7a791dbe00 R15: ffffffff984e3be9
FS:  0000000000000000(0000) GS:ffff9f7a7fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005648a7fe1fa8 CR3: 00000000ab3bc000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc026c632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff984e4e90>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff984dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff984e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff983716de>] ? deadline_add_request+0x6e/0x90
[<ffffffff9834cf59>] __blk_run_queue+0x39/0x50
[<ffffffff98350c63>] blk_queue_bio+0x3b3/0x400
[<ffffffff9834ebf7>] generic_make_request+0x147/0x380
[<ffffffff9834eea0>] submit_bio+0x70/0x150
[<ffffffffc08dbcd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc08dc00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff981c8d31>] do_writepages+0x21/0x50
[<ffffffff982776c0>] __writeback_single_inode+0x40/0x260
[<ffffffff9877ec28>] ? __schedule+0x448/0x9c0
[<ffffffff98278254>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9827855f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff98278a43>] wb_writeback+0x263/0x2f0
[<ffffffff9826490c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff9827963b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff980bd0ff>] process_one_work+0x17f/0x440
[<ffffffff980be216>] worker_thread+0x126/0x3c0
[<ffffffff980be0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff980c50d1>] kthread+0xd1/0xe0
[<ffffffff980c5000>] ? insert_kthread_work+0x40/0x40
[<ffffffff9878bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff980c5000>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #563254
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc034e3be
Build id: 3.10.0-1160.76.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc037071e
Build id: 3.10.0-1160.76.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03412e0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa06e4690
Build id: 3.10.0-1160.76.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa06eddbc
Build id: 3.10.0-1160.76.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffa05789ee
Build id: 3.10.0-1160.76.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa0553d19
Build id: 3.10.0-1160.76.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa0557ae3
Build id: 3.10.0-1160.76.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa05559c7
Build id: 3.10.0-1160.76.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffa0555c70
Build id: 3.10.0-1160.76.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0606f31
Build id: 3.10.0-1160.76.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc060726f
Build id: 3.10.0-1160.76.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffa03cb9f1
Build id: 3.10.0-1160.76.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffa047df50
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
15
? update_curr
vmlinux 0xffffffffa02e2acc
Build id: 3.10.0-1160.76.1.el7.x86_64
0x14c
16
writeback_sb_inodes
vmlinux 0xffffffffa047ead4
Build id: 3.10.0-1160.76.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffa047eddf
Build id: 3.10.0-1160.76.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffa047f2b3
Build id: 3.10.0-1160.76.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffa046ac2c
Build id: 3.10.0-1160.76.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffa047feab
Build id: 3.10.0-1160.76.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffa02bdfbf
Build id: 3.10.0-1160.76.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffa02bf0d6
Build id: 3.10.0-1160.76.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffa02befb0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffa02c5f91
Build id: 3.10.0-1160.76.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffa02c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffa0999ddd
Build id: 3.10.0-1160.76.1.el7.x86_64
0x7
27
? insert_kthread_work
vmlinux 0xffffffffa02c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 27s! [kworker/u128:3:1776]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vmw_vsock_vmci_transport vsock vboxdrv(OE) iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c
sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix nfit serio_raw libata mptspi libnvdimm scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 1776 Comm: kworker/u128:3 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9b4588dd9080 ti: ffff9b3ff722c000 task.ti: ffff9b3ff722c000
RIP: 0010:[<ffffffffc034e3be>]  [<ffffffffc034e3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9b3ff722f888  EFLAGS: 00000246
RAX: ffffb4eb46440000 RBX: ffff9b40431d51a0 RCX: 0000000000000037
RDX: ffff9b4588f8c800 RSI: ffff9b4589259000 RDI: 000000000000000e
RBP: ffff9b3ff722f898 R08: 0000000000000002 R09: 00000009afd94fff
R10: ffff9b39b5dd0400 R11: ffff9b4478c26680 R12: ffff9b3a0b811f80
R13: ffff9b3ff722f7fc R14: ffffffffa055c6fb R15: ffff9b3ff722f840
FS:  0000000000000000(0000) GS:ffff9b458e700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3f0014c000 CR3: 0000000ba0c94000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc037071e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03412e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa06e4690>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa06eddbc>] scsi_request_fn+0x4ac/0x680
[<ffffffffa05789ee>] ? deadline_add_request+0x6e/0x90
[<ffffffffa0553d19>] __blk_run_queue+0x39/0x50
[<ffffffffa0557ae3>] blk_queue_bio+0x3b3/0x400
[<ffffffffa05559c7>] generic_make_request+0x147/0x380
[<ffffffffa0555c70>] submit_bio+0x70/0x150
[<ffffffffc0606f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc060726f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffa03cb9f1>] do_writepages+0x21/0x50
[<ffffffffa047df50>] __writeback_single_inode+0x40/0x260
[<ffffffffa02e2acc>] ? update_curr+0x14c/0x1e0
[<ffffffffa047ead4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa047eddf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa047f2b3>] wb_writeback+0x263/0x2f0
[<ffffffffa046ac2c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa047feab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa02bdfbf>] process_one_work+0x17f/0x440
[<ffffffffa02bf0d6>] worker_thread+0x126/0x3c0
[<ffffffffa02befb0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa02c5f91>] kthread+0xd1/0xe0
[<ffffffffa02c5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa0999ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa02c5ec0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 c3 35 
Complete report #524916
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc003e3be
Build id: 3.10.0-693.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc0072729
Build id: 3.10.0-693.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffc01062b0
Build id: 3.10.0-693.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64
0x33
7
blk_queue_bio
vmlinux 0xffffffff812faad3
Build id: 3.10.0-693.el7.x86_64
0x363
8
generic_make_request
vmlinux 0xffffffff812f8b45
Build id: 3.10.0-693.el7.x86_64
0x105
9
submit_bio
vmlinux 0xffffffff812f8dc0
Build id: 3.10.0-693.el7.x86_64
0x70
10
xfs_submit_ioend.isra.16
xfs 0xffffffffc02745e1
Build id: 3.10.0-693.el7.x86_64
0x61
11
xfs_vm_writepages
xfs 0xffffffffc02747a1
Build id: 3.10.0-693.el7.x86_64
0xd1
12
do_writepages
vmlinux 0xffffffff8118effe
Build id: 3.10.0-693.el7.x86_64
0x1e
13
__writeback_single_inode
vmlinux 0xffffffff8122d7d0
Build id: 3.10.0-693.el7.x86_64
0x40
14
writeback_sb_inodes
vmlinux 0xffffffff8122e414
Build id: 3.10.0-693.el7.x86_64
0x1c4
15
__writeback_inodes_wb
vmlinux 0xffffffff8122e77f
Build id: 3.10.0-693.el7.x86_64
0x9f
16
wb_writeback
vmlinux 0xffffffff8122efb3
Build id: 3.10.0-693.el7.x86_64
0x263
17
? get_nr_inodes
vmlinux 0xffffffff8121bc3c
Build id: 3.10.0-693.el7.x86_64
0x4c
18
bdi_writeback_workfn
vmlinux 0xffffffff8122f45b
Build id: 3.10.0-693.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64
0x17a
20
worker_thread
vmlinux 0xffffffff810a94e6
Build id: 3.10.0-693.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
0xcf
23
? do_exit
vmlinux 0xffffffff8108ddeb
Build id: 3.10.0-693.el7.x86_64
0x6bb
24
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
25
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
0x58
26
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 56s! [kworker/u256:2:92833]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac edac_core iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec ac97_bus snd_seq aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd snd_seq_device snd_pcm vmw_balloon joydev pcspkr snd_timer snd soundcore nfit sg libnvdimm shpchp vmw_vmci
i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata mptspi serio_raw scsi_transport_spi e1000 mptscsih i2c_core mptbase dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 92833 Comm: kworker/u256:2 Tainted: G        W    L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8800adae4f10 ti: ffff880135fd8000 task.ti: ffff880135fd8000
RIP: 0010:[<ffffffffc003e3be>]  [<ffffffffc003e3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880135fdb890  EFLAGS: 00000246
RAX: ffffc90000980000 RBX: ffff880076e5b100 RCX: 000000000000001e
RDX: ffff88003668be00 RSI: ffff88013339d000 RDI: 000000000000000e
RBP: ffff880135fdb8a0 R08: 0000000000000002 R09: ffff8801332ee0d8
R10: ffff8800ae0bd180 R11: ffff8800ae0bd180 R12: 0000000100000001
R13: ffff880025ee5500 R14: ffff880135fdb804 R15: ffffffff812ff13b
FS:  0000000000000000(0000) GS:ffff880139600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f7d8ece2ed8 CR3: 00000000ae06e000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88013339dd30 000000000000003c ffff880135fdb948 ffffffffc0072729
ffff88013339d008 0400000035fdb908 000000000000001e ffff8800ae0bd180
0000000000000078 ffff88013339d188 000000600000001e ffff880035f03370
Call Trace:
[<ffffffffc0072729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc01062b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812faad3>] blk_queue_bio+0x363/0x3a0
[<ffffffff812f8b45>] generic_make_request+0x105/0x310
[<ffffffff812f8dc0>] submit_bio+0x70/0x150
[<ffffffffc02745e1>] xfs_submit_ioend.isra.16+0x61/0xe0 [xfs]
[<ffffffffc02747a1>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff8118effe>] do_writepages+0x1e/0x40
[<ffffffff8122d7d0>] __writeback_single_inode+0x40/0x220
[<ffffffff8122e414>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8122e77f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8122efb3>] wb_writeback+0x263/0x2f0
[<ffffffff8121bc3c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8122f45b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a94e6>] worker_thread+0x126/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 c3 04 
Complete report #394055
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8a11b2c5
Build id: 3.10.0-862.11.6.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc027e622
Build id: 3.10.0-862.11.6.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff89ea8500
Build id: 3.10.0-862.11.6.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff89ea0010
Build id: 3.10.0-862.11.6.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff89ea957c
Build id: 3.10.0-862.11.6.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff89d4122e
Build id: 3.10.0-862.11.6.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff89d1d4e9
Build id: 3.10.0-862.11.6.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff89d21173
Build id: 3.10.0-862.11.6.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff89d1f0cb
Build id: 3.10.0-862.11.6.el7.x86_64
0x10b
10
submit_bio
vmlinux 0xffffffff89d1f350
Build id: 3.10.0-862.11.6.el7.x86_64
0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc04cf4b1
Build id: 3.10.0-862.11.6.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc04cf671
Build id: 3.10.0-862.11.6.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffff89ba3ac1
Build id: 3.10.0-862.11.6.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff89c4cf00
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff89c4d994
Build id: 3.10.0-862.11.6.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff89c4dcff
Build id: 3.10.0-862.11.6.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff89c4e533
Build id: 3.10.0-862.11.6.el7.x86_64
0x263
18
? get_nr_inodes
vmlinux 0xffffffff89c3aa4c
Build id: 3.10.0-862.11.6.el7.x86_64
0x4c
19
bdi_writeback_workfn
vmlinux 0xffffffff89c4eebb
Build id: 3.10.0-862.11.6.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffff89ab613f
Build id: 3.10.0-862.11.6.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff89ab71d6
Build id: 3.10.0-862.11.6.el7.x86_64
0x126
22
? manage_workers.isra.24
vmlinux 0xffffffff89ab70b0
Build id: 3.10.0-862.11.6.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff89abdf21
Build id: 3.10.0-862.11.6.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff89abde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a1255dd
Build id: 3.10.0-862.11.6.el7.x86_64
0x7
26
? insert_kthread_work
vmlinux 0xffffffff89abde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u21:2:14069]
Modules linked in: veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_1_9_8557(POE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon ppdev
sg pcspkr joydev vmw_vmci parport_pc parport shpchp i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 14069 Comm: kworker/u21:2 Kdump: loaded Tainted: P           OEL ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:6)
task: ffff8cce43fc2f70 ti: ffff8ccd80e48000 task.ti: ffff8ccd80e48000
RIP: 0010:[<ffffffff8a11b2c5>]  [<ffffffff8a11b2c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff8ccd80e4b8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff89d795b3 RCX: ffff8cdbcd4b0060
RDX: ffffacef4ce20000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8ccd80e4b8f0 R08: 0000000000000000 R09: ffff8ccb116e1000
R10: ffff8ccf4b7750d8 R11: ffff8ccf4b639000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8cc98e26f080 R15: ffffffff89ea7259
FS:  0000000000000000(0000) GS:ffff8ccf4d800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000147891200 CR3: 0000000ec540e000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc027e622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff89ea8500>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff89ea0010>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff89ea957c>] scsi_request_fn+0x4cc/0x680
[<ffffffff89d4122e>] ? deadline_add_request+0x6e/0x90
[<ffffffff89d1d4e9>] __blk_run_queue+0x39/0x50
[<ffffffff89d21173>] blk_queue_bio+0x3b3/0x400
[<ffffffff89d1f0cb>] generic_make_request+0x10b/0x320
[<ffffffff89d1f350>] submit_bio+0x70/0x150
[<ffffffffc04cf4b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc04cf671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff89ba3ac1>] do_writepages+0x21/0x50
[<ffffffff89c4cf00>] __writeback_single_inode+0x40/0x260
[<ffffffff89c4d994>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff89c4dcff>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff89c4e533>] wb_writeback+0x263/0x2f0
[<ffffffff89c3aa4c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff89c4eebb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff89ab613f>] process_one_work+0x17f/0x440
[<ffffffff89ab71d6>] worker_thread+0x126/0x3c0
[<ffffffff89ab70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff89abdf21>] kthread+0xd1/0xe0
[<ffffffff89abde50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a1255dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff89abde50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #389919
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04473be
Build id: 3.10.0-1160.59.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc049f71e
Build id: 3.10.0-1160.59.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04b42e0
Build id: 3.10.0-1160.59.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb20e4870
Build id: 3.10.0-1160.59.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb20edf9c
Build id: 3.10.0-1160.59.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb1f78bfe
Build id: 3.10.0-1160.59.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffb1f53f29
Build id: 3.10.0-1160.59.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffb1f57cf3
Build id: 3.10.0-1160.59.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffb1f55bd7
Build id: 3.10.0-1160.59.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffb1f55e80
Build id: 3.10.0-1160.59.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc067ef31
Build id: 3.10.0-1160.59.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc067f26f
Build id: 3.10.0-1160.59.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffb1dcb9b1
Build id: 3.10.0-1160.59.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffb1e7df10
Build id: 3.10.0-1160.59.1.el7.x86_64
0x40
15
? copy_process
vmlinux 0xffffffffb1c99573
Build id: 3.10.0-1160.59.1.el7.x86_64
0xf23
16
writeback_sb_inodes
vmlinux 0xffffffffb1e7ea94
Build id: 3.10.0-1160.59.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffb1e7ed9f
Build id: 3.10.0-1160.59.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffb1e7f273
Build id: 3.10.0-1160.59.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffb1e6ae6c
Build id: 3.10.0-1160.59.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffb1e7fe6b
Build id: 3.10.0-1160.59.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffb1cbde8f
Build id: 3.10.0-1160.59.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffb1cbefa6
Build id: 3.10.0-1160.59.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffb1cbee80
Build id: 3.10.0-1160.59.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffb1cc5e61
Build id: 3.10.0-1160.59.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffb1cc5d90
Build id: 3.10.0-1160.59.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffb2399df7
Build id: 3.10.0-1160.59.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffffb1cc5d90
Build id: 3.10.0-1160.59.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 38s! [kworker/u8:0:6748]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd joydev pcspkr i2c_piix4 sg vmw_vmci parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea crc32c_intel sysfillrect sysimgblt serio_raw fb_sys_fops ttm mptspi scsi_transport_spi ata_piix mptscsih drm mptbase
nfit vmxnet3 libata libnvdimm drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 6748 Comm: kworker/u8:0 Kdump: loaded Not tainted 3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff88f8b820e300 ti: ffff88f8b30dc000 task.ti: ffff88f8b30dc000
RIP: 0010:[<ffffffffc04473be>]  [<ffffffffc04473be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff88f8b30df888  EFLAGS: 00000246
RAX: ffffb29d80ec0000 RBX: ffff88f8b86a86b0 RCX: 000000000000000f
RDX: ffff88fa36c0ec00 RSI: ffff88fa3242e000 RDI: 000000000000000e
RBP: ffff88f8b30df898 R08: 0000000000000003 R09: 0000000232774fff
R10: ffff88fa31c4c800 R11: ffff88fa3280d6c0 R12: ffff88fa34218300
R13: ffff88f8b30df7fc R14: ffffffffb1f5c90b R15: ffff88f8b30df840
FS:  0000000000000000(0000) GS:ffff88fa3fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efc427179e5 CR3: 00000002313d8000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc049f71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04b42e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb20e4870>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb20edf9c>] scsi_request_fn+0x4ac/0x680
[<ffffffffb1f78bfe>] ? deadline_add_request+0x6e/0x90
[<ffffffffb1f53f29>] __blk_run_queue+0x39/0x50
[<ffffffffb1f57cf3>] blk_queue_bio+0x3b3/0x400
[<ffffffffb1f55bd7>] generic_make_request+0x147/0x380
[<ffffffffb1f55e80>] submit_bio+0x70/0x150
[<ffffffffc067ef31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc067f26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffb1dcb9b1>] do_writepages+0x21/0x50
[<ffffffffb1e7df10>] __writeback_single_inode+0x40/0x260
[<ffffffffb1c99573>] ? copy_process+0xf23/0x1a80
[<ffffffffb1e7ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb1e7ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb1e7f273>] wb_writeback+0x263/0x2f0
[<ffffffffb1e6ae6c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffb1e7fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb1cbde8f>] process_one_work+0x17f/0x440
[<ffffffffb1cbefa6>] worker_thread+0x126/0x3c0
[<ffffffffb1cbee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb1cc5e61>] kthread+0xd1/0xe0
[<ffffffffb1cc5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffb2399df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb1cc5d90>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 45 
Complete report #327489
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02313be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02ef71e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc01da2e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa5e9b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa5ea45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffa5d3c84e
Build id: 3.10.0-862.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa5d18ad9
Build id: 3.10.0-862.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa5d1c763
Build id: 3.10.0-862.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa5d1a6bb
Build id: 3.10.0-862.el7.x86_64
0x10b
10
submit_bio
vmlinux 0xffffffffa5d1a940
Build id: 3.10.0-862.el7.x86_64
0x70
11
xfs_submit_ioend.isra.14
xfs 0xffffffffc04224b1
Build id: 3.10.0-862.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc0422671
Build id: 3.10.0-862.el7.x86_64
0xd1
13
do_writepages
vmlinux 0xffffffffa5ba0e21
Build id: 3.10.0-862.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffa5c48af0
Build id: 3.10.0-862.el7.x86_64
0x40
15
? update_curr
vmlinux 0xffffffffa5ad816c
Build id: 3.10.0-862.el7.x86_64
0x14c
16
writeback_sb_inodes
vmlinux 0xffffffffa5c49584
Build id: 3.10.0-862.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffa5c498ef
Build id: 3.10.0-862.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffa5c4a123
Build id: 3.10.0-862.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffa5c3665c
Build id: 3.10.0-862.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffa5c4aaab
Build id: 3.10.0-862.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffa5ab2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffa5ab3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
23
? manage_workers.isra.24
vmlinux 0xffffffffa5ab39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffa5abae31
Build id: 3.10.0-862.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffa5abad60
Build id: 3.10.0-862.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffa611f61d
Build id: 3.10.0-862.el7.x86_64
0x7
27
? insert_kthread_work
vmlinux 0xffffffffa5abad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#12 stuck for 23s! [kworker/u256:0:82408]
Modules linked in: xt_nat veth vxlan ip6_udp_tunnel udp_tunnel xt_mark nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) vsock_diag sctp_diag sctp tcp_diag udp_diag inet_diag unix_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus iosf_mbi snd_seq crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel lrw gf128mul snd_timer glue_helper snd ablk_helper pcspkr cryptd joydev ppdev soundcore vmw_balloon nfit sg parport_pc
libnvdimm parport vmw_vmci i2c_piix4 shpchp ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common e1000 mptbase crc32c_intel i2c_core serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 12 PID: 82408 Comm: kworker/u256:0 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff89297417af70 ti: ffff89386ea60000 task.ti: ffff89386ea60000
RIP: 0010:[<ffffffffc02313be>]  [<ffffffffc02313be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff89386ea63888  EFLAGS: 00000246
RAX: ffff967206300000 RBX: ffff893867e287d0 RCX: 000000000000001d
RDX: ffff892975e7dc00 RSI: ffff8939391b0000 RDI: 000000000000000e
RBP: ffff89386ea63898 R08: 0000000000000003 R09: 0000000fe665efff
R10: ffff8939366220d8 R11: ffff8939367f9500 R12: ffff89387830cc00
R13: ffff89386ea637fc R14: ffffffffa5d212cb R15: ffff89386ea63840
FS:  0000000000000000(0000) GS:ffff89393d900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5715774a08 CR3: 0000000fa5310000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc02ef71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01da2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa5e9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa5ea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffffa5d3c84e>] ? deadline_add_request+0x6e/0x90
[<ffffffffa5d18ad9>] __blk_run_queue+0x39/0x50
[<ffffffffa5d1c763>] blk_queue_bio+0x3b3/0x400
[<ffffffffa5d1a6bb>] generic_make_request+0x10b/0x320
[<ffffffffa5d1a940>] submit_bio+0x70/0x150
[<ffffffffc04224b1>] xfs_submit_ioend.isra.14+0x61/0xe0 [xfs]
[<ffffffffc0422671>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffffa5ba0e21>] do_writepages+0x21/0x50
[<ffffffffa5c48af0>] __writeback_single_inode+0x40/0x260
[<ffffffffa5ad816c>] ? update_curr+0x14c/0x1e0
[<ffffffffa5c49584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa5c498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa5c4a123>] wb_writeback+0x263/0x2f0
[<ffffffffa5c3665c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa5c4aaab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa5ab2dff>] process_one_work+0x17f/0x440
[<ffffffffa5ab3ac6>] worker_thread+0x126/0x3c0
[<ffffffffa5ab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffa5abae31>] kthread+0xd1/0xe0
[<ffffffffa5abad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa611f61d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa5abad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 f3 23 
Complete report #308940
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffad383815
Build id: 3.10.0-1062.18.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc02fa632
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffad0e72d0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffad0decd0
Build id: 3.10.0-1062.18.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffad0e833c
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffacf73aae
Build id: 3.10.0-1062.18.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffacf4f289
Build id: 3.10.0-1062.18.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffacf52fa3
Build id: 3.10.0-1062.18.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffacf50f37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffacf511e0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x70
11
? cpumask_next_and
vmlinux 0xffffffffacf81845
Build id: 3.10.0-1062.18.1.el7.x86_64
0x35
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc056acd1
Build id: 3.10.0-1062.18.1.el7.x86_64
0x61
13
xfs_vm_writepages
xfs 0xffffffffc056b00f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x7f
14
do_writepages
vmlinux 0xffffffffacdca6a1
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
15
__writeback_single_inode
vmlinux 0xfffffffface79b00
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
16
writeback_sb_inodes
vmlinux 0xfffffffface7a694
Build id: 3.10.0-1062.18.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xfffffffface7a99f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xfffffffface7ae83
Build id: 3.10.0-1062.18.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xfffffffface66d2c
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xfffffffface7ba7b
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffaccbe34f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffaccbf466
Build id: 3.10.0-1062.18.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffaccbf340
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffaccc6321
Build id: 3.10.0-1062.18.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffaccc6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffad38dd37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffffaccc6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [kworker/u34:1:12074]
Modules linked in: veth rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache xt_statistic xt_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 ip_set iptable_nat nf_nat_ipv4 nfnetlink nf_nat xt_conntrack nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ipt_REJECT ip_vs_rr nf_reject_ipv4 ip_vs nf_conntrack tun ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio ppdev vmw_balloon sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc vmw_vmci i2c_piix4 parport auth_rpcgss
sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod vmwgfx crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci ata_piix nfit crct10dif_pclmul crct10dif_common libata crc32c_intel libnvdimm serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc
CPU: 14 PID: 12074 Comm: kworker/u34:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:136)
task: ffff8cff1fb48000 ti: ffff8cfe97240000 task.ti: ffff8cfe97240000
RIP: 0010:[<ffffffffad383815>]  [<ffffffffad383815>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8cfe972438f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffacfadbd3 RCX: ffff8cfb02941060
RDX: ffff9e71835d0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8cfe972438f0 R08: 0000000000000000 R09: ffff8d0010700100
R10: ffff8d012ca7f8d8 R11: ffff8d012ca7e800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8d00fb798000 R15: ffffffffad0e6029
FS:  0000000000000000(0000) GS:ffff8d013fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000730443894 CR3: 000000081b730000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc02fa632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffad0e72d0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffad0decd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffad0e833c>] scsi_request_fn+0x4cc/0x680
[<ffffffffacf73aae>] ? deadline_add_request+0x6e/0x90
[<ffffffffacf4f289>] __blk_run_queue+0x39/0x50
[<ffffffffacf52fa3>] blk_queue_bio+0x3b3/0x400
[<ffffffffacf50f37>] generic_make_request+0x147/0x380
[<ffffffffacf511e0>] submit_bio+0x70/0x150
[<ffffffffacf81845>] ? cpumask_next_and+0x35/0x50
[<ffffffffc056acd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc056b00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffacdca6a1>] do_writepages+0x21/0x50
[<fffffffface79b00>] __writeback_single_inode+0x40/0x260
[<fffffffface7a694>] writeback_sb_inodes+0x1c4/0x430
[<fffffffface7a99f>] __writeback_inodes_wb+0x9f/0xd0
[<fffffffface7ae83>] wb_writeback+0x263/0x2f0
[<fffffffface66d2c>] ? get_nr_inodes+0x4c/0x70
[<fffffffface7ba7b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffaccbe34f>] process_one_work+0x17f/0x440
[<ffffffffaccbf466>] worker_thread+0x126/0x3c0
[<ffffffffaccbf340>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffaccc6321>] kthread+0xd1/0xe0
[<ffffffffaccc6250>] ? insert_kthread_work+0x40/0x40
[<ffffffffad38dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaccc6250>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #270560
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02d43be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc032e71e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03272e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb52e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb52ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffb51771ae
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffb51527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffb51564b3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffb5154437
Build id: 3.10.0-1127.19.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffb51546e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0545fb1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc05462ef
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffb4fcb261
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffb507cc40
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
15
? __schedule
vmlinux 0xffffffffb5585942
Build id: 3.10.0-1127.19.1.el7.x86_64
0x402
16
writeback_sb_inodes
vmlinux 0xffffffffb507d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffb507dacf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffb507dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffb5069c5c
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffb507eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffb4ebe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffb4ebf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffb4ebf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffb4ec6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffb4ec65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffb5592d1d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7
27
? insert_kthread_work
vmlinux 0xffffffffb4ec65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 1397s! [kworker/u256:1:28679]
Modules linked in: tcp_lp rfcomm fuse xt_nat xt_mark xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmhgfs(OE) vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec btusb ppdev btrtl ac97_bus btbcm btintel snd_seq snd_seq_device bluetooth vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_pcm vmw_vmci sg snd_timer rfkill joydev pcspkr snd i2c_piix4 soundcore parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ata_piix fb_sys_fops crct10dif_pclmul crct10dif_common ttm libata crc32c_intel mptspi nfit drm scsi_transport_spi mptscsih libnvdimm mptbase serio_raw e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 28679 Comm: kworker/u256:1 Kdump: loaded Tainted: G           OE  ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9eced02341c0 ti: ffff9ecc03028000 task.ti: ffff9ecc03028000
RIP: 0010:[<ffffffffc02d43be>]  [<ffffffffc02d43be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9ecc0302b888  EFLAGS: 00000246
RAX: ffffc3bb01d00000 RBX: ffff9ecf0b010fa0 RCX: 000000000000000d
RDX: ffff9ecf2c073400 RSI: ffff9ecf2c85d000 RDI: 000000000000000e
RBP: ffff9ecc0302b898 R08: 0000000000000002 R09: 00000002fe886fff
R10: ffff9ecf2c299800 R11: ffff9ecc35d841c0 R12: ffff9ecd5669b480
R13: ffff9ecc0302b7fc R14: ffffffffb515b0cb R15: ffff9ecc0302b840
FS:  0000000000000000(0000) GS:ffff9ecf31700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006dee1c CR3: 0000000285810000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc032e71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03272e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb52e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb52ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffffb51771ae>] ? deadline_add_request+0x6e/0x90
[<ffffffffb51527a9>] __blk_run_queue+0x39/0x50
[<ffffffffb51564b3>] blk_queue_bio+0x3b3/0x400
[<ffffffffb5154437>] generic_make_request+0x147/0x380
[<ffffffffb51546e0>] submit_bio+0x70/0x150
[<ffffffffc0545fb1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc05462ef>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffb4fcb261>] do_writepages+0x21/0x50
[<ffffffffb507cc40>] __writeback_single_inode+0x40/0x260
[<ffffffffb5585942>] ? __schedule+0x402/0x840
[<ffffffffb507d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb507dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb507dfa3>] wb_writeback+0x263/0x2f0
[<ffffffffb5069c5c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffb507eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb4ebe6bf>] process_one_work+0x17f/0x440
[<ffffffffb4ebf7d6>] worker_thread+0x126/0x3c0
[<ffffffffb4ebf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb4ec6691>] kthread+0xd1/0xe0
[<ffffffffb4ec65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb5592d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffb4ec65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 2e 
Complete report #221570
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffad98b565
Build id: 3.10.0-1160.25.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0258632
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffad6ec6b0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffad6e3fa0
Build id: 3.10.0-1160.25.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffad6ed6cc
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffad57853e
Build id: 3.10.0-1160.25.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffad553939
Build id: 3.10.0-1160.25.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffad557743
Build id: 3.10.0-1160.25.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffad5555c7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffad555870
Build id: 3.10.0-1160.25.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0410f31
Build id: 3.10.0-1160.25.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc041126f
Build id: 3.10.0-1160.25.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffad3cb001
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffad47d9a0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
15
? __schedule
vmlinux 0xffffffffad988a77
Build id: 3.10.0-1160.25.1.el7.x86_64
0x3c7
16
writeback_sb_inodes
vmlinux 0xffffffffad47e524
Build id: 3.10.0-1160.25.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffad47e82f
Build id: 3.10.0-1160.25.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffad47ed03
Build id: 3.10.0-1160.25.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffad46a92c
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffad47f8fb
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffad2bddcf
Build id: 3.10.0-1160.25.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffad2beee6
Build id: 3.10.0-1160.25.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffad2bedc0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffad2c5da1
Build id: 3.10.0-1160.25.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffad2c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffad995df7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffffad2c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u4:1:17069]
Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm dns_resolver ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper pcspkr joydev cryptd vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic
pata_acpi sd_mod crc_t10dif vmwgfx crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix vmxnet3 vmw_pvscsi libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 17069 Comm: kworker/u4:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff95a4b1a5a100 ti: ffff95a39f048000 task.ti: ffff95a39f048000
RIP: 0010:[<ffffffffad98b565>]  [<ffffffffad98b565>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff95a39f04b8f0  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffad5b28d3 RCX: ffff95a4b479e060
RDX: ffffa8da007f0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff95a39f04b8f0 R08: 0000000000000000 R09: ffff95a3c97aa900
R10: ffff95a3b6756000 R11: ffff95a4b00ab000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff95a4b1a96d80 R15: ffffffffad6eb369
FS:  0000000000000000(0000) GS:ffff95a4bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055c0252e3340 CR3: 00000000b286c000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0258632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffad6ec6b0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffad6e3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffad6ed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffffad57853e>] ? deadline_add_request+0x6e/0x90
[<ffffffffad553939>] __blk_run_queue+0x39/0x50
[<ffffffffad557743>] blk_queue_bio+0x3b3/0x400
[<ffffffffad5555c7>] generic_make_request+0x147/0x380
[<ffffffffad555870>] submit_bio+0x70/0x150
[<ffffffffc0410f31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc041126f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffad3cb001>] do_writepages+0x21/0x50
[<ffffffffad47d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffffad988a77>] ? __schedule+0x3c7/0x870
[<ffffffffad47e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffad47e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffad47ed03>] wb_writeback+0x263/0x2f0
[<ffffffffad46a92c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffad47f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffad2bddcf>] process_one_work+0x17f/0x440
[<ffffffffad2beee6>] worker_thread+0x126/0x3c0
[<ffffffffad2bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffad2c5da1>] kthread+0xd1/0xe0
[<ffffffffad2c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffad995df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffad2c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #164332
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01483be
Build id: 3.10.0-1127.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc016a71e
Build id: 3.10.0-1127.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc01632e0
Build id: 3.10.0-1127.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff988e2850
Build id: 3.10.0-1127.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff988ebefc
Build id: 3.10.0-1127.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffff9877709e
Build id: 3.10.0-1127.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff987526f9
Build id: 3.10.0-1127.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff98756403
Build id: 3.10.0-1127.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff98754387
Build id: 3.10.0-1127.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff98754630
Build id: 3.10.0-1127.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc03a0071
Build id: 3.10.0-1127.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc03a03af
Build id: 3.10.0-1127.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff985cb1b1
Build id: 3.10.0-1127.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff9867cbd0
Build id: 3.10.0-1127.el7.x86_64
0x40
15
? __switch_to
vmlinux 0xffffffff9842b59e
Build id: 3.10.0-1127.el7.x86_64
0xce
16
writeback_sb_inodes
vmlinux 0xffffffff9867d764
Build id: 3.10.0-1127.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff9867da6f
Build id: 3.10.0-1127.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff9867df53
Build id: 3.10.0-1127.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffff98669bac
Build id: 3.10.0-1127.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff9867eb4b
Build id: 3.10.0-1127.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff984be6bf
Build id: 3.10.0-1127.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff984bf7d6
Build id: 3.10.0-1127.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffff984bf6b0
Build id: 3.10.0-1127.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff984c6691
Build id: 3.10.0-1127.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff984c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff98b92d37
Build id: 3.10.0-1127.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffff984c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 56s! [kworker/u256:1:50343]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter devlink ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_ens1371 snd_rawmidi vmw_balloon snd_ac97_codec ac97_bus lrw gf128mul snd_seq glue_helper ablk_helper cryptd snd_seq_device joydev pcspkr snd_pcm snd_timer snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi crct10dif_pclmul
crct10dif_common vmwgfx crc32c_intel drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt serio_raw fb_sys_fops ttm e1000 libata mptspi drm scsi_transport_spi mptscsih mptbase nfit drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 50343 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff987507ba1070 ti: ffff9874fd08c000 task.ti: ffff9874fd08c000
RIP: 0010:[<ffffffffc01483be>]  [<ffffffffc01483be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9874fd08f888  EFLAGS: 00000246
RAX: ffffa8c4c1cc0000 RBX: ffff9874f12bd020 RCX: 0000000000000006
RDX: ffff98752f509c00 RSI: ffff98752f48e000 RDI: 000000000000000e
RBP: ffff9874fd08f898 R08: 0000000000000002 R09: 000000022ee18fff
R10: ffff9872f5f17400 R11: ffff987362667800 R12: ffff9874fb4c9c80
R13: ffff9874fd08f7fc R14: ffffffff9875b01b R15: ffff9874fd08f840
FS:  0000000000000000(0000) GS:ffff987534640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f69b1c6bc70 CR3: 000000023b614000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc016a71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01632e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff988e2850>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff988ebefc>] scsi_request_fn+0x4ac/0x660
[<ffffffff9877709e>] ? deadline_add_request+0x6e/0x90
[<ffffffff987526f9>] __blk_run_queue+0x39/0x50
[<ffffffff98756403>] blk_queue_bio+0x3b3/0x400
[<ffffffff98754387>] generic_make_request+0x147/0x380
[<ffffffff98754630>] submit_bio+0x70/0x150
[<ffffffffc03a0071>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc03a03af>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff985cb1b1>] do_writepages+0x21/0x50
[<ffffffff9867cbd0>] __writeback_single_inode+0x40/0x260
[<ffffffff9842b59e>] ? __switch_to+0xce/0x580
[<ffffffff9867d764>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9867da6f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9867df53>] wb_writeback+0x263/0x2f0
[<ffffffff98669bac>] ? get_nr_inodes+0x4c/0x70
[<ffffffff9867eb4b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff984be6bf>] process_one_work+0x17f/0x440
[<ffffffff984bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff984bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff984c6691>] kthread+0xd1/0xe0
[<ffffffff984c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff98b92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff984c65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 15 
Complete report #101639
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc028a3be
Build id: 3.10.0-1160.11.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02d271e
Build id: 3.10.0-1160.11.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03432e0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa74e3b90
Build id: 3.10.0-1160.11.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa74ed2bc
Build id: 3.10.0-1160.11.1.el7.x86_64
0x4ac
6
? deadline_add_request
vmlinux 0xffffffffa737818e
Build id: 3.10.0-1160.11.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffa73536d9
Build id: 3.10.0-1160.11.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffa73573e3
Build id: 3.10.0-1160.11.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffa7355367
Build id: 3.10.0-1160.11.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffa7355610
Build id: 3.10.0-1160.11.1.el7.x86_64
0x70
11
? __switch_to
vmlinux 0xffffffffa702b59e
Build id: 3.10.0-1160.11.1.el7.x86_64
0xce
12
xfs_submit_ioend.isra.12
xfs 0xffffffffc052ef31
Build id: 3.10.0-1160.11.1.el7.x86_64
0x61
13
xfs_vm_writepages
xfs 0xffffffffc052f26f
Build id: 3.10.0-1160.11.1.el7.x86_64
0x7f
14
do_writepages
vmlinux 0xffffffffa71cafb1
Build id: 3.10.0-1160.11.1.el7.x86_64
0x21
15
__writeback_single_inode
vmlinux 0xffffffffa727d8d0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
16
writeback_sb_inodes
vmlinux 0xffffffffa727e454
Build id: 3.10.0-1160.11.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffa727e75f
Build id: 3.10.0-1160.11.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffa727ec33
Build id: 3.10.0-1160.11.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffa726a85c
Build id: 3.10.0-1160.11.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffa727f82b
Build id: 3.10.0-1160.11.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffa70bde9f
Build id: 3.10.0-1160.11.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffa70befb6
Build id: 3.10.0-1160.11.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffa70bee90
Build id: 3.10.0-1160.11.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffa70c5e71
Build id: 3.10.0-1160.11.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffa70c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffa7793df7
Build id: 3.10.0-1160.11.1.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffffa70c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u256:2:2884]
Modules linked in: rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw snd_ens1371 gf128mul glue_helper snd_rawmidi ablk_helper snd_ac97_codec cryptd ac97_bus btusb snd_seq
btrtl snd_seq_device btbcm vmw_balloon snd_pcm btintel bluetooth pcspkr snd_timer snd soundcore rfkill joydev sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c vmwgfx sr_mod cdrom ata_generic pata_acpi drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix drm nfit libnvdimm mptspi scsi_transport_spi libata serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 2884 Comm: kworker/u256:2 Not tainted 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff8b2e821f8000 ti: ffff8b2eb811c000 task.ti: ffff8b2eb811c000
RIP: 0010:[<ffffffffc028a3be>]  [<ffffffffc028a3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8b2eb811f888  EFLAGS: 00000246
RAX: ffff9dcf40200000 RBX: ffff8b2eb71d1010 RCX: 000000000000000b
RDX: ffff8b2eb64cc000 RSI: ffff8b2e80011000 RDI: 000000000000000e
RBP: ffff8b2eb811f898 R08: 0000000000000002 R09: 000000002cf4ffff
R10: ffff8b2eb5b63c00 R11: ffff8b2ea8d50c40 R12: ffff8b2eb3c41080
R13: ffff8b2eb811f7fc R14: ffffffffa735bffb R15: ffff8b2eb811f840
FS:  0000000000000000(0000) GS:ffff8b2ebc600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa6124ac000 CR3: 0000000000154000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc02d271e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03432e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa74e3b90>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa74ed2bc>] scsi_request_fn+0x4ac/0x680
[<ffffffffa737818e>] ? deadline_add_request+0x6e/0x90
[<ffffffffa73536d9>] __blk_run_queue+0x39/0x50
[<ffffffffa73573e3>] blk_queue_bio+0x3b3/0x400
[<ffffffffa7355367>] generic_make_request+0x147/0x380
[<ffffffffa7355610>] submit_bio+0x70/0x150
[<ffffffffa702b59e>] ? __switch_to+0xce/0x580
[<ffffffffc052ef31>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc052f26f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffa71cafb1>] do_writepages+0x21/0x50
[<ffffffffa727d8d0>] __writeback_single_inode+0x40/0x260
[<ffffffffa727e454>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa727e75f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa727ec33>] wb_writeback+0x263/0x2f0
[<ffffffffa726a85c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa727f82b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa70bde9f>] process_one_work+0x17f/0x440
[<ffffffffa70befb6>] worker_thread+0x126/0x3c0
[<ffffffffa70bee90>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa70c5e71>] kthread+0xd1/0xe0
[<ffffffffa70c5da0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa7793df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa70c5da0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 83 29 
Complete report #79237
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02be3be
Build id: 3.10.0-1062.12.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02e071e
Build id: 3.10.0-1062.12.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc02d92e0
Build id: 3.10.0-1062.12.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffbdcdeb10
Build id: 3.10.0-1062.12.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffbdce817c
Build id: 3.10.0-1062.12.1.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffffbdb738ee
Build id: 3.10.0-1062.12.1.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffffbdb4f0c9
Build id: 3.10.0-1062.12.1.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffffbdb52de3
Build id: 3.10.0-1062.12.1.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffffbdb50d77
Build id: 3.10.0-1062.12.1.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffffbdb51020
Build id: 3.10.0-1062.12.1.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0559cd1
Build id: 3.10.0-1062.12.1.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc055a00f
Build id: 3.10.0-1062.12.1.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffffbd9cabe1
Build id: 3.10.0-1062.12.1.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffffbda798c0
Build id: 3.10.0-1062.12.1.el7.x86_64
0x40
15
? __schedule
vmlinux 0xffffffffbdf805c2
Build id: 3.10.0-1062.12.1.el7.x86_64
0x402
16
writeback_sb_inodes
vmlinux 0xffffffffbda7a454
Build id: 3.10.0-1062.12.1.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffffbda7a75f
Build id: 3.10.0-1062.12.1.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffffbda7ac43
Build id: 3.10.0-1062.12.1.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffffbda66aec
Build id: 3.10.0-1062.12.1.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffffbda7b83b
Build id: 3.10.0-1062.12.1.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffffbd8be21f
Build id: 3.10.0-1062.12.1.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffffbd8bf336
Build id: 3.10.0-1062.12.1.el7.x86_64
0x126
23
? manage_workers.isra.26
vmlinux 0xffffffffbd8bf210
Build id: 3.10.0-1062.12.1.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffffbd8c61f1
Build id: 3.10.0-1062.12.1.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffffbd8c6120
Build id: 3.10.0-1062.12.1.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffbdf8dd24
Build id: 3.10.0-1062.12.1.el7.x86_64
0xe
27
? insert_kthread_work
vmlinux 0xffffffffbd8c6120
Build id: 3.10.0-1062.12.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 58s! [kworker/u256:1:113354]
Modules linked in: tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter fuse vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul vmw_balloon glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic
pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel nfit ata_piix libnvdimm drm libata mptspi serio_raw scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 113354 Comm: kworker/u256:1 Not tainted 3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9671b54fb150 ti: ffff9672a9664000 task.ti: ffff9672a9664000
RIP: 0010:[<ffffffffc02be3be>]  [<ffffffffc02be3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9672a9667888  EFLAGS: 00000246
RAX: ffffba7b80f80000 RBX: ffff96713e22b000 RCX: 0000000000000009
RDX: ffff967135f9de00 RSI: ffff967331171000 RDI: 000000000000000e
RBP: ffff9672a9667898 R08: 0000000000000002 R09: 000000014ac8dfff
R10: ffff9673311700d8 R11: ffff967326b5b2c0 R12: ffff9672b8afa100
R13: ffff9672a96677fc R14: ffffffffbdb5792b R15: ffff9672a9667840
FS:  00007f3d58e4e700(0000) GS:ffff967335600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f60d9f61000 CR3: 00000002266ea000 CR4: 00000000003407f0
Call Trace:
[<ffffffffc02e071e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02d92e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffbdcdeb10>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbdce817c>] scsi_request_fn+0x4cc/0x680
[<ffffffffbdb738ee>] ? deadline_add_request+0x6e/0x90
[<ffffffffbdb4f0c9>] __blk_run_queue+0x39/0x50
[<ffffffffbdb52de3>] blk_queue_bio+0x3b3/0x400
[<ffffffffbdb50d77>] generic_make_request+0x147/0x380
[<ffffffffbdb51020>] submit_bio+0x70/0x150
[<ffffffffc0559cd1>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc055a00f>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffffbd9cabe1>] do_writepages+0x21/0x50
[<ffffffffbda798c0>] __writeback_single_inode+0x40/0x260
[<ffffffffbdf805c2>] ? __schedule+0x402/0x840
[<ffffffffbda7a454>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffbda7a75f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbda7ac43>] wb_writeback+0x263/0x2f0
[<ffffffffbda66aec>] ? get_nr_inodes+0x4c/0x70
[<ffffffffbda7b83b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbd8be21f>] process_one_work+0x17f/0x440
[<ffffffffbd8bf336>] worker_thread+0x126/0x3c0
[<ffffffffbd8bf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffbd8c61f1>] kthread+0xd1/0xe0
[<ffffffffbd8c6120>] ? insert_kthread_work+0x40/0x40
[<ffffffffbdf8dd24>] ret_from_fork_nospec_begin+0xe/0x21
[<ffffffffbd8c6120>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 c3 2c 
Complete report #6072
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00503be
Build id: 3.10.0-693.17.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc0081729
Build id: 3.10.0-693.17.1.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffc00e92b0
Build id: 3.10.0-693.17.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146782a
Build id: 3.10.0-693.17.1.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff814705ef
Build id: 3.10.0-693.17.1.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f96a3
Build id: 3.10.0-693.17.1.el7.x86_64
0x33
7
blk_queue_bio
vmlinux 0xffffffff812fcfa3
Build id: 3.10.0-693.17.1.el7.x86_64
0x363
8
generic_make_request
vmlinux 0xffffffff812fb015
Build id: 3.10.0-693.17.1.el7.x86_64
0x105
9
submit_bio
vmlinux 0xffffffff812fb290
Build id: 3.10.0-693.17.1.el7.x86_64
0x70
10
xfs_submit_ioend.isra.16
xfs 0xffffffffc028b661
Build id: 3.10.0-693.17.1.el7.x86_64
0x61
11
xfs_vm_writepages
xfs 0xffffffffc028b821
Build id: 3.10.0-693.17.1.el7.x86_64
0xd1
12
do_writepages
vmlinux 0xffffffff81190efe
Build id: 3.10.0-693.17.1.el7.x86_64
0x1e
13
__writeback_single_inode
vmlinux 0xffffffff8122fa80
Build id: 3.10.0-693.17.1.el7.x86_64
0x40
14
writeback_sb_inodes
vmlinux 0xffffffff812306c4
Build id: 3.10.0-693.17.1.el7.x86_64
0x1c4
15
__writeback_inodes_wb
vmlinux 0xffffffff81230a2f
Build id: 3.10.0-693.17.1.el7.x86_64
0x9f
16
wb_writeback
vmlinux 0xffffffff81231263
Build id: 3.10.0-693.17.1.el7.x86_64
0x263
17
? get_nr_inodes
vmlinux 0xffffffff8121de9c
Build id: 3.10.0-693.17.1.el7.x86_64
0x4c
18
bdi_writeback_workfn
vmlinux 0xffffffff8123170b
Build id: 3.10.0-693.17.1.el7.x86_64
0x2cb
19
process_one_work
vmlinux 0xffffffff810aa59a
Build id: 3.10.0-693.17.1.el7.x86_64
0x17a
20
worker_thread
vmlinux 0xffffffff810ab266
Build id: 3.10.0-693.17.1.el7.x86_64
0x126
21
? manage_workers.isra.24
vmlinux 0xffffffff810ab140
Build id: 3.10.0-693.17.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff810b270f
Build id: 3.10.0-693.17.1.el7.x86_64
0xcf
23
? insert_kthread_work
vmlinux 0xffffffff810b2640
Build id: 3.10.0-693.17.1.el7.x86_64
0x40
24
ret_from_fork
vmlinux 0xffffffff816b8798
Build id: 3.10.0-693.17.1.el7.x86_64
0x58
25
? insert_kthread_work
vmlinux 0xffffffff810b2640
Build id: 3.10.0-693.17.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/u256:0:6167]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) ext4 mbcache jbd2 snd_seq_midi snd_seq_midi_event snd_ens1371 iosf_mbi crc32_pclmul ghash_clmulni_intel snd_rawmidi snd_ac97_codec aesni_intel lrw gf128mul ppdev ac97_bus glue_helper ablk_helper cryptd snd_seq vmw_balloon snd_seq_device joydev
snd_pcm pcspkr snd_timer snd soundcore i2c_piix4 vmw_vmci nfit sg libnvdimm parport_pc parport shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ata_piix mptspi libata scsi_transport_spi mptscsih vmxnet3 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 6167 Comm: kworker/u256:0 Tainted: G           OEL ------------   3.10.0-693.17.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff88009e9e0000 ti: ffff880122020000 task.ti: ffff880122020000
RIP: 0010:[<ffffffffc00503be>]  [<ffffffffc00503be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff880122023890  EFLAGS: 00000246
RAX: ffffc90000700000 RBX: ffff8800b0671440 RCX: 0000000000000015
RDX: ffff88013182da00 RSI: ffff880035d02000 RDI: 000000000000000e
RBP: ffff8801220238a0 R08: 0000000000000003 R09: ffff88013691e0d8
R10: ffff8800b5cdb640 R11: ffff8800b5cdb640 R12: 0000000100000003
R13: ffff8800bbb04180 R14: ffff880122023804 R15: ffffffff8130160b
FS:  0000000000000000(0000) GS:ffff880139640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005581bd821010 CR3: 0000000134946000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0081729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00e92b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146782a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff814705ef>] scsi_request_fn+0x4df/0x680
[<ffffffff812f96a3>] __blk_run_queue+0x33/0x40
[<ffffffff812fcfa3>] blk_queue_bio+0x363/0x3a0
[<ffffffff812fb015>] generic_make_request+0x105/0x310
[<ffffffff812fb290>] submit_bio+0x70/0x150
[<ffffffffc028b661>] xfs_submit_ioend.isra.16+0x61/0xe0 [xfs]
[<ffffffffc028b821>] xfs_vm_writepages+0xd1/0xe0 [xfs]
[<ffffffff81190efe>] do_writepages+0x1e/0x40
[<ffffffff8122fa80>] __writeback_single_inode+0x40/0x220
[<ffffffff812306c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff81230a2f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff81231263>] wb_writeback+0x263/0x2f0
[<ffffffff8121de9c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8123170b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810aa59a>] process_one_work+0x17a/0x440
[<ffffffff810ab266>] worker_thread+0x126/0x3c0
[<ffffffff810ab140>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b270f>] kthread+0xcf/0xe0
[<ffffffff810b2640>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b8798>] ret_from_fork+0x58/0x90
[<ffffffff810b2640>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 05 
Complete report #1454
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01c33be
Build id: 3.10.0-957.21.3.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc01e671e
Build id: 3.10.0-957.21.3.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc01de2e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff960d1700
Build id: 3.10.0-957.21.3.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff960dac9c
Build id: 3.10.0-957.21.3.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff95f67e6e
Build id: 3.10.0-957.21.3.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff95f43849
Build id: 3.10.0-957.21.3.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff95f47553
Build id: 3.10.0-957.21.3.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff95f454e7
Build id: 3.10.0-957.21.3.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff95f45790
Build id: 3.10.0-957.21.3.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc0416471
Build id: 3.10.0-957.21.3.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc04165df
Build id: 3.10.0-957.21.3.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff95dc36d1
Build id: 3.10.0-957.21.3.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff95e701b0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
15
? __schedule
vmlinux 0xffffffff96368a7a
Build id: 3.10.0-957.21.3.el7.x86_64
0x42a
16
writeback_sb_inodes
vmlinux 0xffffffff95e70c44
Build id: 3.10.0-957.21.3.el7.x86_64
0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff95e70faf
Build id: 3.10.0-957.21.3.el7.x86_64
0x9f
18
wb_writeback
vmlinux 0xffffffff95e717e3
Build id: 3.10.0-957.21.3.el7.x86_64
0x263
19
? get_nr_inodes
vmlinux 0xffffffff95e5d5fc
Build id: 3.10.0-957.21.3.el7.x86_64
0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff95e7216b
Build id: 3.10.0-957.21.3.el7.x86_64
0x2cb
21
process_one_work
vmlinux 0xffffffff95cb9ebf
Build id: 3.10.0-957.21.3.el7.x86_64
0x17f
22
worker_thread
vmlinux 0xffffffff95cbaf56
Build id: 3.10.0-957.21.3.el7.x86_64
0x126
23
? manage_workers.isra.25
vmlinux 0xffffffff95cbae30
Build id: 3.10.0-957.21.3.el7.x86_64
0x2a0
24
kthread
vmlinux 0xffffffff95cc1da1
Build id: 3.10.0-957.21.3.el7.x86_64
0xd1
25
? insert_kthread_work
vmlinux 0xffffffff95cc1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff96375c37
Build id: 3.10.0-957.21.3.el7.x86_64
0x21
27
? insert_kthread_work
vmlinux 0xffffffff95cc1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/u16:1:18926]
Modules linked in: tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon pcspkr sg joydev vmw_vmci i2c_piix4 parport_pc parport
ip_tables xfs libcrc32c vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper ata_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm pata_acpi drm nfit mptspi scsi_transport_spi e1000 mptscsih ata_piix libnvdimm libata crct10dif_pclmul crct10dif_common mptbase crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 18926 Comm: kworker/u16:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff94cccfe16180 ti: ffff94ca86c2c000 task.ti: ffff94ca86c2c000
RIP: 0010:[<ffffffffc01c33be>]  [<ffffffffc01c33be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff94ca86c2f888  EFLAGS: 00000246
RAX: ffffaaa401c00000 RBX: ffff94cd26ef4000 RCX: 000000000000001e
RDX: ffff94cd2a306800 RSI: ffff94cd2acef000 RDI: 000000000000000e
RBP: ffff94ca86c2f898 R08: 0000000000000002 R09: 00000002a4938fff
R10: ffff94cd2afc50d8 R11: ffff94cd183d1c00 R12: ffff94c93501af00
R13: ffff94ca86c2f7fc R14: ffffffff95f4c09b R15: ffff94ca86c2f840
FS:  0000000000000000(0000) GS:ffff94cd3fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f91153c8000 CR3: 00000003f7aae000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc01e671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01de2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff960d1700>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff960dac9c>] scsi_request_fn+0x4cc/0x680
[<ffffffff95f67e6e>] ? deadline_add_request+0x6e/0x90
[<ffffffff95f43849>] __blk_run_queue+0x39/0x50
[<ffffffff95f47553>] blk_queue_bio+0x3b3/0x400
[<ffffffff95f454e7>] generic_make_request+0x147/0x380
[<ffffffff95f45790>] submit_bio+0x70/0x150
[<ffffffffc0416471>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc04165df>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff95dc36d1>] do_writepages+0x21/0x50
[<ffffffff95e701b0>] __writeback_single_inode+0x40/0x260
[<ffffffff96368a7a>] ? __schedule+0x42a/0x860
[<ffffffff95e70c44>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff95e70faf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff95e717e3>] wb_writeback+0x263/0x2f0
[<ffffffff95e5d5fc>] ? get_nr_inodes+0x4c/0x70
[<ffffffff95e7216b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff95cb9ebf>] process_one_work+0x17f/0x440
[<ffffffff95cbaf56>] worker_thread+0x126/0x3c0
[<ffffffff95cbae30>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff95cc1da1>] kthread+0xd1/0xe0
[<ffffffff95cc1cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff96375c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff95cc1cd0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 13 1d 
Complete report #327
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03983be
Build id: 3.10.0-957.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03b371e
Build id: 3.10.0-957.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04432e0
Build id: 3.10.0-957.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff85ad0960
Build id: 3.10.0-957.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff85ad9efc
Build id: 3.10.0-957.el7.x86_64
0x4cc
6
? deadline_add_request
vmlinux 0xffffffff859675fe
Build id: 3.10.0-957.el7.x86_64
0x6e
7
__blk_run_queue
vmlinux 0xffffffff85942ff9
Build id: 3.10.0-957.el7.x86_64
0x39
8
blk_queue_bio
vmlinux 0xffffffff85946cf3
Build id: 3.10.0-957.el7.x86_64
0x3b3
9
generic_make_request
vmlinux 0xffffffff85944c87
Build id: 3.10.0-957.el7.x86_64
0x147
10
submit_bio
vmlinux 0xffffffff85944f30
Build id: 3.10.0-957.el7.x86_64
0x70
11
xfs_submit_ioend.isra.12
xfs 0xffffffffc05e9251
Build id: 3.10.0-957.el7.x86_64
0x61
12
xfs_vm_writepages
xfs 0xffffffffc05e93bf
Build id: 3.10.0-957.el7.x86_64
0x7f
13
do_writepages
vmlinux 0xffffffff857c3181
Build id: 3.10.0-957.el7.x86_64
0x21
14
__writeback_single_inode
vmlinux 0xffffffff8586f930
Build id: 3.10.0-957.el7.x86_64
0x40
15
writeback_sb_inodes
vmlinux 0xffffffff858703c4
Build id: 3.10.0-957.el7.x86_64
0x1c4
16
__writeback_inodes_wb
vmlinux 0xffffffff8587072f
Build id: 3.10.0-957.el7.x86_64
0x9f
17
wb_writeback
vmlinux 0xffffffff85870f63
Build id: 3.10.0-957.el7.x86_64
0x263
18
? get_nr_inodes
vmlinux 0xffffffff8585cd7c
Build id: 3.10.0-957.el7.x86_64
0x4c
19
bdi_writeback_workfn
vmlinux 0xffffffff858718eb
Build id: 3.10.0-957.el7.x86_64
0x2cb
20
process_one_work
vmlinux 0xffffffff856b9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff856bade6
Build id: 3.10.0-957.el7.x86_64
0x126
22
? manage_workers.isra.25
vmlinux 0xffffffff856bacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff856c1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff856c1b60
Build id: 3.10.0-957.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff85d74c1d
Build id: 3.10.0-957.el7.x86_64
0x7
26
? insert_kthread_work
vmlinux 0xffffffff856c1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 33s! [kworker/u256:1:47297]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vmw_vmci vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore joydev pcspkr sg parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel ata_generic
pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptspi scsi_transport_spi e1000 libata mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 47297 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8e33951b1040 ti: ffff8e3461fb0000 task.ti: ffff8e3461fb0000
RIP: 0010:[<ffffffffc03983be>]  [<ffffffffc03983be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8e3461fb3888  EFLAGS: 00000246
RAX: ffffb9dec1080000 RBX: ffff8e339192e5d0 RCX: 0000000000000008
RDX: ffff8e3335784600 RSI: ffff8e346fc8c000 RDI: 000000000000000e
RBP: ffff8e3461fb3898 R08: 0000000000000002 R09: 0000000034827fff
R10: ffff8e346fc8e0d8 R11: ffff8e346ffb2300 R12: ffff8e3423f2de00
R13: ffff8e3461fb37fc R14: ffffffff8594b83b R15: ffff8e3461fb3840
FS:  0000000000000000(0000) GS:ffff8e3478600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa6f2f14000 CR3: 0000000035c66000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc03b371e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04432e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff85ad0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff85ad9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff859675fe>] ? deadline_add_request+0x6e/0x90
[<ffffffff85942ff9>] __blk_run_queue+0x39/0x50
[<ffffffff85946cf3>] blk_queue_bio+0x3b3/0x400
[<ffffffff85944c87>] generic_make_request+0x147/0x380
[<ffffffff85944f30>] submit_bio+0x70/0x150
[<ffffffffc05e9251>] xfs_submit_ioend.isra.12+0x61/0xe0 [xfs]
[<ffffffffc05e93bf>] xfs_vm_writepages+0x7f/0xa0 [xfs]
[<ffffffff857c3181>] do_writepages+0x21/0x50
[<ffffffff8586f930>] __writeback_single_inode+0x40/0x260
[<ffffffff858703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8587072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff85870f63>] wb_writeback+0x263/0x2f0
[<ffffffff8585cd7c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff858718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff856b9d4f>] process_one_work+0x17f/0x440
[<ffffffff856bade6>] worker_thread+0x126/0x3c0
[<ffffffff856bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff856c1c31>] kthread+0xd1/0xe0
[<ffffffff856c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff85d74c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff856c1b60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 3a