Info

Permalink
Function
_raw_spin_unlock_irqrestore
First occurrence
2020-09-23
Last occurrence
2026-08-04
State
NEW
Components
kernel
Quality
-50

Statistics

Operating system Count
CentOS 7 418
Architecture Count
x86_64 418
Related packages Count
kernel 418
 0:3.10.0-1062.el7 81
 0:3.10.0-693.5.2.el7 70
 0:3.10.0-862.el7 59
 0:3.10.0-514.16.1.el7 52
 0:3.10.0-957.el7 27
 0:3.10.0-1062.18.1.el7 22
 0:3.10.0-957.21.3.el7 12
 0:3.10.0-327.el7 11
 0:3.10.0-693.el7 11
 0:3.10.0-957.27.2.el7 8
 0:3.10.0-1062.12.1.el7 8
 0:3.10.0-862.11.6.el7 8
 0:3.10.0-1062.9.1.el7 6
 0:3.10.0-693.11.6.el7 5
 0:3.10.0-693.21.1.el7 5
 0:3.10.0-862.2.3.el7 4
 0:3.10.0-957.1.3.el7 4
 0:3.10.0-514.el7 3
 0:3.10.0-862.9.1.el7 3
 0:3.10.0-1062.1.2.el7 3
 0:3.10.0-957.12.2.el7 3
 0:3.10.0-957.10.1.el7 2
 0:3.10.0-957.12.1.el7 2
 0:3.10.0-1062.4.3.el7 2
 0:3.10.0-514.10.2.el7 1
 0:3.10.0-514.21.2.el7 1
 0:3.10.0-514.26.2.el7 1
 0:3.10.0-693.17.1.el7 1
 0:3.10.0-693.2.2.el7 1
 0:3.10.0-862.14.4.el7 1
 0:3.10.0-862.6.3.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1182567
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff816ab845
Build id: 3.10.0-693.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0052632
Build id: 3.10.0-693.el7.x86_64 		0x442
3
? scsi_test_unit_ready
vmlinux 0xffffffff8146d190
Build id: 3.10.0-693.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64 		0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64 		0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64 		0x33
7
queue_unplugged
vmlinux 0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64 		0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc02c3354
Build id: 3.10.0-693.el7.x86_64 		0x334
11
? xlog_bdstrat
xfs 0xffffffffc02e4d5b
Build id: 3.10.0-693.el7.x86_64 		0x2b
12
xfs_buf_submit
xfs 0xffffffffc02c4b70
Build id: 3.10.0-693.el7.x86_64 		0x70
13
xlog_bdstrat
xfs 0xffffffffc02e4d5b
Build id: 3.10.0-693.el7.x86_64 		0x2b
14
xlog_sync
xfs 0xffffffffc02e6c7e
Build id: 3.10.0-693.el7.x86_64 		0x2fe
15
xlog_state_release_iclog
xfs 0xffffffffc02e6e1b
Build id: 3.10.0-693.el7.x86_64 		0x7b
16
_xfs_log_force
xfs 0xffffffffc02e7d63
Build id: 3.10.0-693.el7.x86_64 		0x233
17
? set_next_entity
vmlinux 0xffffffff810cb62c
Build id: 3.10.0-693.el7.x86_64 		0x3c
18
? __switch_to
vmlinux 0xffffffff81029557
Build id: 3.10.0-693.el7.x86_64 		0xd7
19
? xfs_log_worker
xfs 0xffffffffc02e7e84
Build id: 3.10.0-693.el7.x86_64 		0x24
20
xfs_log_force
xfs 0xffffffffc02e7e1c
Build id: 3.10.0-693.el7.x86_64 		0x2c
21
xfs_log_worker
xfs 0xffffffffc02e7e84
Build id: 3.10.0-693.el7.x86_64 		0x24
22
process_one_work
vmlinux 0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64 		0x17a
23
worker_thread
vmlinux 0xffffffff810a94e6
Build id: 3.10.0-693.el7.x86_64 		0x126
24
? manage_workers.isra.24
vmlinux 0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64 		0x2a0
25
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64 		0xcf
26
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#21 stuck for 23s! [kworker/21:2:11040]
Modules linked in: udp_diag tcp_diag inet_diag xt_comment xt_multiport xt_set ip_set_hash_ip ip_set nf_conntrack_netlink nfnetlink sfesp(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg vmw_vmci i2c_piix4 shpchp nfit parport_pc parport libnvdimm nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm crct10dif_pclmul e1000e crct10dif_common libahci ata_piix crc32c_intel libata serio_raw ptp vmw_pvscsi pps_core i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 21 PID: 11040 Comm: kworker/21:2 Tainted: G           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/09/2019
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff88082adc9fa0 ti: ffff88044608c000 task.ti: ffff88044608c000
RIP: 0010:[<ffffffff816ab845>]  [<ffffffff816ab845>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff88044608fa18  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff88044608f9d8 RCX: 000000082c277fff
RDX: ffffc90003170000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff88044608fa18 R08: 0000000000000001 R09: 0000000000000000
R10: ffff88082b733100 R11: ffff88082c2e8000 R12: 0000000000000020
R13: ffff8805d28d8c00 R14: ffffffff8146bdb9 R15: ffff88044608f990
FS:  0000000000000000(0000) GS:ffff88083f740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fec14005660 CR3: 000000082772e000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88044608fa90 ffffffffc0052632 ffff88079b323a00 ffff88082be68000
ffff88082c016b40 ffff88082d7437e8 ffffffff8146d190 0000000000000006
ffff88082d743754 0000000000000286 ffff88082b733100 0000000000000000
Call Trace:
[<ffffffffc0052632>] pvscsi_queue+0x442/0x7e1 [vmw_pvscsi]
[<ffffffff8146d190>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffffc02c3354>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc02e4d5b>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc02c4b70>] xfs_buf_submit+0x70/0x1f0 [xfs]
[<ffffffffc02e4d5b>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc02e6c7e>] xlog_sync+0x2fe/0x420 [xfs]
[<ffffffffc02e6e1b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc02e7d63>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffff810cb62c>] ? set_next_entity+0x3c/0xe0
[<ffffffff81029557>] ? __switch_to+0xd7/0x510
[<ffffffffc02e7e84>] ? xfs_log_worker+0x24/0x50 [xfs]
[<ffffffffc02e7e1c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc02e7e84>] xfs_log_worker+0x24/0x50 [xfs]
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a94e6>] worker_thread+0x126/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Complete report #806440
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03383be
Build id: 3.10.0-957.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03b271e
Build id: 3.10.0-957.el7.x86_64 		0x24e
3
mptsas_qcmd
mptsas 0xffffffffc0397cc9
Build id: 3.10.0-957.el7.x86_64 		0x79
4
? blk_add_timer
vmlinux 0xffffffffa1f4d06b
Build id: 3.10.0-957.el7.x86_64 		0xcb
5
scsi_dispatch_cmd
vmlinux 0xffffffffa20d0960
Build id: 3.10.0-957.el7.x86_64 		0xb0
6
scsi_request_fn
vmlinux 0xffffffffa20d9efc
Build id: 3.10.0-957.el7.x86_64 		0x4cc
7
__blk_run_queue
vmlinux 0xffffffffa1f42ff9
Build id: 3.10.0-957.el7.x86_64 		0x39
8
queue_unplugged
vmlinux 0xffffffffa1f430ba
Build id: 3.10.0-957.el7.x86_64 		0x2a
9
blk_flush_plug_list
vmlinux 0xffffffffa1f468e8
Build id: 3.10.0-957.el7.x86_64 		0x1d8
10
blk_finish_plug
vmlinux 0xffffffffa1f46d54
Build id: 3.10.0-957.el7.x86_64 		0x14
11
_xfs_buf_ioapply
xfs 0xffffffffc05fb454
Build id: 3.10.0-957.el7.x86_64 		0x334
12
? xlog_bdstrat
xfs 0xffffffffc061dacb
Build id: 3.10.0-957.el7.x86_64 		0x2b
13
xfs_buf_submit
xfs 0xffffffffc05fcfcc
Build id: 3.10.0-957.el7.x86_64 		0x6c
14
xlog_bdstrat
xfs 0xffffffffc061dacb
Build id: 3.10.0-957.el7.x86_64 		0x2b
15
xlog_sync
xfs 0xffffffffc061f956
Build id: 3.10.0-957.el7.x86_64 		0x2e6
16
xlog_state_release_iclog
xfs 0xffffffffc061fadb
Build id: 3.10.0-957.el7.x86_64 		0x7b
17
_xfs_log_force
xfs 0xffffffffc0620c33
Build id: 3.10.0-957.el7.x86_64 		0x233
18
? xfs_log_worker
xfs 0xffffffffc0620d66
Build id: 3.10.0-957.el7.x86_64 		0x36
19
xfs_log_force
xfs 0xffffffffc0620cec
Build id: 3.10.0-957.el7.x86_64 		0x2c
20
xfs_log_worker
xfs 0xffffffffc0620d66
Build id: 3.10.0-957.el7.x86_64 		0x36
21
process_one_work
vmlinux 0xffffffffa1cb9d4f
Build id: 3.10.0-957.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffffa1cbade6
Build id: 3.10.0-957.el7.x86_64 		0x126
23
? manage_workers.isra.25
vmlinux 0xffffffffa1cbacc0
Build id: 3.10.0-957.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffffa1cc1c31
Build id: 3.10.0-957.el7.x86_64 		0xd1
25
? SyS_futex
vmlinux 0xffffffffa1d0fda0
Build id: 3.10.0-957.el7.x86_64 		0x80
26
? insert_kthread_work
vmlinux 0xffffffffa1cc1b60
Build id: 3.10.0-957.el7.x86_64 		0x40
27
ret_from_fork_nospec_begin
vmlinux 0xffffffffa2374c37
Build id: 3.10.0-957.el7.x86_64 		0x21
28
? insert_kthread_work
vmlinux 0xffffffffa1cc1b60
Build id: 3.10.0-957.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 63s! [kworker/11:0:30257]
Modules linked in: vmw_vsock_vmci_transport vsock vfat fat sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport tpm_crb vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm e1000e ahci ata_piix mptsas libahci drm scsi_transport_sas mptscsih ptp libata mptbase pps_core nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 30257 Comm: kworker/11:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.17369862.B64.2012240522 12/24/2020
Workqueue: xfs-sync/dm-2 xfs_log_worker [xfs]
task: ffff8b8df8c9d140 ti: ffff8b888d2dc000 task.ti: ffff8b888d2dc000
RIP: 0010:[<ffffffffc03383be>]  [<ffffffffc03383be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8b888d2df9a8  EFLAGS: 00000246
RAX: ffff9b60c31b8000 RBX: 000000003728e025 RCX: 000000000000005a
RDX: ffff8b90bfe78a00 RSI: ffff8b90bed7f000 RDI: 000000000000000e
RBP: ffff8b888d2df9b8 R08: 0000000000000002 R09: 00000002ebc3cfff
R10: ffff8b89f40ff0d8 R11: ffff8b90b6ec7b80 R12: ffff8b8fe7320aa0
R13: 0000000100000001 R14: ffff8b8be8f2f600 R15: ffff8b888d2df90c
FS:  0000000000000000(0000) GS:ffff8b90bd8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000448d70 CR3: 00000006df348000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc03b271e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc0397cc9>] mptsas_qcmd+0x79/0xa0 [mptsas]
[<ffffffffa1f4d06b>] ? blk_add_timer+0xcb/0x110
[<ffffffffa20d0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa20d9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffa1f42ff9>] __blk_run_queue+0x39/0x50
[<ffffffffa1f430ba>] queue_unplugged+0x2a/0xa0
[<ffffffffa1f468e8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa1f46d54>] blk_finish_plug+0x14/0x40
[<ffffffffc05fb454>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc061dacb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc05fcfcc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc061dacb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc061f956>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc061fadb>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc0620c33>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffffc0620d66>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc0620cec>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc0620d66>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffa1cb9d4f>] process_one_work+0x17f/0x440
[<ffffffffa1cbade6>] worker_thread+0x126/0x3c0
[<ffffffffa1cbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa1cc1c31>] kthread+0xd1/0xe0
[<ffffffffa1d0fda0>] ? SyS_futex+0x80/0x190
[<ffffffffa1cc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa2374c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa1cc1b60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 34
Complete report #895964
Frame # Function Binary Source or offset Line
1
? scsi_request_fn
vmlinux 0xffffffffb16e5efc
Build id: 3.10.0-1062.el7.x86_64 		0x4cc
2
__blk_run_queue
vmlinux 0xffffffffb154cf59
Build id: 3.10.0-1062.el7.x86_64 		0x39
3
queue_unplugged
vmlinux 0xffffffffb154d01a
Build id: 3.10.0-1062.el7.x86_64 		0x2a
4
blk_flush_plug_list
vmlinux 0xffffffffb1550858
Build id: 3.10.0-1062.el7.x86_64 		0x1d8
5
blk_finish_plug
vmlinux 0xffffffffb1550cc4
Build id: 3.10.0-1062.el7.x86_64 		0x14
6
_xfs_buf_ioapply
xfs 0xffffffffc0a1b044
Build id: 3.10.0-1062.el7.x86_64 		0x334
7
? xfs_buf_submit
xfs 0xffffffffc0a1cb4a
Build id: 3.10.0-1062.el7.x86_64 		0x1a
8
__xfs_buf_submit
xfs 0xffffffffc0a1c8c1
Build id: 3.10.0-1062.el7.x86_64 		0x61
9
xfs_buf_submit
xfs 0xffffffffc0a1cb4a
Build id: 3.10.0-1062.el7.x86_64 		0x1a
10
xlog_bdstrat
xfs 0xffffffffc0a3d0cb
Build id: 3.10.0-1062.el7.x86_64 		0x2b
11
xlog_sync
xfs 0xffffffffc0a3eec6
Build id: 3.10.0-1062.el7.x86_64 		0x2e6
12
xlog_state_release_iclog
xfs 0xffffffffc0a3f04b
Build id: 3.10.0-1062.el7.x86_64 		0x7b
13
_xfs_log_force
xfs 0xffffffffc0a4019b
Build id: 3.10.0-1062.el7.x86_64 		0x22b
14
? __queue_delayed_work
vmlinux 0xffffffffb12bb3db
Build id: 3.10.0-1062.el7.x86_64 		0x8b
15
? mutex_lock
vmlinux 0xffffffffb197d2c2
Build id: 3.10.0-1062.el7.x86_64 		0x12
16
? xfs_log_worker
xfs 0xffffffffc0a402b6
Build id: 3.10.0-1062.el7.x86_64 		0x36
17
xfs_log_force
xfs 0xffffffffc0a4023c
Build id: 3.10.0-1062.el7.x86_64 		0x2c
18
xfs_log_worker
xfs 0xffffffffc0a402b6
Build id: 3.10.0-1062.el7.x86_64 		0x36
19
process_one_work
vmlinux 0xffffffffb12bd0ff
Build id: 3.10.0-1062.el7.x86_64 		0x17f
20
worker_thread
vmlinux 0xffffffffb12be216
Build id: 3.10.0-1062.el7.x86_64 		0x126
21
? manage_workers.isra.26
vmlinux 0xffffffffb12be0f0
Build id: 3.10.0-1062.el7.x86_64 		0x2a0
22
kthread
vmlinux 0xffffffffb12c50d1
Build id: 3.10.0-1062.el7.x86_64 		0xd1
23
? insert_kthread_work
vmlinux 0xffffffffb12c5000
Build id: 3.10.0-1062.el7.x86_64 		0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffb198bd37
Build id: 3.10.0-1062.el7.x86_64 		0x21
25
? insert_kthread_work
vmlinux 0xffffffffb12c5000
Build id: 3.10.0-1062.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 56s! [ora_clmn_testdb:17533]
Workqueue: xfs-sync/dm-2 xfs_log_worker [xfs]
Call Trace:
[<ffffffffb16e5efc>] ? scsi_request_fn+0x4cc/0x680
[<ffffffffb154cf59>] __blk_run_queue+0x39/0x50
[<ffffffffb154d01a>] queue_unplugged+0x2a/0xa0
[<ffffffffb1550858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb1550cc4>] blk_finish_plug+0x14/0x40
[<ffffffffc0a1b044>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0a1cb4a>] ? xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc0a1c8c1>] __xfs_buf_submit+0x61/0x210 [xfs]
[<ffffffffc0a1cb4a>] xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc0a3d0cb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc0a3eec6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc0a3f04b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc0a4019b>] _xfs_log_force+0x22b/0x2a0 [xfs]
[<ffffffffb12bb3db>] ? __queue_delayed_work+0x8b/0x1a0
[<ffffffffb197d2c2>] ? mutex_lock+0x12/0x2f
[<ffffffffc0a402b6>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc0a4023c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc0a402b6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffb12bd0ff>] process_one_work+0x17f/0x440
[<ffffffffb12be216>] worker_thread+0x126/0x3c0
[<ffffffffb12be0f0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb12c50d1>] kthread+0xd1/0xe0
[<ffffffffb12c5000>] ? insert_kthread_work+0x40/0x40
[<ffffffffb198bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb12c5000>] ? insert_kthread_work+0x40/0x40
Complete report #55666
Frame # Function Binary Source or offset Line
1
queue_unplugged
vmlinux 0xffffffffb854e8ba
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x2a
2
blk_flush_plug_list
vmlinux 0xffffffffb8552108
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x1d8
3
blk_finish_plug
vmlinux 0xffffffffb8552574
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x14
4
_xfs_buf_ioapply
xfs 0xffffffffc04c2044
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x334
5
? xfs_buf_submit
xfs 0xffffffffc04c3b4a
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x1a
6
__xfs_buf_submit
xfs 0xffffffffc04c38c1
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x61
7
xfs_buf_submit
xfs 0xffffffffc04c3b4a
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x1a
8
xlog_bdstrat
xfs 0xffffffffc04e40cb
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x2b
9
xlog_sync
xfs 0xffffffffc04e5ec6
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x2e6
10
xlog_state_release_iclog
xfs 0xffffffffc04e604b
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x7b
11
_xfs_log_force
xfs 0xffffffffc04e719b
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x22b
12
? mod_timer
vmlinux 0xffffffffb82af14c
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x10c
13
? xfs_log_worker
xfs 0xffffffffc04e72b6
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x36
14
xfs_log_force
xfs 0xffffffffc04e723c
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x2c
15
xfs_log_worker
xfs 0xffffffffc04e72b6
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x36
16
process_one_work
vmlinux 0xffffffffb82be21f
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x17f
17
worker_thread
vmlinux 0xffffffffb82bf336
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x126
18
? manage_workers.isra.26
vmlinux 0xffffffffb82bf210
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x2a0
19
kthread
vmlinux 0xffffffffb82c61f1
Build id: 3.10.0-1062.4.3.el7.x86_64 		0xd1
20
? insert_kthread_work
vmlinux 0xffffffffb82c6120
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x40
21
ret_from_fork_nospec_begin
vmlinux 0xffffffffb898cd37
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x21
22
? insert_kthread_work
vmlinux 0xffffffffb82c6120
Build id: 3.10.0-1062.4.3.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 81s! [kworker/2:3:4979]
[<ffffffffb854e8ba>] queue_unplugged+0x2a/0xa0
[<ffffffffb8552108>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb8552574>] blk_finish_plug+0x14/0x40
[<ffffffffc04c2044>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc04c3b4a>] ? xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc04c38c1>] __xfs_buf_submit+0x61/0x210 [xfs]
[<ffffffffc04c3b4a>] xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc04e40cb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc04e5ec6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc04e604b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc04e719b>] _xfs_log_force+0x22b/0x2a0 [xfs]
[<ffffffffb82af14c>] ? mod_timer+0x10c/0x230
[<ffffffffc04e72b6>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc04e723c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc04e72b6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffb82be21f>] process_one_work+0x17f/0x440
[<ffffffffb82bf336>] worker_thread+0x126/0x3c0
[<ffffffffb82bf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb82c61f1>] kthread+0xd1/0xe0
[<ffffffffb82c6120>] ? insert_kthread_work+0x40/0x40
[<ffffffffb898cd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb82c6120>] ? insert_kthread_work+0x40/0x40
Complete report #442932
Frame # Function Binary Source or offset Line
1
scsi_request_fn
vmlinux 0xffffffffa24dabcc
Build id: 3.10.0-957.12.2.el7.x86_64 		0x4cc
2
__blk_run_queue
vmlinux 0xffffffffa2343789
Build id: 3.10.0-957.12.2.el7.x86_64 		0x39
3
queue_unplugged
vmlinux 0xffffffffa234384a
Build id: 3.10.0-957.12.2.el7.x86_64 		0x2a
4
blk_flush_plug_list
vmlinux 0xffffffffa2347078
Build id: 3.10.0-957.12.2.el7.x86_64 		0x1d8
5
blk_finish_plug
vmlinux 0xffffffffa23474e4
Build id: 3.10.0-957.12.2.el7.x86_64 		0x14
6
_xfs_buf_ioapply
xfs 0xffffffffc02d4454
Build id: 3.10.0-957.12.2.el7.x86_64 		0x334
7
? xlog_bdstrat
xfs 0xffffffffc02f6acb
Build id: 3.10.0-957.12.2.el7.x86_64 		0x2b
8
xfs_buf_submit
xfs 0xffffffffc02d5fcc
Build id: 3.10.0-957.12.2.el7.x86_64 		0x6c
9
xlog_bdstrat
xfs 0xffffffffc02f6acb
Build id: 3.10.0-957.12.2.el7.x86_64 		0x2b
10
xlog_sync
xfs 0xffffffffc02f8956
Build id: 3.10.0-957.12.2.el7.x86_64 		0x2e6
11
xlog_state_release_iclog
xfs 0xffffffffc02f8adb
Build id: 3.10.0-957.12.2.el7.x86_64 		0x7b
12
_xfs_log_force
xfs 0xffffffffc02f9c33
Build id: 3.10.0-957.12.2.el7.x86_64 		0x233
13
? __switch_to
vmlinux 0xffffffffa202a59e
Build id: 3.10.0-957.12.2.el7.x86_64 		0xce
14
? xfs_log_worker
xfs 0xffffffffc02f9d66
Build id: 3.10.0-957.12.2.el7.x86_64 		0x36
15
xfs_log_force
xfs 0xffffffffc02f9cec
Build id: 3.10.0-957.12.2.el7.x86_64 		0x2c
16
xfs_log_worker
xfs 0xffffffffc02f9d66
Build id: 3.10.0-957.12.2.el7.x86_64 		0x36
17
process_one_work
vmlinux 0xffffffffa20b9e3f
Build id: 3.10.0-957.12.2.el7.x86_64 		0x17f
18
worker_thread
vmlinux 0xffffffffa20baed6
Build id: 3.10.0-957.12.2.el7.x86_64 		0x126
19
? manage_workers.isra.25
vmlinux 0xffffffffa20badb0
Build id: 3.10.0-957.12.2.el7.x86_64 		0x2a0
20
kthread
vmlinux 0xffffffffa20c1d21
Build id: 3.10.0-957.12.2.el7.x86_64 		0xd1
21
? insert_kthread_work
vmlinux 0xffffffffa20c1c50
Build id: 3.10.0-957.12.2.el7.x86_64 		0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffa2775c37
Build id: 3.10.0-957.12.2.el7.x86_64 		0x21
23
? insert_kthread_work
vmlinux 0xffffffffa20c1c50
Build id: 3.10.0-957.12.2.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 28s! [kworker/u128:4:61856]
[<ffffffffa24dabcc>] scsi_request_fn+0x4cc/0x680
[<ffffffffa2343789>] __blk_run_queue+0x39/0x50
[<ffffffffa234384a>] queue_unplugged+0x2a/0xa0
[<ffffffffa2347078>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa23474e4>] blk_finish_plug+0x14/0x40
[<ffffffffc02d4454>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc02f6acb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc02d5fcc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc02f6acb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc02f8956>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc02f8adb>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc02f9c33>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffffa202a59e>] ? __switch_to+0xce/0x580
[<ffffffffc02f9d66>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc02f9cec>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc02f9d66>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffa20b9e3f>] process_one_work+0x17f/0x440
[<ffffffffa20baed6>] worker_thread+0x126/0x3c0
[<ffffffffa20badb0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa20c1d21>] kthread+0xd1/0xe0
[<ffffffffa20c1c50>] ? insert_kthread_work+0x40/0x40
[<ffffffffa2775c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa20c1c50>] ? insert_kthread_work+0x40/0x40
Complete report #354378
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01883be
Build id: 3.10.0-862.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc01a371e
Build id: 3.10.0-862.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc01b82e0
Build id: 3.10.0-862.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8229b050
Build id: 3.10.0-862.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff822a45bc
Build id: 3.10.0-862.el7.x86_64 		0x4cc
6
__blk_run_queue
vmlinux 0xffffffff82118ad9
Build id: 3.10.0-862.el7.x86_64 		0x39
7
queue_unplugged
vmlinux 0xffffffff82118b9a
Build id: 3.10.0-862.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8211c358
Build id: 3.10.0-862.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8211c7c4
Build id: 3.10.0-862.el7.x86_64 		0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc03d3224
Build id: 3.10.0-862.el7.x86_64 		0x334
11
? xlog_bdstrat
xfs 0xffffffffc03f4cbb
Build id: 3.10.0-862.el7.x86_64 		0x2b
12
xfs_buf_submit
xfs 0xffffffffc03d4abc
Build id: 3.10.0-862.el7.x86_64 		0x6c
13
xlog_bdstrat
xfs 0xffffffffc03f4cbb
Build id: 3.10.0-862.el7.x86_64 		0x2b
14
xlog_sync
xfs 0xffffffffc03f6bae
Build id: 3.10.0-862.el7.x86_64 		0x2fe
15
xlog_state_release_iclog
xfs 0xffffffffc03f6d4b
Build id: 3.10.0-862.el7.x86_64 		0x7b
16
_xfs_log_force
xfs 0xffffffffc03f7ea3
Build id: 3.10.0-862.el7.x86_64 		0x233
17
? __switch_to
vmlinux 0xffffffff81e2959e
Build id: 3.10.0-862.el7.x86_64 		0xce
18
? xfs_log_worker
xfs 0xffffffffc03f7fd6
Build id: 3.10.0-862.el7.x86_64 		0x36
19
xfs_log_force
xfs 0xffffffffc03f7f5c
Build id: 3.10.0-862.el7.x86_64 		0x2c
20
xfs_log_worker
xfs 0xffffffffc03f7fd6
Build id: 3.10.0-862.el7.x86_64 		0x36
21
process_one_work
vmlinux 0xffffffff81eb2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffff81eb3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
23
? manage_workers.isra.24
vmlinux 0xffffffff81eb39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffff81ebae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffff81ebad60
Build id: 3.10.0-862.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff8251f61d
Build id: 3.10.0-862.el7.x86_64 		0x7
27
? insert_kthread_work
vmlinux 0xffffffff81ebad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 28s! [kworker/1:1:80714]
Modules linked in: can_bcm sctp can_raw can isofs rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack vmhgfs(OE) ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev snd_ens1371 vmw_balloon snd_rawmidi
snd_ac97_codec ac97_bus snd_seq snd_seq_device btusb joydev btrtl snd_pcm pcspkr btbcm btintel bluetooth snd_timer sg i2c_piix4 snd rfkill shpchp nfit vmw_vmci soundcore libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel e1000 mptspi scsi_transport_spi mptscsih serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: stap_caafe05487afccc6d600147f696715_88888]
CPU: 1 PID: 80714 Comm: kworker/1:1 Tainted: G           OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: xfs-sync/dm-2 xfs_log_worker [xfs]
task: ffff9b14025daf70 ti: ffff9b1405cf0000 task.ti: ffff9b1405cf0000
RIP: 0010:[<ffffffffc01883be>]  [<ffffffffc01883be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9b1405cf3998  EFLAGS: 00000246
RAX: ffffa98440240000 RBX: ffff9b140339a200 RCX: 000000000000001c
RDX: ffff9b143500e400 RSI: ffff9b1435b4f000 RDI: 000000000000000e
RBP: ffff9b1405cf39a8 R08: 0000000000000002 R09: 00000000386da9ff
R10: ffff9b1435be10d8 R11: ffff9b143470ddc0 R12: ffff9b14259eb480
R13: ffff9b1405cf390c R14: ffffffff821212cb R15: ffff9b1405cf3950
FS:  0000000000000000(0000) GS:ffff9b143c640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f394f8af000 CR3: 0000000002d1a000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc01a371e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01b82e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8229b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff822a45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff82118ad9>] __blk_run_queue+0x39/0x50
[<ffffffff82118b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff8211c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8211c7c4>] blk_finish_plug+0x14/0x40
[<ffffffffc03d3224>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc03f4cbb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc03d4abc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc03f4cbb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc03f6bae>] xlog_sync+0x2fe/0x420 [xfs]
[<ffffffffc03f6d4b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc03f7ea3>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffff81e2959e>] ? __switch_to+0xce/0x580
[<ffffffffc03f7fd6>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc03f7f5c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc03f7fd6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff81eb2dff>] process_one_work+0x17f/0x440
[<ffffffff81eb3ac6>] worker_thread+0x126/0x3c0
[<ffffffff81eb39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff81ebae31>] kthread+0xd1/0xe0
[<ffffffff81ebad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff8251f61d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff81ebad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 19
Complete report #302940
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb9b1b2c5
Build id: 3.10.0-862.11.6.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc00ce622
Build id: 3.10.0-862.11.6.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb98a8500
Build id: 3.10.0-862.11.6.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb98a0010
Build id: 3.10.0-862.11.6.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffb98a957c
Build id: 3.10.0-862.11.6.el7.x86_64 		0x4cc
6
__blk_run_queue
vmlinux 0xffffffffb971d4e9
Build id: 3.10.0-862.11.6.el7.x86_64 		0x39
7
queue_unplugged
vmlinux 0xffffffffb971d5aa
Build id: 3.10.0-862.11.6.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb9720d68
Build id: 3.10.0-862.11.6.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb97211d4
Build id: 3.10.0-862.11.6.el7.x86_64 		0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0310224
Build id: 3.10.0-862.11.6.el7.x86_64 		0x334
11
? xlog_bdstrat
xfs 0xffffffffc0331cab
Build id: 3.10.0-862.11.6.el7.x86_64 		0x2b
12
xfs_buf_submit
xfs 0xffffffffc0311abc
Build id: 3.10.0-862.11.6.el7.x86_64 		0x6c
13
xlog_bdstrat
xfs 0xffffffffc0331cab
Build id: 3.10.0-862.11.6.el7.x86_64 		0x2b
14
xlog_sync
xfs 0xffffffffc0333b9e
Build id: 3.10.0-862.11.6.el7.x86_64 		0x2fe
15
xlog_state_release_iclog
xfs 0xffffffffc0333d3b
Build id: 3.10.0-862.11.6.el7.x86_64 		0x7b
16
_xfs_log_force
xfs 0xffffffffc0334e93
Build id: 3.10.0-862.11.6.el7.x86_64 		0x233
17
? add_timer
vmlinux 0xffffffffb94a7768
Build id: 3.10.0-862.11.6.el7.x86_64 		0x18
18
? xfs_log_worker
xfs 0xffffffffc0334fc6
Build id: 3.10.0-862.11.6.el7.x86_64 		0x36
19
xfs_log_force
xfs 0xffffffffc0334f4c
Build id: 3.10.0-862.11.6.el7.x86_64 		0x2c
20
xfs_log_worker
xfs 0xffffffffc0334fc6
Build id: 3.10.0-862.11.6.el7.x86_64 		0x36
21
process_one_work
vmlinux 0xffffffffb94b613f
Build id: 3.10.0-862.11.6.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffffb94b71d6
Build id: 3.10.0-862.11.6.el7.x86_64 		0x126
23
? manage_workers.isra.24
vmlinux 0xffffffffb94b70b0
Build id: 3.10.0-862.11.6.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffffb94bdf21
Build id: 3.10.0-862.11.6.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffffb94bde50
Build id: 3.10.0-862.11.6.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffb9b255dd
Build id: 3.10.0-862.11.6.el7.x86_64 		0x7
27
? insert_kthread_work
vmlinux 0xffffffffb94bde50
Build id: 3.10.0-862.11.6.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 21s! [kworker/9:31:21427]
Modules linked in: veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_1_9_8557(POE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg vmw_balloon
ppdev joydev parport_pc parport i2c_piix4 vmw_vmci shpchp auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops sd_mod ttm crc_t10dif crct10dif_generic drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw i2c_core vmxnet3 vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 21427 Comm: kworker/9:31 Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff92f38f3daf70 ti: ffff92f3f03d8000 task.ti: ffff92f3f03d8000
RIP: 0010:[<ffffffffb9b1b2c5>]  [<ffffffffb9b1b2c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff92f3f03dba00  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffffb97795b3 RCX: ffff92e7ffe58060
RDX: ffffa6f84ce20000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff92f3f03dba00 R08: 0000000000000000 R09: ffff92f3a7f76100
R10: ffff92e7cb2c68d8 R11: ffff92e7cb299000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff92e3eba7d200 R15: ffffffffb98a7259
FS:  0000000000000000(0000) GS:ffff92f44d400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c420191010 CR3: 0000000c85702000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc00ce622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffb98a8500>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffb98a0010>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb98a957c>] scsi_request_fn+0x4cc/0x680
[<ffffffffb971d4e9>] __blk_run_queue+0x39/0x50
[<ffffffffb971d5aa>] queue_unplugged+0x2a/0xa0
[<ffffffffb9720d68>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb97211d4>] blk_finish_plug+0x14/0x40
[<ffffffffc0310224>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0331cab>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc0311abc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc0331cab>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc0333b9e>] xlog_sync+0x2fe/0x420 [xfs]
[<ffffffffc0333d3b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc0334e93>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffffb94a7768>] ? add_timer+0x18/0x20
[<ffffffffc0334fc6>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc0334f4c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc0334fc6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffb94b613f>] process_one_work+0x17f/0x440
[<ffffffffb94b71d6>] worker_thread+0x126/0x3c0
[<ffffffffb94b70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffb94bdf21>] kthread+0xd1/0xe0
[<ffffffffb94bde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb9b255dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffb94bde50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #16501
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffaf36a175
Build id: 3.10.0-957.1.3.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc04cc622
Build id: 3.10.0-957.1.3.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffaf0d9100
Build id: 3.10.0-957.1.3.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffaf0d0bd0
Build id: 3.10.0-957.1.3.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffaf0da16c
Build id: 3.10.0-957.1.3.el7.x86_64 		0x4cc
6
__blk_run_queue
vmlinux 0xffffffffaef43269
Build id: 3.10.0-957.1.3.el7.x86_64 		0x39
7
queue_unplugged
vmlinux 0xffffffffaef4332a
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffaef46b58
Build id: 3.10.0-957.1.3.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffffaef46fc4
Build id: 3.10.0-957.1.3.el7.x86_64 		0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc068d454
Build id: 3.10.0-957.1.3.el7.x86_64 		0x334
11
? xlog_bdstrat
xfs 0xffffffffc06afacb
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2b
12
xfs_buf_submit
xfs 0xffffffffc068efcc
Build id: 3.10.0-957.1.3.el7.x86_64 		0x6c
13
xlog_bdstrat
xfs 0xffffffffc06afacb
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2b
14
xlog_sync
xfs 0xffffffffc06b1956
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc06b1adb
Build id: 3.10.0-957.1.3.el7.x86_64 		0x7b
16
_xfs_log_force
xfs 0xffffffffc06b2c33
Build id: 3.10.0-957.1.3.el7.x86_64 		0x233
17
? __switch_to
vmlinux 0xffffffffaec2a59e
Build id: 3.10.0-957.1.3.el7.x86_64 		0xce
18
? xfs_log_worker
xfs 0xffffffffc06b2d66
Build id: 3.10.0-957.1.3.el7.x86_64 		0x36
19
xfs_log_force
xfs 0xffffffffc06b2cec
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2c
20
xfs_log_worker
xfs 0xffffffffc06b2d66
Build id: 3.10.0-957.1.3.el7.x86_64 		0x36
21
process_one_work
vmlinux 0xffffffffaecb9d4f
Build id: 3.10.0-957.1.3.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffffaecbade6
Build id: 3.10.0-957.1.3.el7.x86_64 		0x126
23
? manage_workers.isra.25
vmlinux 0xffffffffaecbacc0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffffaecc1c31
Build id: 3.10.0-957.1.3.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffffaecc1b60
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffffaf374c1d
Build id: 3.10.0-957.1.3.el7.x86_64 		0x7
27
? insert_kthread_work
vmlinux 0xffffffffaecc1b60
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [kworker/0:0:28455]
Modules linked in: arc4 md4 nls_utf8 cifs ccm dns_resolver binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ahci vmw_pvscsi libahci drm_panel_orientation_quirks
ata_generic pata_acpi ata_piix nfit libata libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 28455 Comm: kworker/0:0 Kdump: loaded Tainted: G             L ------------   3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Workqueue: xfs-sync/dm-1 xfs_log_worker [xfs]
task: ffff91a5e1900000 ti: ffff91a883484000 task.ti: ffff91a883484000
RIP: 0010:[<ffffffffaf36a175>]  [<ffffffffaf36a175>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff91a883487a00  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffffaefa1363 RCX: ffff91a8bfca3060
RDX: ffffb12181b80000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff91a883487a00 R08: 0000000000000000 R09: ffff91a84fc96100
R10: ffff91a8aa1fb0d8 R11: ffff91a8aa1fb800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff91a485efe000 R15: ffffffffaf0d7e59
FS:  0000000000000000(0000) GS:ffff91a8bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe3b70af100 CR3: 000000041d704000 CR4: 00000000007607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffc04cc622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffaf0d9100>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffaf0d0bd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffaf0da16c>] scsi_request_fn+0x4cc/0x680
[<ffffffffaef43269>] __blk_run_queue+0x39/0x50
[<ffffffffaef4332a>] queue_unplugged+0x2a/0xa0
[<ffffffffaef46b58>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffaef46fc4>] blk_finish_plug+0x14/0x40
[<ffffffffc068d454>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc06afacb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc068efcc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc06afacb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc06b1956>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc06b1adb>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc06b2c33>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffffaec2a59e>] ? __switch_to+0xce/0x580
[<ffffffffc06b2d66>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc06b2cec>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc06b2d66>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffaecb9d4f>] process_one_work+0x17f/0x440
[<ffffffffaecbade6>] worker_thread+0x126/0x3c0
[<ffffffffaecbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffaecc1c31>] kthread+0xd1/0xe0
[<ffffffffaecc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf374c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffaecc1b60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #14939
Frame # Function Binary Source or offset Line
1
scsi_request_fn
vmlinux 0xffffffffb44a4225
Build id: 3.10.0-862.el7.x86_64 		0x135
2
__blk_run_queue
vmlinux 0xffffffffb4318ad9
Build id: 3.10.0-862.el7.x86_64 		0x39
3
queue_unplugged
vmlinux 0xffffffffb4318b9a
Build id: 3.10.0-862.el7.x86_64 		0x2a
4
blk_flush_plug_list
vmlinux 0xffffffffb431c358
Build id: 3.10.0-862.el7.x86_64 		0x1d8
5
blk_finish_plug
vmlinux 0xffffffffb431c7c4
Build id: 3.10.0-862.el7.x86_64 		0x14
6
_xfs_buf_ioapply
xfs 0xffffffffc02a7224
Build id: 3.10.0-862.el7.x86_64 		0x334
7
? xlog_bdstrat
xfs 0xffffffffc02c8cbb
Build id: 3.10.0-862.el7.x86_64 		0x2b
8
xfs_buf_submit
xfs 0xffffffffc02a8abc
Build id: 3.10.0-862.el7.x86_64 		0x6c
9
xlog_bdstrat
xfs 0xffffffffc02c8cbb
Build id: 3.10.0-862.el7.x86_64 		0x2b
10
xlog_sync
xfs 0xffffffffc02cabae
Build id: 3.10.0-862.el7.x86_64 		0x2fe
11
xlog_state_release_iclog
xfs 0xffffffffc02cad4b
Build id: 3.10.0-862.el7.x86_64 		0x7b
12
_xfs_log_force
xfs 0xffffffffc02cbea3
Build id: 3.10.0-862.el7.x86_64 		0x233
13
? add_timer
vmlinux 0xffffffffb40a4558
Build id: 3.10.0-862.el7.x86_64 		0x18
14
? xfs_log_worker
xfs 0xffffffffc02cbfd6
Build id: 3.10.0-862.el7.x86_64 		0x36
15
xfs_log_force
xfs 0xffffffffc02cbf5c
Build id: 3.10.0-862.el7.x86_64 		0x2c
16
xfs_log_worker
xfs 0xffffffffc02cbfd6
Build id: 3.10.0-862.el7.x86_64 		0x36
17
process_one_work
vmlinux 0xffffffffb40b2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
18
worker_thread
vmlinux 0xffffffffb40b3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
19
? manage_workers.isra.24
vmlinux 0xffffffffb40b39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
20
kthread
vmlinux 0xffffffffb40bae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
21
? insert_kthread_work
vmlinux 0xffffffffb40bad60
Build id: 3.10.0-862.el7.x86_64 		0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffb471f637
Build id: 3.10.0-862.el7.x86_64 		0x21
23
? insert_kthread_work
vmlinux 0xffffffffb40bad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 55s! [kworker/0:0:25409]
Modules linked in: ext4 mbcache jbd2 rbd libceph dns_resolver ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 veth xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipport ip_set_hash_ipportip ip_set dummy xt_comment xt_mark ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw
ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg nfit vmw_vmci shpchp i2c_piix4 libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sd_mod sysfillrect sysimgblt crc_t10dif fb_sys_fops ahci crct10dif_generic ttm ata_piix libahci drm crct10dif_pclmul crct10dif_common serio_raw libata crc32c_intel vmxnet3 i2c_core vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 25409 Comm: kworker/0:0 Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff967a1fe28000 ti: ffff967a35bd4000 task.ti: ffff967a35bd4000
RIP: 0010:[<ffffffffb44a4225>]  [<ffffffffb44a4225>] scsi_request_fn+0x135/0x680
RSP: 0018:ffff967a35bd7ab0  EFLAGS: 00000246
RAX: 000000010114fd00 RBX: ffff9677afed0180 RCX: 000000010114fb8d
RDX: 000000010114e805 RSI: ffff9677b547d158 RDI: ffff9677b547d010
RBP: ffff967a35bd7b08 R08: ffff9677b547d158 R09: 0000000000000000
R10: ffff967aadee5c00 R11: ffff9677e44e8c40 R12: ffff967a35bd7a40
R13: ffffffffc01b0d3f R14: ffff967a35bd7a20 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff967aadc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00286d000 CR3: 0000000315f40000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb4318ad9>] __blk_run_queue+0x39/0x50
[<ffffffffb4318b9a>] queue_unplugged+0x2a/0xa0
[<ffffffffb431c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb431c7c4>] blk_finish_plug+0x14/0x40
[<ffffffffc02a7224>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc02c8cbb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc02a8abc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc02c8cbb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc02cabae>] xlog_sync+0x2fe/0x420 [xfs]
[<ffffffffc02cad4b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc02cbea3>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffffb40a4558>] ? add_timer+0x18/0x20
[<ffffffffc02cbfd6>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc02cbf5c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc02cbfd6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffb40b2dff>] process_one_work+0x17f/0x440
[<ffffffffb40b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffb40b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffb40bae31>] kthread+0xd1/0xe0
[<ffffffffb40bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffb471f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb40bad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 85 40 04 00 00 a8 02 0f 85 18 01 00 00 4c 89 ff e8 10 7a e7 ff 49 8b bd 58 04 00 00 c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <4d> 8b b7 00 01 00 00 4d 85 f6 0f 84 5d 04 00 00 49 8b 85 40 04
Complete report #14625
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8a183815
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc021a632
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff89ee72d0
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff89edecd0
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff89ee833c
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x4cc
6
__blk_run_queue
vmlinux 0xffffffff89d4f289
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x39
7
queue_unplugged
vmlinux 0xffffffff89d4f34a
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff89d52b98
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffff89d53004
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0479044
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x334
11
? xfs_buf_submit
xfs 0xffffffffc047ab4a
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x1a
12
__xfs_buf_submit
xfs 0xffffffffc047a8c1
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x61
13
xfs_buf_submit
xfs 0xffffffffc047ab4a
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x1a
14
xlog_bdstrat
xfs 0xffffffffc049b0ab
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x2b
15
xlog_sync
xfs 0xffffffffc049cea6
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x2e6
16
xlog_state_release_iclog
xfs 0xffffffffc049d02b
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x7b
17
_xfs_log_force
xfs 0xffffffffc049e17b
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x22b
18
? __switch_to
vmlinux 0xffffffff89a2b59e
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xce
19
? xfs_log_worker
xfs 0xffffffffc049e296
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x36
20
xfs_log_force
xfs 0xffffffffc049e21c
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x2c
21
xfs_log_worker
xfs 0xffffffffc049e296
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x36
22
process_one_work
vmlinux 0xffffffff89abe34f
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x17f
23
worker_thread
vmlinux 0xffffffff89abf466
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x126
24
? manage_workers.isra.26
vmlinux 0xffffffff89abf340
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x2a0
25
kthread
vmlinux 0xffffffff89ac6321
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xd1
26
? insert_kthread_work
vmlinux 0xffffffff89ac6250
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x40
27
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a18dd37
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x21
28
? insert_kthread_work
vmlinux 0xffffffff89ac6250
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 133s! [kworker/2:1:9330]
Modules linked in: iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock ppdev sb_edac vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmxnet3 vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif syscopyarea crct10dif_generic sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata nfit serio_raw libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 9330 Comm: kworker/2:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: xfs-sync/sda5 xfs_log_worker [xfs]
task: ffff90ebc5dab150 ti: ffff90efe9408000 task.ti: ffff90efe9408000
RIP: 0010:[<ffffffff8a183815>]  [<ffffffff8a183815>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff90efe940b9e8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff90efe940b9a8 RCX: ffff90f97f121060
RDX: ffffad41c69d0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff90efe940b9e8 R08: 0000000000000000 R09: ffff90f96fc57e00
R10: ffff90f9bfc790d8 R11: ffff90f97a074800 R12: 0000000000000020
R13: ffff90f97110e480 R14: ffffffff89ee6029 R15: ffff90efe940b960
FS:  0000000000000000(0000) GS:ffff90f9bfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcebc000008 CR3: 0000000ffcb4e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc021a632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff89ee72d0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff89edecd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff89ee833c>] scsi_request_fn+0x4cc/0x680
[<ffffffff89d4f289>] __blk_run_queue+0x39/0x50
[<ffffffff89d4f34a>] queue_unplugged+0x2a/0xa0
[<ffffffff89d52b98>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff89d53004>] blk_finish_plug+0x14/0x40
[<ffffffffc0479044>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc047ab4a>] ? xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc047a8c1>] __xfs_buf_submit+0x61/0x210 [xfs]
[<ffffffffc047ab4a>] xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc049b0ab>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc049cea6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc049d02b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc049e17b>] _xfs_log_force+0x22b/0x2a0 [xfs]
[<ffffffff89a2b59e>] ? __switch_to+0xce/0x580
[<ffffffffc049e296>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc049e21c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc049e296>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff89abe34f>] process_one_work+0x17f/0x440
[<ffffffff89abf466>] worker_thread+0x126/0x3c0
[<ffffffff89abf340>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff89ac6321>] kthread+0xd1/0xe0
[<ffffffff89ac6250>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a18dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff89ac6250>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #3649
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01cb3be
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02b171e
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03352e0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa70deab0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffffa70e811c
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x4cc
6
__blk_run_queue
vmlinux 0xffffffffa6f4f079
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x39
7
queue_unplugged
vmlinux 0xffffffffa6f4f13a
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffa6f52988
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffffa6f52df4
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0467044
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x334
11
? xfs_buf_submit
xfs 0xffffffffc0468b4a
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x1a
12
__xfs_buf_submit
xfs 0xffffffffc04688c1
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x61
13
xfs_buf_submit
xfs 0xffffffffc0468b4a
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x1a
14
xlog_bdstrat
xfs 0xffffffffc04890ab
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2b
15
xlog_sync
xfs 0xffffffffc048aea6
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2e6
16
xlog_state_release_iclog
xfs 0xffffffffc048b02b
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x7b
17
_xfs_log_force
xfs 0xffffffffc048c17b
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22b
18
? add_timer
vmlinux 0xffffffffa6caf288
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x18
19
? xfs_log_worker
xfs 0xffffffffc048c296
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x36
20
xfs_log_force
xfs 0xffffffffc048c21c
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2c
21
xfs_log_worker
xfs 0xffffffffc048c296
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x36
22
process_one_work
vmlinux 0xffffffffa6cbe21f
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x17f
23
worker_thread
vmlinux 0xffffffffa6cbf336
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x126
24
? manage_workers.isra.26
vmlinux 0xffffffffa6cbf210
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2a0
25
kthread
vmlinux 0xffffffffa6cc61f1
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xd1
26
? insert_kthread_work
vmlinux 0xffffffffa6cc6120
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x40
27
ret_from_fork_nospec_begin
vmlinux 0xffffffffa738dd37
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x21
28
? insert_kthread_work
vmlinux 0xffffffffa6cc6120
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 27s! [kworker/0:3:81409]
Modules linked in: xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables veth tcp_lp fuse br_netfilter tun devlink overlay(T) bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vmw_vmci vsock sunrpc snd_seq_midi snd_seq_midi_event coretemp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus aesni_intel snd_seq lrw gf128mul glue_helper snd_seq_device snd_pcm ablk_helper cryptd snd_timer pcspkr joydev snd sg soundcore parport_pc parport pcc_cpufreq i2c_piix4 xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi serio_raw drm mptscsih ata_piix e1000 nfit libnvdimm libata mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 81409 Comm: kworker/0:3 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff96b30f6b20e0 ti: ffff96b2e6854000 task.ti: ffff96b2e6854000
RIP: 0010:[<ffffffffc01cb3be>]  [<ffffffffc01cb3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff96b2e6857980  EFLAGS: 00000246
RAX: ffffad5f806c0000 RBX: ffff96b311694280 RCX: 0000000000000000
RDX: ffff96b33c26e600 RSI: ffff96b2b4a21000 RDI: 000000000000000e
RBP: ffff96b2e6857990 R08: 0000000000000003 R09: 00000000b4700dff
R10: ffff96b2b591f0d8 R11: ffff96b328ab3b80 R12: 0000000100000005
R13: ffff96b2deba7480 R14: ffff96b2e68578f4 R15: ffffffffa6f578db
FS:  0000000000000000(0000) GS:ffff96b33a600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f871e400190 CR3: 0000000094e8a000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc02b171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03352e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa70deab0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa70e811c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa6f4f079>] __blk_run_queue+0x39/0x50
[<ffffffffa6f4f13a>] queue_unplugged+0x2a/0xa0
[<ffffffffa6f52988>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa6f52df4>] blk_finish_plug+0x14/0x40
[<ffffffffc0467044>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0468b4a>] ? xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc04688c1>] __xfs_buf_submit+0x61/0x210 [xfs]
[<ffffffffc0468b4a>] xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc04890ab>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc048aea6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc048b02b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc048c17b>] _xfs_log_force+0x22b/0x2a0 [xfs]
[<ffffffffa6caf288>] ? add_timer+0x18/0x20
[<ffffffffc048c296>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc048c21c>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc048c296>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffa6cbe21f>] process_one_work+0x17f/0x440
[<ffffffffa6cbf336>] worker_thread+0x126/0x3c0
[<ffffffffa6cbf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa6cc61f1>] kthread+0xd1/0xe0
[<ffffffffa6cc6120>] ? insert_kthread_work+0x40/0x40
[<ffffffffa738dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa6cc6120>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 93 1d
Complete report #979
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04203be
Build id: 3.10.0-957.12.1.el7.x86_64 		0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc044971e
Build id: 3.10.0-957.12.1.el7.x86_64 		0x24e
3
mptspi_qcmd
mptspi 0xffffffffc043b2e0
Build id: 3.10.0-957.12.1.el7.x86_64 		0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9d6d14f0
Build id: 3.10.0-957.12.1.el7.x86_64 		0xb0
5
scsi_request_fn
vmlinux 0xffffffff9d6daa8c
Build id: 3.10.0-957.12.1.el7.x86_64 		0x4cc
6
__blk_run_queue
vmlinux 0xffffffff9d543679
Build id: 3.10.0-957.12.1.el7.x86_64 		0x39
7
queue_unplugged
vmlinux 0xffffffff9d54373a
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff9d546f68
Build id: 3.10.0-957.12.1.el7.x86_64 		0x1d8
9
blk_finish_plug
vmlinux 0xffffffff9d5473d4
Build id: 3.10.0-957.12.1.el7.x86_64 		0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0688454
Build id: 3.10.0-957.12.1.el7.x86_64 		0x334
11
? xlog_bdstrat
xfs 0xffffffffc06aaacb
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2b
12
xfs_buf_submit
xfs 0xffffffffc0689fcc
Build id: 3.10.0-957.12.1.el7.x86_64 		0x6c
13
xlog_bdstrat
xfs 0xffffffffc06aaacb
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2b
14
xlog_sync
xfs 0xffffffffc06ac956
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc06acadb
Build id: 3.10.0-957.12.1.el7.x86_64 		0x7b
16
_xfs_log_force
xfs 0xffffffffc06adc33
Build id: 3.10.0-957.12.1.el7.x86_64 		0x233
17
? __switch_to
vmlinux 0xffffffff9d22a59e
Build id: 3.10.0-957.12.1.el7.x86_64 		0xce
18
? xfs_log_worker
xfs 0xffffffffc06add66
Build id: 3.10.0-957.12.1.el7.x86_64 		0x36
19
xfs_log_force
xfs 0xffffffffc06adcec
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2c
20
xfs_log_worker
xfs 0xffffffffc06add66
Build id: 3.10.0-957.12.1.el7.x86_64 		0x36
21
process_one_work
vmlinux 0xffffffff9d2b9d8f
Build id: 3.10.0-957.12.1.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffff9d2bae26
Build id: 3.10.0-957.12.1.el7.x86_64 		0x126
23
? manage_workers.isra.25
vmlinux 0xffffffff9d2bad00
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffff9d2c1c71
Build id: 3.10.0-957.12.1.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffff9d2c1ba0
Build id: 3.10.0-957.12.1.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff9d975bf7
Build id: 3.10.0-957.12.1.el7.x86_64 		0x21
27
? insert_kthread_work
vmlinux 0xffffffff9d2c1ba0
Build id: 3.10.0-957.12.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 31s! [kworker/1:0:21987]
Modules linked in: xt_nat veth fuse nf_conntrack_netlink nfnetlink xt_addrtype xt_CHECKSUM br_netfilter iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc ppdev iosf_mbi kvm_intel kvm vmw_balloon irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio parport_pc parport sg vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic
pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm e1000 drm ata_piix libata mptspi scsi_transport_spi mptscsih nfit mptbase drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 21987 Comm: kworker/1:0 Tainted: G             L ------------   3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffffa0e9e7169040 ti: ffffa0eb0d290000 task.ti: ffffa0eb0d290000
RIP: 0010:[<ffffffffc04203be>]  [<ffffffffc04203be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffffa0eb0d293998  EFLAGS: 00000246
RAX: ffffb5f780cc0000 RBX: ffffa0e974e87020 RCX: 0000000000000008
RDX: ffffa0eb70fece00 RSI: ffffa0eb71522000 RDI: 000000000000000e
RBP: ffffa0eb0d2939a8 R08: 0000000000000003 R09: 000000022f425fff
R10: ffffa0eb715810d8 R11: ffffa0eb20764380 R12: ffffa0e979cf7780
R13: ffffa0eb0d29390c R14: ffffffff9d54bebb R15: ffffa0eb0d293950
FS:  0000000000000000(0000) GS:ffffa0eb75640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0ac0975000 CR3: 000000011a410000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc044971e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc043b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9d6d14f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9d6daa8c>] scsi_request_fn+0x4cc/0x680
[<ffffffff9d543679>] __blk_run_queue+0x39/0x50
[<ffffffff9d54373a>] queue_unplugged+0x2a/0xa0
[<ffffffff9d546f68>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff9d5473d4>] blk_finish_plug+0x14/0x40
[<ffffffffc0688454>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc06aaacb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc0689fcc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc06aaacb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc06ac956>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc06acadb>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc06adc33>] _xfs_log_force+0x233/0x2c0 [xfs]
[<ffffffff9d22a59e>] ? __switch_to+0xce/0x580
[<ffffffffc06add66>] ? xfs_log_worker+0x36/0x100 [xfs]
[<ffffffffc06adcec>] xfs_log_force+0x2c/0x70 [xfs]
[<ffffffffc06add66>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff9d2b9d8f>] process_one_work+0x17f/0x440
[<ffffffff9d2bae26>] worker_thread+0x126/0x3c0
[<ffffffff9d2bad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff9d2c1c71>] kthread+0xd1/0xe0
[<ffffffff9d2c1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9d975bf7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9d2c1ba0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 42