Info

Permalink
Function
mpt_put_msg_frame
First occurrence
2020-09-30
Last occurrence
2026-06-30
State
NEW
Components
kernel
Quality
-52

Statistics

Operating system Count
CentOS 7 1,356
Architecture Count
x86_64 1,356
Related packages Count
kernel 1,356
 0:3.10.0-1160.53.1.el7 197
 0:3.10.0-1160.15.2.el7 162
 0:3.10.0-1127.19.1.el7 111
 0:3.10.0-1160.el7 91
 0:3.10.0-1160.62.1.el7 72
 0:3.10.0-1160.31.1.el7 65
 0:3.10.0-693.11.1.el7 60
 0:3.10.0-1160.42.2.el7 56
 0:3.10.0-1160.88.1.el7 52
 0:3.10.0-957.12.2.el7 49
 0:3.10.0-1160.71.1.el7 49
 0:3.10.0-1160.118.1.el7 34
 0:3.10.0-1160.41.1.el7 33
 0:3.10.0-1160.36.2.el7 32
 0:3.10.0-1160.59.1.el7 31
 0:3.10.0-1160.25.1.el7 28
 0:3.10.0-1127.el7 21
 0:3.10.0-1160.108.1.el7 21
 0:3.10.0-1160.11.1.el7 16
 0:3.10.0-957.el7 14
 0:3.10.0-1160.76.1.el7 13
 0:3.10.0-862.el7 10
 0:3.10.0-1127.18.2.el7 10
 0:3.10.0-1160.66.1.el7 10
 0:3.10.0-1127.10.1.el7 9
 0:3.10.0-1160.24.1.el7 9
 0:3.10.0-1160.49.1.el7 9
 0:3.10.0-1160.119.1.el7 8
 0:3.10.0-1160.6.1.el7 8
 0:3.10.0-1160.21.1.el7 8
 0:3.10.0-1160.80.1.el7 7
 0:3.10.0-1160.102.1.el7 6
 0:3.10.0-1160.83.1.el7 6
 0:3.10.0-1160.99.1.el7 6
 0:3.10.0-1160.95.1.el7 5
 0:3.10.0-1160.81.1.el7 5
 0:3.10.0-1160.45.1.el7 4
 0:3.10.0-514.el7 3
 0:3.10.0-693.el7 3
 0:3.10.0-1160.90.1.el7 3
 0:3.10.0-862.6.3.el7 3
 0:3.10.0-862.14.4.el7 2
 0:3.10.0-957.5.1.el7 2
 0:3.10.0-1062.9.1.el7 2
 0:3.10.0-1160.105.1.el7 2
 0:3.10.0-327.36.1.el7 1
 0:3.10.0-514.10.2.el7 1
 0:3.10.0-693.17.1.el7 1
 0:3.10.0-1127.13.1.el7 1
 0:3.10.0-1127.8.2.el7 1
 0:3.10.0-1160.114.2.el7 1
 0:3.10.0-1160.92.1.el7 1
 0:3.10.0-957.10.1.el7 1
 0:3.10.0-1062.18.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1105122
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04543be
Build id: 3.10.0-1160.36.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc047671e
Build id: 3.10.0-1160.36.2.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc046f2e0
Build id: 3.10.0-1160.36.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff878e4810
Build id: 3.10.0-1160.36.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff878edf3c
Build id: 3.10.0-1160.36.2.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff87753ee9
Build id: 3.10.0-1160.36.2.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff87753faa
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff87757888
Build id: 3.10.0-1160.36.2.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff87757cf4
Build id: 3.10.0-1160.36.2.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc06cf2c4
Build id: 3.10.0-1160.36.2.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc06f16c7
Build id: 3.10.0-1160.36.2.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc06d0c72
Build id: 3.10.0-1160.36.2.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc06f16c7
Build id: 3.10.0-1160.36.2.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc06f34d6
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc06f365b
Build id: 3.10.0-1160.36.2.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc06f47ba
Build id: 3.10.0-1160.36.2.el7.x86_64
0x23a
17
? add_timer
vmlinux 0xffffffff874aeea8
Build id: 3.10.0-1160.36.2.el7.x86_64
0x18
18
xfs_log_worker
xfs 0xffffffffc06f4896
Build id: 3.10.0-1160.36.2.el7.x86_64
0x36
19
process_one_work
vmlinux 0xffffffff874bde6f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff874bef86
Build id: 3.10.0-1160.36.2.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff874bee60
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff874c5e41
Build id: 3.10.0-1160.36.2.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff874c5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff87b95ddd
Build id: 3.10.0-1160.36.2.el7.x86_64
0x7
25
? insert_kthread_work
vmlinux 0xffffffff874c5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/1:2:30517]
Modules linked in: openvswitch xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_17306(E) ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev parport_pc sg parport
i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul sysimgblt crct10dif_common fb_sys_fops crc32c_intel ttm serio_raw ata_piix drm nfit vmxnet3 libnvdimm libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 30517 Comm: kworker/1:2 Tainted: P            EL ------------   3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff9012f100c200 ti: ffff9012e8d54000 task.ti: ffff9012e8d54000
RIP: 0010:[<ffffffffc04543be>]  [<ffffffffc04543be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9012e8d579c8  EFLAGS: 00000246
RAX: ffffaad543600000 RBX: ffff901290b349d0 RCX: 0000000000000014
RDX: ffff9013627fa200 RSI: ffff90134d994000 RDI: 000000000000000e
RBP: ffff9012e8d579d8 R08: 0000000000000003 R09: 00000005dcd79fff
R10: ffff901360b82400 R11: ffff9012e8f208c0 R12: ffff9012cc152300
R13: ffff9012e8d5793c R14: ffffffff8775c8ab R15: ffff9012e8d57980
FS:  0000000000000000(0000) GS:ffff90137fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2a36c5c8c0 CR3: 00000005be514000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffffc047671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc046f2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff878e4810>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff878edf3c>] scsi_request_fn+0x4ac/0x680
[<ffffffff87753ee9>] __blk_run_queue+0x39/0x50
[<ffffffff87753faa>] queue_unplugged+0x2a/0xa0
[<ffffffff87757888>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff87757cf4>] blk_finish_plug+0x14/0x40
[<ffffffffc06cf2c4>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc06f16c7>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc06d0c72>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc06f16c7>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc06f34d6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc06f365b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc06f47ba>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffff874aeea8>] ? add_timer+0x18/0x20
[<ffffffffc06f4896>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff874bde6f>] process_one_work+0x17f/0x440
[<ffffffff874bef86>] worker_thread+0x126/0x3c0
[<ffffffff874bee60>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff874c5e41>] kthread+0xd1/0xe0
[<ffffffff874c5d70>] ? insert_kthread_work+0x40/0x40
[<ffffffff87b95ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff874c5d70>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 46 
Complete report #934223
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02633be
Build id: 3.10.0-957.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc023971e
Build id: 3.10.0-957.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc025c2e0
Build id: 3.10.0-957.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffa0ed0960
Build id: 3.10.0-957.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffa0ed9efc
Build id: 3.10.0-957.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffffa0d42ff9
Build id: 3.10.0-957.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffa0d430ba
Build id: 3.10.0-957.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffa0d468e8
Build id: 3.10.0-957.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffa0d46d54
Build id: 3.10.0-957.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc04ad454
Build id: 3.10.0-957.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc04cfacb
Build id: 3.10.0-957.el7.x86_64
0x2b
12
xfs_buf_submit
xfs 0xffffffffc04aefcc
Build id: 3.10.0-957.el7.x86_64
0x6c
13
xlog_bdstrat
xfs 0xffffffffc04cfacb
Build id: 3.10.0-957.el7.x86_64
0x2b
14
xlog_sync
xfs 0xffffffffc04d1956
Build id: 3.10.0-957.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc04d1adb
Build id: 3.10.0-957.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc04d279a
Build id: 3.10.0-957.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc04d4138
Build id: 3.10.0-957.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc04d42d5
Build id: 3.10.0-957.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffffa0ab9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffa0abaf38
Build id: 3.10.0-957.el7.x86_64
0x278
21
? manage_workers.isra.25
vmlinux 0xffffffffa0abacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffa0ac1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffa0ac1b60
Build id: 3.10.0-957.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa1174c1d
Build id: 3.10.0-957.el7.x86_64
0x7
25
? insert_kthread_work
vmlinux 0xffffffffa0ac1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 50s! [kworker/0:1:123585]
Modules linked in: tcp_lp isofs rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw vmhgfs(OE) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_seq_midi snd_seq_midi_event joydev pcspkr btusb
btrtl btbcm snd_ens1371 btintel bluetooth snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm rfkill snd_timer snd soundcore sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw ata_generic pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi e1000 mptscsih libata mptbase nfit drm_panel_orientation_quirks libnvdimm
CPU: 0 PID: 123585 Comm: kworker/0:1 Tainted: G           OE  ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: xfs-cil/sda3 xlog_cil_push_work [xfs]
task: ffff90b6dbeb4100 ti: ffff90b6f6a08000 task.ti: ffff90b6f6a08000
RIP: 0010:[<ffffffffc02633be>]  [<ffffffffc02633be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff90b6f6a0b8e8  EFLAGS: 00000246
RAX: ffffb95900240000 RBX: ffff90b6d77de370 RCX: 000000000000001e
RDX: ffff90b6f63efa00 RSI: ffff90b6f63e4000 RDI: 000000000000000e
RBP: ffff90b6f6a0b8f8 R08: 0000000000000003 R09: 00000000364aafff
R10: ffff90b6f63e60d8 R11: ffff90b6dbf0c000 R12: ffff90b6e0b5a280
R13: ffff90b6f6a0b85c R14: ffffffffa0d4b83b R15: ffff90b6f6a0b8a0
FS:  0000000000000000(0000) GS:ffff90b6fc600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006dee1c CR3: 000000003a91a000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc023971e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc025c2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa0ed0960>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa0ed9efc>] scsi_request_fn+0x4cc/0x680
[<ffffffffa0d42ff9>] __blk_run_queue+0x39/0x50
[<ffffffffa0d430ba>] queue_unplugged+0x2a/0xa0
[<ffffffffa0d468e8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa0d46d54>] blk_finish_plug+0x14/0x40
[<ffffffffc04ad454>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc04cfacb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc04aefcc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc04cfacb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc04d1956>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc04d1adb>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc04d279a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc04d4138>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc04d42d5>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffffa0ab9d4f>] process_one_work+0x17f/0x440
[<ffffffffa0abaf38>] worker_thread+0x278/0x3c0
[<ffffffffa0abacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa0ac1c31>] kthread+0xd1/0xe0
[<ffffffffa0ac1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa1174c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa0ac1b60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 13 27 
Complete report #371534
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc01fc3be
Build id: 3.10.0-1160.41.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc021771e
Build id: 3.10.0-1160.41.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc01e92e0
Build id: 3.10.0-1160.41.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb8ae4840
Build id: 3.10.0-1160.41.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb8aedf6c
Build id: 3.10.0-1160.41.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffb8953f19
Build id: 3.10.0-1160.41.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb8953fda
Build id: 3.10.0-1160.41.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb89578b8
Build id: 3.10.0-1160.41.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb8957d24
Build id: 3.10.0-1160.41.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc04992c4
Build id: 3.10.0-1160.41.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc04bb6c7
Build id: 3.10.0-1160.41.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc049ac72
Build id: 3.10.0-1160.41.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc04bb6c7
Build id: 3.10.0-1160.41.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc04bd526
Build id: 3.10.0-1160.41.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc04bd6ab
Build id: 3.10.0-1160.41.1.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc04be36a
Build id: 3.10.0-1160.41.1.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc04bfd08
Build id: 3.10.0-1160.41.1.el7.x86_64
0x2a8
18
? add_timer
vmlinux 0xffffffffb86aeec8
Build id: 3.10.0-1160.41.1.el7.x86_64
0x18
19
xlog_cil_push_work
xfs 0xffffffffc04bfea5
Build id: 3.10.0-1160.41.1.el7.x86_64
0x15
20
process_one_work
vmlinux 0xffffffffb86bde8f
Build id: 3.10.0-1160.41.1.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffffb86bf0f8
Build id: 3.10.0-1160.41.1.el7.x86_64
0x278
22
? manage_workers.isra.26
vmlinux 0xffffffffb86bee80
Build id: 3.10.0-1160.41.1.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffffb86c5e61
Build id: 3.10.0-1160.41.1.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffffb86c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffffb8d95df7
Build id: 3.10.0-1160.41.1.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffffb86c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/3:2:56192]
Modules linked in: lp veeamsnap(OE) tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 iosf_mbi crc32_pclmul snd_rawmidi snd_ac97_codec ghash_clmulni_intel aesni_intel ac97_bus lrw snd_seq gf128mul glue_helper ablk_helper snd_seq_device snd_pcm
ppdev cryptd pcspkr snd_timer joydev snd soundcore vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx nfit libnvdimm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix serio_raw mptspi scsi_transport_spi libata e1000 mptscsih mptbase drm_panel_orientation_quirks fuse
CPU: 3 PID: 56192 Comm: kworker/3:2 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: xfs-cil/sda3 xlog_cil_push_work [xfs]
task: ffff944bab82a100 ti: ffff944ab9828000 task.ti: ffff944ab9828000
RIP: 0010:[<ffffffffc01fc3be>]  [<ffffffffc01fc3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff944ab982b8e0  EFLAGS: 00000246
RAX: ffffaeeb40fc0000 RBX: ffff944abac1d8e0 RCX: 0000000000000006
RDX: ffff944bf33eb800 RSI: ffff944bf2c2a000 RDI: 000000000000000e
RBP: ffff944ab982b8f0 R08: 0000000000000003 R09: 00000001b2421fff
R10: ffff944bf2e68800 R11: ffff944bf5495500 R12: 0000000100000008
R13: ffff944bd2496280 R14: ffff944ab982b854 R15: ffffffffb895c8db
FS:  0000000000000000(0000) GS:ffff944bf76c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000a59b0d2e683 CR3: 00000000b9282000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc021771e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01e92e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb8ae4840>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb8aedf6c>] scsi_request_fn+0x4ac/0x680
[<ffffffffb8953f19>] __blk_run_queue+0x39/0x50
[<ffffffffb8953fda>] queue_unplugged+0x2a/0xa0
[<ffffffffb89578b8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb8957d24>] blk_finish_plug+0x14/0x40
[<ffffffffc04992c4>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc04bb6c7>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc049ac72>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc04bb6c7>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc04bd526>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc04bd6ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc04be36a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc04bfd08>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffb86aeec8>] ? add_timer+0x18/0x20
[<ffffffffc04bfea5>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffffb86bde8f>] process_one_work+0x17f/0x440
[<ffffffffb86bf0f8>] worker_thread+0x278/0x3c0
[<ffffffffb86bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb86c5e61>] kthread+0xd1/0xe0
[<ffffffffb86c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffb8d95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb86c5d90>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 a3 20 
Complete report #355516
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00503be
Build id: 3.10.0-693.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc009e729
Build id: 3.10.0-693.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffc006b2b0
Build id: 3.10.0-693.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0287354
Build id: 3.10.0-693.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc02a8d5b
Build id: 3.10.0-693.el7.x86_64
0x2b
12
xfs_buf_submit
xfs 0xffffffffc0288b70
Build id: 3.10.0-693.el7.x86_64
0x70
13
xlog_bdstrat
xfs 0xffffffffc02a8d5b
Build id: 3.10.0-693.el7.x86_64
0x2b
14
xlog_sync
xfs 0xffffffffc02aac7e
Build id: 3.10.0-693.el7.x86_64
0x2fe
15
xlog_state_release_iclog
xfs 0xffffffffc02aae1b
Build id: 3.10.0-693.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc02ab8fa
Build id: 3.10.0-693.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc02ad1d8
Build id: 3.10.0-693.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc02ad375
Build id: 3.10.0-693.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64
0x17a
20
worker_thread
vmlinux 0xffffffff810a9638
Build id: 3.10.0-693.el7.x86_64
0x278
21
? manage_workers.isra.24
vmlinux 0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
0xcf
23
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
24
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
0x58
25
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [kworker/0:1:117229]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 ppdev ghash_clmulni_intel aesni_intel lrw snd_rawmidi snd_ac97_codec gf128mul glue_helper ablk_helper cryptd ac97_bus snd_seq vmw_balloon snd_seq_device snd_pcm snd_timer
pcspkr snd joydev soundcore nfit libnvdimm sg parport_pc parport i2c_piix4 vmw_vmci shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm mptspi scsi_transport_spi ata_piix libata mptscsih e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 117229 Comm: kworker/0:1 Tainted: G        W    L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff8800235d4f10 ti: ffff880064f44000 task.ti: ffff880064f44000
RIP: 0010:[<ffffffffc00503be>]  [<ffffffffc00503be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff880064f478e8  EFLAGS: 00000246
RAX: ffffc900006c0000 RBX: ffff880008537870 RCX: 0000000000000002
RDX: ffff88003599dc00 RSI: ffff8800359c5000 RDI: 000000000000000e
RBP: ffff880064f478f8 R08: 0000000000000002 R09: 0000000089cbdfff
R10: ffff88001eb0e680 R11: ffff88001eb0e680 R12: ffff88000ab68d80
R13: ffff880064f4785c R14: ffffffff812ff13b R15: ffff880064f478a0
FS:  0000000000000000(0000) GS:ffff88008e000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055c27d2fa2c0 CR3: 0000000058a29000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8800359c5d30 0000000000000038 ffff880064f479a0 ffffffffc009e729
ffff8800359c5008 0400000064f47960 0000000000000002 ffff88001eb0e680
0000000000000008 ffff8800359c5188 0000006000000002 ffff8800344028f0
Call Trace:
[<ffffffffc009e729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc006b2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffffc0287354>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc02a8d5b>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc0288b70>] xfs_buf_submit+0x70/0x1f0 [xfs]
[<ffffffffc02a8d5b>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc02aac7e>] xlog_sync+0x2fe/0x420 [xfs]
[<ffffffffc02aae1b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc02ab8fa>] xlog_write+0x5da/0x720 [xfs]
[<ffffffffc02ad1d8>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc02ad375>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a9638>] worker_thread+0x278/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 05 
Complete report #231191
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8b78b565
Build id: 3.10.0-1160.25.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0239632
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8b4ec6b0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8b4e3fa0
Build id: 3.10.0-1160.25.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8b4ed6cc
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff8b353939
Build id: 3.10.0-1160.25.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8b3539fa
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8b357338
Build id: 3.10.0-1160.25.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8b3577a4
Build id: 3.10.0-1160.25.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc04672c4
Build id: 3.10.0-1160.25.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc04896c7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc0468c72
Build id: 3.10.0-1160.25.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc04896c7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc048b4d6
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc048b65b
Build id: 3.10.0-1160.25.1.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc048c31a
Build id: 3.10.0-1160.25.1.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc048dcd8
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc048de75
Build id: 3.10.0-1160.25.1.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffff8b0bddcf
Build id: 3.10.0-1160.25.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff8b0bf038
Build id: 3.10.0-1160.25.1.el7.x86_64
0x278
21
? manage_workers.isra.26
vmlinux 0xffffffff8b0bedc0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff8b0c5da1
Build id: 3.10.0-1160.25.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8b0c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8b795df7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff8b0c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 24s! [kworker/4:1:10186]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg ppdev joydev pcspkr vmw_balloon i2c_piix4 parport_pc parport vmw_vmci binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common e1000e syscopyarea sysfillrect
crc32c_intel serio_raw sysimgblt ahci fb_sys_fops ttm libahci ptp pps_core drm nfit ata_piix vmw_pvscsi libnvdimm libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 10186 Comm: kworker/4:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff94d548265280 ti: ffff94d6cba2c000 task.ti: ffff94d6cba2c000
RIP: 0010:[<ffffffff8b78b565>]  [<ffffffff8b78b565>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff94d6cba2f948  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff94d6cba2f908 RCX: ffffffff8bc34f40
RDX: ffffbcc8036c0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff94d6cba2f948 R08: 0000000000000001 R09: 0000000000000000
R10: ffff94d329fac800 R11: ffff94d9ffca9800 R12: 0000000000000020
R13: ffff94d804d22900 R14: ffffffff8b4eb369 R15: ffff94d6cba2f8c0
FS:  0000000000000000(0000) GS:ffff94d9ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0e62cdd8c0 CR3: 000000016fbd0000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0239632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8b4ec6b0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8b4e3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8b4ed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffff8b353939>] __blk_run_queue+0x39/0x50
[<ffffffff8b3539fa>] queue_unplugged+0x2a/0xa0
[<ffffffff8b357338>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8b3577a4>] blk_finish_plug+0x14/0x40
[<ffffffffc04672c4>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc04896c7>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0468c72>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc04896c7>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc048b4d6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc048b65b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc048c31a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc048dcd8>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc048de75>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffff8b0bddcf>] process_one_work+0x17f/0x440
[<ffffffff8b0bf038>] worker_thread+0x278/0x3c0
[<ffffffff8b0bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8b0c5da1>] kthread+0xd1/0xe0
[<ffffffff8b0c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8b795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8b0c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #229626
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc04873be
Build id: 3.10.0-862.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc04d671e
Build id: 3.10.0-862.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03e72e0
Build id: 3.10.0-862.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff9fe9b050
Build id: 3.10.0-862.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9fea45bc
Build id: 3.10.0-862.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff9fd18ad9
Build id: 3.10.0-862.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff9fd18b9a
Build id: 3.10.0-862.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff9fd1c358
Build id: 3.10.0-862.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff9fd1c7c4
Build id: 3.10.0-862.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc067d224
Build id: 3.10.0-862.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc069ecbb
Build id: 3.10.0-862.el7.x86_64
0x2b
12
xfs_buf_submit
xfs 0xffffffffc067eabc
Build id: 3.10.0-862.el7.x86_64
0x6c
13
xlog_bdstrat
xfs 0xffffffffc069ecbb
Build id: 3.10.0-862.el7.x86_64
0x2b
14
xlog_sync
xfs 0xffffffffc06a0bae
Build id: 3.10.0-862.el7.x86_64
0x2fe
15
xlog_state_release_iclog
xfs 0xffffffffc06a0d4b
Build id: 3.10.0-862.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc06a1a0a
Build id: 3.10.0-862.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc06a33a8
Build id: 3.10.0-862.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc06a3545
Build id: 3.10.0-862.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffff9fab2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff9fab3c18
Build id: 3.10.0-862.el7.x86_64
0x278
21
? manage_workers.isra.24
vmlinux 0xffffffff9fab39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff9fabae31
Build id: 3.10.0-862.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffa011f637
Build id: 3.10.0-862.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/2:1:60794]
Modules linked in: esn_cfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul
syscopyarea crct10dif_common sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 60794 Comm: kworker/2:1 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff8870b7332f70 ti: ffff887061438000 task.ti: ffff887061438000
RIP: 0010:[<ffffffffc04873be>]  [<ffffffffc04873be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff88706143b8e8  EFLAGS: 00000246
RAX: ffffb42600380000 RBX: ffff887064d59070 RCX: 0000000000000006
RDX: ffff887075b7a200 RSI: ffff887075b88000 RDI: 000000000000000e
RBP: ffff88706143b8f8 R08: 0000000000000003 R09: 00000000358cbfff
R10: ffff887075df10d8 R11: ffff8870b6989f80 R12: ffff88707db51f80
R13: ffff88706143b85c R14: ffffffff9fd212cb R15: ffff88706143b8a0
FS:  0000000000000000(0000) GS:ffff8870bc680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f611d299024 CR3: 0000000024fcc000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc04d671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03e72e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9fe9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9fea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff9fd18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff9fd18b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff9fd1c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff9fd1c7c4>] blk_finish_plug+0x14/0x40
[<ffffffffc067d224>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc069ecbb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc067eabc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc069ecbb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc06a0bae>] xlog_sync+0x2fe/0x420 [xfs]
[<ffffffffc06a0d4b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc06a1a0a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc06a33a8>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc06a3545>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffff9fab2dff>] process_one_work+0x17f/0x440
[<ffffffff9fab3c18>] worker_thread+0x278/0x3c0
[<ffffffff9fab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff9fabae31>] kthread+0xd1/0xe0
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa011f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 49 
Complete report #186178
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb43883c5
Build id: 3.10.0-1127.10.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0188632
Build id: 3.10.0-1127.10.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb40eb030
Build id: 3.10.0-1127.10.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb40e2920
Build id: 3.10.0-1127.10.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb40ec04c
Build id: 3.10.0-1127.10.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffb3f52769
Build id: 3.10.0-1127.10.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb3f5282a
Build id: 3.10.0-1127.10.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb3f56068
Build id: 3.10.0-1127.10.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb3f564d4
Build id: 3.10.0-1127.10.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc03fc334
Build id: 3.10.0-1127.10.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc041e617
Build id: 3.10.0-1127.10.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc03fdce2
Build id: 3.10.0-1127.10.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc041e617
Build id: 3.10.0-1127.10.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc0420426
Build id: 3.10.0-1127.10.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc04205ab
Build id: 3.10.0-1127.10.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc042170a
Build id: 3.10.0-1127.10.1.el7.x86_64
0x23a
17
? add_timer
vmlinux 0xffffffffb3caf728
Build id: 3.10.0-1127.10.1.el7.x86_64
0x18
18
xfs_log_worker
xfs 0xffffffffc04217e6
Build id: 3.10.0-1127.10.1.el7.x86_64
0x36
19
process_one_work
vmlinux 0xffffffffb3cbe6bf
Build id: 3.10.0-1127.10.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffb3cbf7d6
Build id: 3.10.0-1127.10.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffffb3cbf6b0
Build id: 3.10.0-1127.10.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffb3cc6691
Build id: 3.10.0-1127.10.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffb3cc65c0
Build id: 3.10.0-1127.10.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffb4392d37
Build id: 3.10.0-1127.10.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffb3cc65c0
Build id: 3.10.0-1127.10.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 36s! [kworker/0:3:2782]
Modules linked in: xt_CHECKSUM ebtable_filter ebtables ip6table_filter ip6_tables xt_dscp nfnetlink_queue nfnetlink_log bluetooth rfkill xt_multiport iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_policy xt_nat ipt_REJECT nf_reject_ipv4 xt_geoip(OE) xt_conntrack iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables tcp_lp fuse devlink tun bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_balloon parport_pc i2c_piix4 parport vmw_vmci xfs libcrc32c ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt sd_mod fb_sys_fops crc_t10dif crct10dif_generic ttm ahci drm nfit crct10dif_pclmul
 crct10dif_common crc32c_intel libnvdimm ata_piix libahci serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 2782 Comm: kworker/0:3 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: xfs-sync/dm-2 xfs_log_worker [xfs]
task: ffff95f113e6e2a0 ti: ffff95f0c8954000 task.ti: ffff95f0c8954000
RIP: 0010:[<ffffffffb43883c5>]  [<ffffffffb43883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff95f0c8957a30  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffffb40e9d5e RCX: 0000000000000800
RDX: ffffba5dc0420000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff95f0c8957a30 R08: ffffffffffffffff R09: ffff95f0c78b1e00
R10: ffff95f0f5a4b000 R11: ffff95f0f59dc000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff95f0d5382180 R15: ffffffffb40e9ce9
FS:  0000000000000000(0000) GS:ffff95f13fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007eff1dfef000 CR3: 000000007af8a000 CR4: 00000000000607f0
Call Trace:
 [<ffffffffc0188632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
 [<ffffffffb40eb030>] ? scsi_test_unit_ready+0x170/0x170
 [<ffffffffb40e2920>] scsi_dispatch_cmd+0xb0/0x240
 [<ffffffffb40ec04c>] scsi_request_fn+0x4ac/0x680
 [<ffffffffb3f52769>] __blk_run_queue+0x39/0x50
 [<ffffffffb3f5282a>] queue_unplugged+0x2a/0xa0
 [<ffffffffb3f56068>] blk_flush_plug_list+0x1d8/0x230
 [<ffffffffb3f564d4>] blk_finish_plug+0x14/0x40
 [<ffffffffc03fc334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
 [<ffffffffc041e617>] ? xlog_bdstrat+0x37/0x70 [xfs]
 [<ffffffffc03fdce2>] __xfs_buf_submit+0x72/0x250 [xfs]
 [<ffffffffc041e617>] xlog_bdstrat+0x37/0x70 [xfs]
 [<ffffffffc0420426>] xlog_sync+0x2e6/0x3f0 [xfs]
 [<ffffffffc04205ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
 [<ffffffffc042170a>] xfs_log_force+0x23a/0x2e0 [xfs]
 [<ffffffffb3caf728>] ? add_timer+0x18/0x20
 [<ffffffffc04217e6>] xfs_log_worker+0x36/0x100 [xfs]
 [<ffffffffb3cbe6bf>] process_one_work+0x17f/0x440
 [<ffffffffb3cbf7d6>] worker_thread+0x126/0x3c0
 [<ffffffffb3cbf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
 [<ffffffffb3cc6691>] kthread+0xd1/0xe0
 [<ffffffffb3cc65c0>] ? insert_kthread_work+0x40/0x40
 [<ffffffffb4392d37>] ret_from_fork_nospec_begin+0x21/0x21
 [<ffffffffb3cc65c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #155826
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffadb883c5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0101632
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffad8eb0a0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffad8e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffad8ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffad7527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffad75286a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffad7560a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffad756514
Build id: 3.10.0-1127.19.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0384334
Build id: 3.10.0-1127.19.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc03a6617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc0385ce2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc03a6617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc03a8426
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc03a85ab
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc03a92e9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x659
17
xlog_cil_push
xfs 0xffffffffc03aac28
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc03aadc5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffffad4be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffad4bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffffad4bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffad4c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffad4c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffadb92d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffad4c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/1:1:16749]
Modules linked in: ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr vmw_balloon i2c_piix4 parport_pc vmw_vmci parport binfmt_misc ip_tables xfs libcrc32c ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ahci ata_piix e1000e crc32c_intel serio_raw libahci libata ptp pps_core nfit vmw_pvscsi libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 16749 Comm: kworker/1:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff8f244d443150 ti: ffff8f245c1d0000 task.ti: ffff8f245c1d0000
RIP: 0010:[<ffffffffadb883c5>]  [<ffffffffadb883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8f245c1d3948  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff8f245c1d3908 RCX: ffffffffae034e40
RDX: ffff9a2ac0ed0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8f245c1d3948 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8f24b21a2000 R11: ffff8f24b212d000 R12: 0000000000000020
R13: ffff8f24b3f44c00 R14: ffffffffad8e9d59 R15: ffff8f245c1d38c0
FS:  0000000000000000(0000) GS:ffff8f24bfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fecf5346830 CR3: 0000000155816000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0101632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffad8eb0a0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffad8e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffad8ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffffad7527a9>] __blk_run_queue+0x39/0x50
[<ffffffffad75286a>] queue_unplugged+0x2a/0xa0
[<ffffffffad7560a8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffad756514>] blk_finish_plug+0x14/0x40
[<ffffffffc0384334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc03a6617>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0385ce2>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc03a6617>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc03a8426>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc03a85ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc03a92e9>] xlog_write+0x659/0x750 [xfs]
[<ffffffffc03aac28>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc03aadc5>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffffad4be6bf>] process_one_work+0x17f/0x440
[<ffffffffad4bf7d6>] worker_thread+0x126/0x3c0
[<ffffffffad4bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffad4c6691>] kthread+0xd1/0xe0
[<ffffffffad4c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffadb92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffad4c65c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
Complete report #93838
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc00633be
Build id: 3.10.0-1127.18.2.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc008571e
Build id: 3.10.0-1127.18.2.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc00562e0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffb52e2950
Build id: 3.10.0-1127.18.2.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb52ec07c
Build id: 3.10.0-1127.18.2.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffb5152769
Build id: 3.10.0-1127.18.2.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb515282a
Build id: 3.10.0-1127.18.2.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb5156068
Build id: 3.10.0-1127.18.2.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb51564d4
Build id: 3.10.0-1127.18.2.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc02fc334
Build id: 3.10.0-1127.18.2.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc031e617
Build id: 3.10.0-1127.18.2.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc02fdce2
Build id: 3.10.0-1127.18.2.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc031e617
Build id: 3.10.0-1127.18.2.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc0320426
Build id: 3.10.0-1127.18.2.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc03205ab
Build id: 3.10.0-1127.18.2.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc032126a
Build id: 3.10.0-1127.18.2.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc0322c28
Build id: 3.10.0-1127.18.2.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc0322dc5
Build id: 3.10.0-1127.18.2.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffffb4ebe6bf
Build id: 3.10.0-1127.18.2.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffb4ebf928
Build id: 3.10.0-1127.18.2.el7.x86_64
0x278
21
? manage_workers.isra.26
vmlinux 0xffffffffb4ebf6b0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffb4ec6691
Build id: 3.10.0-1127.18.2.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffb4ec65c0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffb5592d37
Build id: 3.10.0-1127.18.2.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffb4ec65c0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 29s! [kworker/0:1:28249]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus aesni_intel snd_seq ppdev lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm vmw_balloon
joydev pcspkr snd_timer snd soundcore sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel drm serio_raw libata mptspi nfit libnvdimm e1000 scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 28249 Comm: kworker/0:1 Tainted: G        W    L ------------   3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff9dad0c53c1c0 ti: ffff9dad81298000 task.ti: ffff9dad81298000
RIP: 0010:[<ffffffffc00633be>]  [<ffffffffc00633be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9dad8129b8e0  EFLAGS: 00000246
RAX: ffffc149c03c0000 RBX: ffff9dad141c31e0 RCX: 0000000000000001
RDX: ffff9dad35046e00 RSI: ffff9dad35adf000 RDI: 000000000000000e
RBP: ffff9dad8129b8f0 R08: 0000000000000003 R09: 0000000035b48fff
R10: ffff9dad35af9400 R11: ffff9dad229248c0 R12: 0000000100000008
R13: ffff9dad0436de00 R14: ffff9dad8129b854 R15: ffffffffb515b08b
FS:  0000000000000000(0000) GS:ffff9dad7c600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f744da2a000 CR3: 000000007647e000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc008571e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00562e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb52e2950>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb52ec07c>] scsi_request_fn+0x4ac/0x680
[<ffffffffb5152769>] __blk_run_queue+0x39/0x50
[<ffffffffb515282a>] queue_unplugged+0x2a/0xa0
[<ffffffffb5156068>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb51564d4>] blk_finish_plug+0x14/0x40
[<ffffffffc02fc334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc031e617>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc02fdce2>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc031e617>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0320426>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc03205ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc032126a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc0322c28>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc0322dc5>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffffb4ebe6bf>] process_one_work+0x17f/0x440
[<ffffffffb4ebf928>] worker_thread+0x278/0x3c0
[<ffffffffb4ebf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb4ec6691>] kthread+0xd1/0xe0
[<ffffffffb4ec65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb5592d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb4ec65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 13 07 
Complete report #65641
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff816ab915
Build id: 3.10.0-693.11.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0059632
Build id: 3.10.0-693.11.1.el7.x86_64
0x442
3
? scsi_test_unit_ready
vmlinux 0xffffffff8146d0e0
Build id: 3.10.0-693.11.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8146535a
Build id: 3.10.0-693.11.1.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8146e11f
Build id: 3.10.0-693.11.1.el7.x86_64
0x4df
6
__blk_run_queue
vmlinux 0xffffffff812f7113
Build id: 3.10.0-693.11.1.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812f71ca
Build id: 3.10.0-693.11.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812fa658
Build id: 3.10.0-693.11.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812faa64
Build id: 3.10.0-693.11.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc026f354
Build id: 3.10.0-693.11.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc0290d1b
Build id: 3.10.0-693.11.1.el7.x86_64
0x2b
12
xfs_buf_submit
xfs 0xffffffffc0270b5c
Build id: 3.10.0-693.11.1.el7.x86_64
0x6c
13
xlog_bdstrat
xfs 0xffffffffc0290d1b
Build id: 3.10.0-693.11.1.el7.x86_64
0x2b
14
xlog_sync
xfs 0xffffffffc0292c3e
Build id: 3.10.0-693.11.1.el7.x86_64
0x2fe
15
xlog_state_release_iclog
xfs 0xffffffffc0292ddb
Build id: 3.10.0-693.11.1.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc0293939
Build id: 3.10.0-693.11.1.el7.x86_64
0x659
17
xlog_cil_push
xfs 0xffffffffc0295198
Build id: 3.10.0-693.11.1.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc0295335
Build id: 3.10.0-693.11.1.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffff810a882a
Build id: 3.10.0-693.11.1.el7.x86_64
0x17a
20
worker_thread
vmlinux 0xffffffff810a9648
Build id: 3.10.0-693.11.1.el7.x86_64
0x278
21
? manage_workers.isra.24
vmlinux 0xffffffff810a93d0
Build id: 3.10.0-693.11.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff810b099f
Build id: 3.10.0-693.11.1.el7.x86_64
0xcf
23
? finish_task_switch
vmlinux 0xffffffff810bf117
Build id: 3.10.0-693.11.1.el7.x86_64
0x57
24
? insert_kthread_work
vmlinux 0xffffffff810b08d0
Build id: 3.10.0-693.11.1.el7.x86_64
0x40
25
ret_from_fork
vmlinux 0xffffffff816b4fd8
Build id: 3.10.0-693.11.1.el7.x86_64
0x58
26
? insert_kthread_work
vmlinux 0xffffffff810b08d0
Build id: 3.10.0-693.11.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 36s! [kworker/2:1:12852]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd nfit ppdev vmw_balloon libnvdimm sg joydev pcspkr parport_pc shpchp vmw_vmci parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common syscopyarea ahci crc32c_intel sysfillrect ata_piix
sysimgblt libahci fb_sys_fops ttm serio_raw libata drm vmxnet3 i2c_core vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 12852 Comm: kworker/2:1 Tainted: G             L ------------   3.10.0-693.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: xfs-cil/dm-2 xlog_cil_push_work [xfs]
task: ffff88099368cf10 ti: ffff8809f8ad4000 task.ti: ffff8809f8ad4000
RIP: 0010:[<ffffffff816ab915>]  [<ffffffff816ab915>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8809f8ad7950  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff813499a0 RCX: 0000000000000200
RDX: ffffc900069a0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8809f8ad7950 R08: 0000000000000000 R09: ffff8809db239500
R10: ffff8808d0fdcc40 R11: ffff880a03157800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff88090f696180 R15: ffffffff8146bd09
FS:  0000000000000000(0000) GS:ffff880a2fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc5eb76d000 CR3: 00000009e1f6e000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8809f8ad79c8 ffffffffc0059632 ffff8809db239500 ffff880a02789000
ffff880a02307680 ffff880a0339a7e8 ffffffff8146d0e0 0000000000000002
ffff880a0339a754 0000000000000286 ffff8808d0fdcc40 0000000000000000
Call Trace:
[<ffffffffc0059632>] pvscsi_queue+0x442/0x7e1 [vmw_pvscsi]
[<ffffffff8146d0e0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8146535a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e11f>] scsi_request_fn+0x4df/0x680
[<ffffffff812f7113>] __blk_run_queue+0x33/0x40
[<ffffffff812f71ca>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa658>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812faa64>] blk_finish_plug+0x14/0x40
[<ffffffffc026f354>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0290d1b>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc0270b5c>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc0290d1b>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc0292c3e>] xlog_sync+0x2fe/0x420 [xfs]
[<ffffffffc0292ddb>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc0293939>] xlog_write+0x659/0x720 [xfs]
[<ffffffffc0295198>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc0295335>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffff810a882a>] process_one_work+0x17a/0x440
[<ffffffff810a9648>] worker_thread+0x278/0x3c0
[<ffffffff810a93d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b099f>] kthread+0xcf/0xe0
[<ffffffff810bf117>] ? finish_task_switch+0x57/0x160
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4fd8>] ret_from_fork+0x58/0x90
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #21221
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9d7883c5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc02a0632
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff9d4eb0a0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff9d4e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff9d4ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff9d3527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff9d35286a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff9d3560a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff9d356514
Build id: 3.10.0-1127.19.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc04e4334
Build id: 3.10.0-1127.19.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc0506617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc04e5ce2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc0506617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc0508426
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc05085ab
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc050970a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x23a
17
? __queue_delayed_work
vmlinux 0xffffffff9d0bc99b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x8b
18
xfs_log_worker
xfs 0xffffffffc05097e6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x36
19
process_one_work
vmlinux 0xffffffff9d0be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff9d0bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff9d0bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff9d0c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff9d0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff9d792d1d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7
25
? insert_kthread_work
vmlinux 0xffffffff9d0c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 34s! [kworker/0:3:29736]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink arc4 md4 nls_utf8 cifs ccm dns_resolver ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel sg lrw gf128mul glue_helper pcspkr joydev vmw_vmci ablk_helper cryptd i2c_piix4 parport_pc parport ip_tables xfs libcrc32c
sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod crc_t10dif sysimgblt fb_sys_fops crct10dif_generic ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ahci libahci libata vmxnet3 nfit vmw_pvscsi libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 29736 Comm: kworker/0:3 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff8b3e0ac41070 ti: ffff8b3f153e8000 task.ti: ffff8b3f153e8000
RIP: 0010:[<ffffffff9d7883c5>]  [<ffffffff9d7883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8b3f153eba30  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff9d3b14b3 RCX: ffff8b3f391c1060
RDX: ffff9bb240eb0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8b3f153eba30 R08: 0000000000000000 R09: ffff8b3dcc9ca400
R10: ffff8b3f346ab000 R11: ffff8b3db5f6b800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8b3dd7642d80 R15: ffffffff9d4e9d59
FS:  0000000000000000(0000) GS:ffff8b3f3fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffee7490be8 CR3: 00000000b34aa000 CR4: 00000000007607f0
PKRU: 00000000
Call Trace:
[<ffffffffc02a0632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff9d4eb0a0>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff9d4e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9d4ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff9d3527a9>] __blk_run_queue+0x39/0x50
[<ffffffff9d35286a>] queue_unplugged+0x2a/0xa0
[<ffffffff9d3560a8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff9d356514>] blk_finish_plug+0x14/0x40
[<ffffffffc04e4334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0506617>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc04e5ce2>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc0506617>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0508426>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc05085ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc050970a>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffff9d0bc99b>] ? __queue_delayed_work+0x8b/0x1a0
[<ffffffffc05097e6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff9d0be6bf>] process_one_work+0x17f/0x440
[<ffffffff9d0bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff9d0bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9d0c6691>] kthread+0xd1/0xe0
[<ffffffff9d0c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9d792d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff9d0c65c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #19532
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03d43be
Build id: 3.10.0-1062.18.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc03b471e
Build id: 3.10.0-1062.18.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03bf2e0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff8c8decd0
Build id: 3.10.0-1062.18.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8c8e833c
Build id: 3.10.0-1062.18.1.el7.x86_64
0x4cc
6
__blk_run_queue
vmlinux 0xffffffff8c74f289
Build id: 3.10.0-1062.18.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8c74f34a
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8c752b98
Build id: 3.10.0-1062.18.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8c753004
Build id: 3.10.0-1062.18.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0592044
Build id: 3.10.0-1062.18.1.el7.x86_64
0x334
11
? xfs_buf_submit
xfs 0xffffffffc0593b4a
Build id: 3.10.0-1062.18.1.el7.x86_64
0x1a
12
__xfs_buf_submit
xfs 0xffffffffc05938c1
Build id: 3.10.0-1062.18.1.el7.x86_64
0x61
13
xfs_buf_submit
xfs 0xffffffffc0593b4a
Build id: 3.10.0-1062.18.1.el7.x86_64
0x1a
14
xlog_bdstrat
xfs 0xffffffffc05b40ab
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2b
15
xlog_sync
xfs 0xffffffffc05b5ea6
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2e6
16
xlog_state_release_iclog
xfs 0xffffffffc05b602b
Build id: 3.10.0-1062.18.1.el7.x86_64
0x7b
17
xlog_write
xfs 0xffffffffc05b6cea
Build id: 3.10.0-1062.18.1.el7.x86_64
0x5da
18
xlog_cil_push
xfs 0xffffffffc05b86e8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a8
19
xlog_cil_push_work
xfs 0xffffffffc05b8885
Build id: 3.10.0-1062.18.1.el7.x86_64
0x15
20
process_one_work
vmlinux 0xffffffff8c4be34f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x17f
21
worker_thread
vmlinux 0xffffffff8c4bf5b8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x278
22
? manage_workers.isra.26
vmlinux 0xffffffff8c4bf340
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a0
23
kthread
vmlinux 0xffffffff8c4c6321
Build id: 3.10.0-1062.18.1.el7.x86_64
0xd1
24
? insert_kthread_work
vmlinux 0xffffffff8c4c6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
25
ret_from_fork_nospec_begin
vmlinux 0xffffffff8cb8dd37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
26
? insert_kthread_work
vmlinux 0xffffffff8c4c6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 90s! [kworker/2:2:51238]
Modules linked in: tcp_lp rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink xt_addrtype br_netfilter cifs ccm dns_resolver fuse ip6t_rpfilter ip6t_REJECT vmhgfs(OE) nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 ppdev vmw_balloon coretemp snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul btusb ac97_bus
snd_seq btrtl ghash_clmulni_intel btbcm btintel snd_seq_device snd_pcm bluetooth aesni_intel lrw gf128mul glue_helper ablk_helper cryptd nfit snd_timer joydev libnvdimm pcspkr snd vmw_vmci rfkill sg soundcore i2c_piix4 parport_pc parport pcc_cpufreq ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common mptbase crc32c_intel ata_piix libata serio_raw e1000 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 51238 Comm: kworker/2:2 Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff931695750000 ti: ffff9315f0410000 task.ti: ffff9315f0410000
RIP: 0010:[<ffffffffc03d43be>]  [<ffffffffc03d43be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9315f04138d0  EFLAGS: 00000246
RAX: ffffa3e7c33c0000 RBX: ffff9315fe473ae0 RCX: 0000000000000015
RDX: ffff931375cc6000 RSI: ffff931851aa8000 RDI: 000000000000000e
RBP: ffff9315f04138e0 R08: 0000000000000003 R09: 0000000510aa6fff
R10: ffff9318521720d8 R11: ffff931568c31500 R12: 0000000100000008
R13: ffff93151177f000 R14: ffff9315f0413844 R15: ffffffff8c757aeb
FS:  0000000000000000(0000) GS:ffff931857680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7228600000 CR3: 00000004d8eb0000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc03b471e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03bf2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8c8decd0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8c8e833c>] scsi_request_fn+0x4cc/0x680
[<ffffffff8c74f289>] __blk_run_queue+0x39/0x50
[<ffffffff8c74f34a>] queue_unplugged+0x2a/0xa0
[<ffffffff8c752b98>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8c753004>] blk_finish_plug+0x14/0x40
[<ffffffffc0592044>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0593b4a>] ? xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc05938c1>] __xfs_buf_submit+0x61/0x210 [xfs]
[<ffffffffc0593b4a>] xfs_buf_submit+0x1a/0x40 [xfs]
[<ffffffffc05b40ab>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc05b5ea6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc05b602b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc05b6cea>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc05b86e8>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc05b8885>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffff8c4be34f>] process_one_work+0x17f/0x440
[<ffffffff8c4bf5b8>] worker_thread+0x278/0x3c0
[<ffffffff8c4bf340>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8c4c6321>] kthread+0xd1/0xe0
[<ffffffff8c4c6250>] ? insert_kthread_work+0x40/0x40
[<ffffffff8cb8dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8c4c6250>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 3e 
Complete report #6824
Frame # Function Binary Source or offset Line
1
mptscsih_qcmd
mptscsih 0xffffffffc047b71e
Build id: 3.10.0-957.10.1.el7.x86_64
0x24e
2
mptspi_qcmd
mptspi 0xffffffffc04b32e0
Build id: 3.10.0-957.10.1.el7.x86_64
0x50
3
scsi_dispatch_cmd
vmlinux 0xffffffffaa4d14c0
Build id: 3.10.0-957.10.1.el7.x86_64
0xb0
4
scsi_request_fn
vmlinux 0xffffffffaa4daa5c
Build id: 3.10.0-957.10.1.el7.x86_64
0x4cc
5
__blk_run_queue
vmlinux 0xffffffffaa3436b9
Build id: 3.10.0-957.10.1.el7.x86_64
0x39
6
queue_unplugged
vmlinux 0xffffffffaa34377a
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a
7
blk_flush_plug_list
vmlinux 0xffffffffaa346fa8
Build id: 3.10.0-957.10.1.el7.x86_64
0x1d8
8
blk_finish_plug
vmlinux 0xffffffffaa347414
Build id: 3.10.0-957.10.1.el7.x86_64
0x14
9
_xfs_buf_ioapply
xfs 0xffffffffc06aa454
Build id: 3.10.0-957.10.1.el7.x86_64
0x334
10
? xlog_bdstrat
xfs 0xffffffffc06ccacb
Build id: 3.10.0-957.10.1.el7.x86_64
0x2b
11
xfs_buf_submit
xfs 0xffffffffc06abfcc
Build id: 3.10.0-957.10.1.el7.x86_64
0x6c
12
xlog_bdstrat
xfs 0xffffffffc06ccacb
Build id: 3.10.0-957.10.1.el7.x86_64
0x2b
13
xlog_sync
xfs 0xffffffffc06ce956
Build id: 3.10.0-957.10.1.el7.x86_64
0x2e6
14
xlog_state_release_iclog
xfs 0xffffffffc06ceadb
Build id: 3.10.0-957.10.1.el7.x86_64
0x7b
15
xlog_write
xfs 0xffffffffc06cf79a
Build id: 3.10.0-957.10.1.el7.x86_64
0x5da
16
xlog_cil_push
xfs 0xffffffffc06d1138
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a8
17
xlog_cil_push_work
xfs 0xffffffffc06d12d5
Build id: 3.10.0-957.10.1.el7.x86_64
0x15
18
process_one_work
vmlinux 0xffffffffaa0b9d8f
Build id: 3.10.0-957.10.1.el7.x86_64
0x17f
19
worker_thread
vmlinux 0xffffffffaa0baf78
Build id: 3.10.0-957.10.1.el7.x86_64
0x278
20
? manage_workers.isra.25
vmlinux 0xffffffffaa0bad00
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a0
21
kthread
vmlinux 0xffffffffaa0c1c71
Build id: 3.10.0-957.10.1.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffffaa0c1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffffaa775c37
Build id: 3.10.0-957.10.1.el7.x86_64
0x21
24
? insert_kthread_work
vmlinux 0xffffffffaa0c1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 25s! [kworker/1:4:46151]
[<ffffffffc047b71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04b32e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffaa4d14c0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffaa4daa5c>] scsi_request_fn+0x4cc/0x680
[<ffffffffaa3436b9>] __blk_run_queue+0x39/0x50
[<ffffffffaa34377a>] queue_unplugged+0x2a/0xa0
[<ffffffffaa346fa8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffaa347414>] blk_finish_plug+0x14/0x40
[<ffffffffc06aa454>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc06ccacb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc06abfcc>] xfs_buf_submit+0x6c/0x220 [xfs]
[<ffffffffc06ccacb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffc06ce956>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc06ceadb>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc06cf79a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc06d1138>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc06d12d5>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffffaa0b9d8f>] process_one_work+0x17f/0x440
[<ffffffffaa0baf78>] worker_thread+0x278/0x3c0
[<ffffffffaa0bad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffaa0c1c71>] kthread+0xd1/0xe0
[<ffffffffaa0c1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffaa775c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaa0c1ba0>] ? insert_kthread_work+0x40/0x40
Complete report #6374
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02f03be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc031271e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc032a2e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffffbd2e2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffbd2ec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffbd1527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffbd15286a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffbd1560a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffbd156514
Build id: 3.10.0-1127.19.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc058b334
Build id: 3.10.0-1127.19.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc05ad617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc058cce2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc05ad617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc05af426
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc05af5ab
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc05b026a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc05b1c28
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc05b1dc5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffffbcebe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffffbcebf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffffbcebf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffffbcec6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffffbcec65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffffbd592d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffffbcec65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 1057s! [kworker/1:0:51971]
Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_REDIRECT nf_nat_redirect xt_nat veth vxlan ip6_udp_tunnel udp_tunnel xt_mark nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi ppdev
crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi btusb btrtl snd_ac97_codec btbcm btintel aesni_intel vmw_balloon bluetooth lrw ac97_bus gf128mul glue_helper ablk_helper snd_seq cryptd snd_seq_device snd_pcm snd_timer rfkill sg snd vmw_vmci pcspkr joydev soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix libata serio_raw e1000 nfit libnvdimm mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 51971 Comm: kworker/1:0 Kdump: loaded Tainted: G               ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff915fd3808000 ti: ffff915fdd5c8000 task.ti: ffff915fdd5c8000
RIP: 0010:[<ffffffffc02f03be>]  [<ffffffffc02f03be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff915fdd5cb8e0  EFLAGS: 00000246
RAX: ffffa96d03480000 RBX: ffff915b96cb80e0 RCX: 000000000000000c
RDX: ffff91609f86dc00 RSI: ffff9160a0a8e000 RDI: 000000000000000e
RBP: ffff915fdd5cb8f0 R08: 0000000000000003 R09: 000000061ea6dfff
R10: ffff9160a057e400 R11: ffff915e1231a300 R12: 0000000100000008
R13: ffff91607b14c780 R14: ffff915fdd5cb854 R15: ffffffffbd15b0cb
FS:  0000000000000000(0000) GS:ffff9160a5640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000987000 CR3: 00000004c9210000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc031271e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc032a2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffbd2e2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffbd2ec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffffbd1527a9>] __blk_run_queue+0x39/0x50
[<ffffffffbd15286a>] queue_unplugged+0x2a/0xa0
[<ffffffffbd1560a8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffbd156514>] blk_finish_plug+0x14/0x40
[<ffffffffc058b334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc05ad617>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc058cce2>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc05ad617>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc05af426>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc05af5ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc05b026a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc05b1c28>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc05b1dc5>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffffbcebe6bf>] process_one_work+0x17f/0x440
[<ffffffffbcebf7d6>] worker_thread+0x126/0x3c0
[<ffffffffbcebf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffbcec6691>] kthread+0xd1/0xe0
[<ffffffffbcec65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd592d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbcec65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 2f 
Complete report #4721
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02b83be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02ef71e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc032b2e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff85ee2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff85eec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff85d527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff85d5286a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff85d560a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff85d56514
Build id: 3.10.0-1127.19.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc0534334
Build id: 3.10.0-1127.19.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc0556617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc0535ce2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc0556617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc0558426
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc05585ab
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7b
16
xlog_write
xfs 0xffffffffc055926a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5da
17
xlog_cil_push
xfs 0xffffffffc055ac28
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a8
18
xlog_cil_push_work
xfs 0xffffffffc055adc5
Build id: 3.10.0-1127.19.1.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffff85abe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff85abf928
Build id: 3.10.0-1127.19.1.el7.x86_64
0x278
21
? manage_workers.isra.26
vmlinux 0xffffffff85abf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff85ac6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff86192d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [kworker/0:3:15454]
Modules linked in: arc4 md4 nls_utf8 cifs ccm dns_resolver fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel ata_piix serio_raw drm libata mptspi scsi_transport_spi nfit libnvdimm mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmfsrtx_bridge_helper]
CPU: 0 PID: 15454 Comm: kworker/0:3 Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff92a4ef6c8000 ti: ffff92a541138000 task.ti: ffff92a541138000
RIP: 0010:[<ffffffffc02b83be>]  [<ffffffffc02b83be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff92a54113b8e0  EFLAGS: 00000246
RAX: ffffae5d407c0000 RBX: ffff92a576425520 RCX: 000000000000001f
RDX: ffff92a475499400 RSI: ffff92a573aba000 RDI: 000000000000000e
RBP: ffff92a54113b8f0 R08: 0000000000000003 R09: 00000000af59ffff
R10: ffff92a4754a1400 R11: ffff92a4b9822f40 R12: 0000000100000002
R13: ffff92a577083c00 R14: ffff92a54113b854 R15: ffffffff85d5b0cb
FS:  0000000000000000(0000) GS:ffff92a57ae00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7702fa4000 CR3: 0000000136894000 CR4: 00000000000007f0
Call Trace:
[<ffffffffc02ef71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc032b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff85ee2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff85eec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff85d527a9>] __blk_run_queue+0x39/0x50
[<ffffffff85d5286a>] queue_unplugged+0x2a/0xa0
[<ffffffff85d560a8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff85d56514>] blk_finish_plug+0x14/0x40
[<ffffffffc0534334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0556617>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0535ce2>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc0556617>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0558426>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc05585ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc055926a>] xlog_write+0x5da/0x750 [xfs]
[<ffffffffc055ac28>] xlog_cil_push+0x2a8/0x430 [xfs]
[<ffffffffc055adc5>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffff85abe6bf>] process_one_work+0x17f/0x440
[<ffffffff85abf928>] worker_thread+0x278/0x3c0
[<ffffffff85abf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff85ac6691>] kthread+0xd1/0xe0
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff86192d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 2c 
Complete report #4480
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03603be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc037b71e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc03432e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff85ce2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff85cec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff85b527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff85b5286a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff85b560a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff85b56514
Build id: 3.10.0-1127.19.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc05c9334
Build id: 3.10.0-1127.19.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc05eb617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc05cace2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc05eb617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc05ed426
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc05ed5ab
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc05ee70a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x23a
17
? __queue_delayed_work
vmlinux 0xffffffff858bc99b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x8b
18
xfs_log_worker
xfs 0xffffffffc05ee7e6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x36
19
process_one_work
vmlinux 0xffffffff858be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff858bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff858bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff858c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff858c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff85f92d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff858c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:3:5185]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper vmw_balloon cryptd joydev pcspkr sg parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod
vmwgfx crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit ata_piix drm crct10dif_pclmul libnvdimm crct10dif_common libata crc32c_intel mptspi serio_raw scsi_transport_spi mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 5185 Comm: kworker/0:3 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff9f05c7bc62a0 ti: ffff9f0676798000 task.ti: ffff9f0676798000
RIP: 0010:[<ffffffffc03603be>]  [<ffffffffc03603be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff9f067679b9c8  EFLAGS: 00000246
RAX: ffffb58700840000 RBX: ffff9f05d5cdf910 RCX: 0000000000000015
RDX: ffff9f0575f1b400 RSI: ffff9f06744cf000 RDI: 000000000000000e
RBP: ffff9f067679b9d8 R08: 0000000000000003 R09: 00000001346cf9ff
R10: ffff9f06745e6400 R11: ffff9f067870ba80 R12: ffff9f06777aaa80
R13: ffff9f067679b93c R14: ffffffff85b5b0cb R15: ffff9f067679b980
FS:  0000000000000000(0000) GS:ffff9f067fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0737386000 CR3: 0000000035e6a000 CR4: 00000000000407f0
Call Trace:
[<ffffffffc037b71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03432e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff85ce2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff85cec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff85b527a9>] __blk_run_queue+0x39/0x50
[<ffffffff85b5286a>] queue_unplugged+0x2a/0xa0
[<ffffffff85b560a8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff85b56514>] blk_finish_plug+0x14/0x40
[<ffffffffc05c9334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc05eb617>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc05cace2>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc05eb617>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc05ed426>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc05ed5ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc05ee70a>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffff858bc99b>] ? __queue_delayed_work+0x8b/0x1a0
[<ffffffffc05ee7e6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff858be6bf>] process_one_work+0x17f/0x440
[<ffffffff858bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff858bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff858c6691>] kthread+0xd1/0xe0
[<ffffffff858c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff85f92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff858c65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 36 
Complete report #4259
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffa00773be
Build id: 3.10.0-514.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffa004f729
Build id: 3.10.0-514.el7.x86_64
0x249
3
mptspi_qcmd
mptspi 0xffffffffa00702b0
Build id: 3.10.0-514.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff814518da
Build id: 3.10.0-514.el7.x86_64
0xaa
5
scsi_request_fn
vmlinux 0xffffffff8145a9e1
Build id: 3.10.0-514.el7.x86_64
0x501
6
__blk_run_queue
vmlinux 0xffffffff812eb053
Build id: 3.10.0-514.el7.x86_64
0x33
7
queue_unplugged
vmlinux 0xffffffff812eb10a
Build id: 3.10.0-514.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff812eff68
Build id: 3.10.0-514.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff812f0374
Build id: 3.10.0-514.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffa02a7014
Build id: 3.10.0-514.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffa02c89eb
Build id: 3.10.0-514.el7.x86_64
0x2b
12
xfs_buf_submit
xfs 0xffffffffa02a8890
Build id: 3.10.0-514.el7.x86_64
0x70
13
xlog_bdstrat
xfs 0xffffffffa02c89eb
Build id: 3.10.0-514.el7.x86_64
0x2b
14
xlog_sync
xfs 0xffffffffa02ca915
Build id: 3.10.0-514.el7.x86_64
0x315
15
xlog_state_release_iclog
xfs 0xffffffffa02caad3
Build id: 3.10.0-514.el7.x86_64
0x93
16
xlog_write
xfs 0xffffffffa02cb586
Build id: 3.10.0-514.el7.x86_64
0x5d6
17
xlog_cil_push
xfs 0xffffffffa02cce89
Build id: 3.10.0-514.el7.x86_64
0x239
18
xlog_cil_push_work
xfs 0xffffffffa02cd085
Build id: 3.10.0-514.el7.x86_64
0x15
19
process_one_work
vmlinux 0xffffffff810a7f3b
Build id: 3.10.0-514.el7.x86_64
0x17b
20
worker_thread
vmlinux 0xffffffff810a8ef3
Build id: 3.10.0-514.el7.x86_64
0x2a3
21
? rescuer_thread
vmlinux 0xffffffff810a8c50
Build id: 3.10.0-514.el7.x86_64
0x460
22
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64
0xcf
23
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
0x140
24
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64
0x58
25
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64
0x140
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [kworker/u32:1:23256]
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [kworker/9:2:23257]
Modules linked in: nfnetlink_queue nfnetlink_log nfnetlink bluetooth rfkill vmw_vsock_vmci_transport vsock intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg vmw_balloon pcspkr parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect mptspi sysimgblt scsi_transport_spi fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix drm crc32c_intel serio_raw mptscsih libata mptbase vmxnet3 i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 23257 Comm: kworker/9:2 Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
Workqueue: xfs-cil/dm-0 xlog_cil_push_work [xfs]
task: ffff88062bac8000 ti: ffff8806302ac000 task.ti: ffff8806302ac000
RIP: 0010:[<ffffffffa00773be>]  [<ffffffffa00773be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8806302af8f0  EFLAGS: 00000246
RAX: ffffc90003500000 RBX: ffff8806302af8c0 RCX: 0000000000000010
RDX: ffff8806373d2600 RSI: ffff8804c503e000 RDI: 000000000000000e
RBP: ffff8806302af900 R08: 0000000000000003 R09: ffff8806c50320d8
R10: ffff880635abdc00 R11: ffff880635abdc00 R12: 0000000100000006
R13: ffff880430ecb900 R14: ffff8806302af864 R15: ffffffff812f4a0b
FS:  0000000000000000(0000) GS:ffff880637c80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f33317d1ab8 CR3: 00000000019ba000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8804c503ed30 0000000000000024 ffff8806302af9a8 ffffffffa004f729
ffff8804c503e008 04000000302af968 0000000000000010 ffff880635abdc00
0000000000000040 ffff8804c503e188 0000006000000010 ffff880636082e30
Call Trace:
[<ffffffffa004f729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa00702b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff814518da>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8145a9e1>] scsi_request_fn+0x501/0x770
[<ffffffff812eb053>] __blk_run_queue+0x33/0x40
[<ffffffff812eb10a>] queue_unplugged+0x2a/0xa0
[<ffffffff812eff68>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812f0374>] blk_finish_plug+0x14/0x40
[<ffffffffa02a7014>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffa02c89eb>] ? xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffa02a8890>] xfs_buf_submit+0x70/0x1f0 [xfs]
[<ffffffffa02c89eb>] xlog_bdstrat+0x2b/0x60 [xfs]
[<ffffffffa02ca915>] xlog_sync+0x315/0x440 [xfs]
[<ffffffffa02caad3>] xlog_state_release_iclog+0x93/0xb0 [xfs]
[<ffffffffa02cb586>] xlog_write+0x5d6/0x720 [xfs]
[<ffffffffa02cce89>] xlog_cil_push+0x239/0x420 [xfs]
[<ffffffffa02cd085>] xlog_cil_push_work+0x15/0x20 [xfs]
[<ffffffff810a7f3b>] process_one_work+0x17b/0x470
[<ffffffff810a8ef3>] worker_thread+0x2a3/0x410
[<ffffffff810a8c50>] ? rescuer_thread+0x460/0x460
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 08 
Complete report #406520
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb398e9f5
Build id: 3.10.0-1160.62.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0317632
Build id: 3.10.0-1160.62.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffffb36ecd50
Build id: 3.10.0-1160.62.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffffb36e4640
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffffb36edd6c
Build id: 3.10.0-1160.62.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffffb3553cf9
Build id: 3.10.0-1160.62.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffffb3553dba
Build id: 3.10.0-1160.62.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffffb35576b8
Build id: 3.10.0-1160.62.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffffb3557b24
Build id: 3.10.0-1160.62.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc05842c4
Build id: 3.10.0-1160.62.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc05a6697
Build id: 3.10.0-1160.62.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc0585c72
Build id: 3.10.0-1160.62.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc05a6697
Build id: 3.10.0-1160.62.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc05a84f6
Build id: 3.10.0-1160.62.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc05a867b
Build id: 3.10.0-1160.62.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc05a97da
Build id: 3.10.0-1160.62.1.el7.x86_64
0x23a
17
? xfs_trans_ail_cursor_first
xfs 0xffffffffc05b5a40
Build id: 3.10.0-1160.62.1.el7.x86_64
0x90
18
xfsaild
xfs 0xffffffffc05b5bd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x190
19
? xfs_trans_ail_cursor_first
xfs 0xffffffffc05b5a40
Build id: 3.10.0-1160.62.1.el7.x86_64
0x90
20
kthread
vmlinux 0xffffffffb32c5fb1
Build id: 3.10.0-1160.62.1.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffffb32c5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffb3999df7
Build id: 3.10.0-1160.62.1.el7.x86_64
0x21
23
? insert_kthread_work
vmlinux 0xffffffffb32c5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [xfsaild/dm-0:505]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13109(E) tcp_lp falcon_lsm_pinned_12905(E) nfsv3 nfs_acl nfs lockd grace fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg parport_pc vmw_vmci parport i2c_piix4 binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic
drm_kms_helper ahci crct10dif_pclmul crct10dif_common syscopyarea sysfillrect ata_piix libahci sysimgblt fb_sys_fops crc32c_intel ttm serio_raw libata drm vmxnet3 nfit libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 1 PID: 505 Comm: xfsaild/dm-0 Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff918bbb4f8000 ti: ffff918bbb500000 task.ti: ffff918bbb500000
RIP: 0010:[<ffffffffb398e9f5>]  [<ffffffffb398e9f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff918bbb503a38  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff918bbb5039f8 RCX: ffffffffb3e34f40
RDX: ffffb6c540780000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff918bbb503a38 R08: 0000000000000001 R09: 0000000000000000
R10: ffff918b34815c00 R11: ffff918c33752800 R12: 0000000000000020
R13: ffff918b4ae37080 R14: ffffffffb36eba09 R15: ffff918bbb5039b0
FS:  0000000000000000(0000) GS:ffff918c3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc8b2fb25f0 CR3: 000000008032e000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc0317632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffffb36ecd50>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffffb36e4640>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb36edd6c>] scsi_request_fn+0x4ac/0x680
[<ffffffffb3553cf9>] __blk_run_queue+0x39/0x50
[<ffffffffb3553dba>] queue_unplugged+0x2a/0xa0
[<ffffffffb35576b8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb3557b24>] blk_finish_plug+0x14/0x40
[<ffffffffc05842c4>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc05a6697>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0585c72>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc05a6697>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc05a84f6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc05a867b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc05a97da>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffffc05b5a40>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffc05b5bd0>] xfsaild+0x190/0x780 [xfs]
[<ffffffffc05b5a40>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffb32c5fb1>] kthread+0xd1/0xe0
[<ffffffffb32c5ee0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb3999df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb32c5ee0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #277037
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8218b565
Build id: 3.10.0-1160.25.1.el7.x86_64
0x15
2
ata_scsi_queuecmd
libata 0xffffffffc02edd45
Build id: 3.10.0-1160.25.1.el7.x86_64
0x155
3
? ata_scsi_invalid_field
libata 0xffffffffc02eaa20
Build id: 3.10.0-1160.25.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff81ee3fa0
Build id: 3.10.0-1160.25.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff81eed6cc
Build id: 3.10.0-1160.25.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff81d53939
Build id: 3.10.0-1160.25.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff81d539fa
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff81d57338
Build id: 3.10.0-1160.25.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff81d577a4
Build id: 3.10.0-1160.25.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc04bf2c4
Build id: 3.10.0-1160.25.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc04e16c7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc04c0c72
Build id: 3.10.0-1160.25.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc04e16c7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc04e34d6
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc04e365b
Build id: 3.10.0-1160.25.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc04e47ba
Build id: 3.10.0-1160.25.1.el7.x86_64
0x23a
17
? xfs_trans_ail_cursor_first
xfs 0xffffffffc04f0a40
Build id: 3.10.0-1160.25.1.el7.x86_64
0x90
18
xfsaild
xfs 0xffffffffc04f0bd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x190
19
? xfs_trans_ail_cursor_first
xfs 0xffffffffc04f0a40
Build id: 3.10.0-1160.25.1.el7.x86_64
0x90
20
kthread
vmlinux 0xffffffff81ac5da1
Build id: 3.10.0-1160.25.1.el7.x86_64
0xd1
21
? insert_kthread_work
vmlinux 0xffffffff81ac5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffff82195df7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
23
? insert_kthread_work
vmlinux 0xffffffff81ac5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [xfsaild/sda2:561]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter sunrpc ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd i2c_piix4 pcspkr sg virtio_balloon joydev ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ahci ata_piix libahci crc32c_intel libata drm_panel_orientation_quirks
serio_raw virtio_pci virtio_ring virtio floppy fuse
CPU: 2 PID: 561 Comm: xfsaild/sda2 Tainted: G             L ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org 04/01/2014
task: ffff942f963a4200 ti: ffff942fa6fb8000 task.ti: ffff942fa6fb8000
RIP: 0010:[<ffffffff8218b565>]  [<ffffffff8218b565>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff942fa6fbba68  EFLAGS: 00000286
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffff942fac7eb018 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff942fa6fbba68 R08: 0000000000000000 R09: 00000001603a9fff
R10: ffff942fadfaa000 R11: ffff942fa479a1c0 R12: ffff942fa6fbb9e8
R13: ffff942f9c3f83d8 R14: ffff9430d7fbb800 R15: ffffffff81eeb3de
FS:  0000000000000000(0000) GS:ffff94346bd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f336aa18000 CR3: 0000000298010000 CR4: 0000000000160fe0
Call Trace:
[<ffffffffc02edd45>] ata_scsi_queuecmd+0x155/0x450 [libata]
[<ffffffffc02eaa20>] ? ata_scsi_invalid_field+0x50/0x50 [libata]
[<ffffffff81ee3fa0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff81eed6cc>] scsi_request_fn+0x4ac/0x680
[<ffffffff81d53939>] __blk_run_queue+0x39/0x50
[<ffffffff81d539fa>] queue_unplugged+0x2a/0xa0
[<ffffffff81d57338>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff81d577a4>] blk_finish_plug+0x14/0x40
[<ffffffffc04bf2c4>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc04e16c7>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc04c0c72>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc04e16c7>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc04e34d6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc04e365b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc04e47ba>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffffc04f0a40>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffffc04f0bd0>] xfsaild+0x190/0x780 [xfs]
[<ffffffffc04f0a40>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
[<ffffffff81ac5da1>] kthread+0xd1/0xe0
[<ffffffff81ac5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff82195df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff81ac5cd0>] ? insert_kthread_work+0x40/0x40
Code: 46 bd 98 ff 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 e8 26 bd 98 ff 66 90 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48 
Complete report #191030
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8a78b495
Build id: 3.10.0-1160.21.1.el7.x86_64
0x15
2
pvscsi_queue
vmw_pvscsi 0xffffffffc0179632
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4a2
3
? scsi_test_unit_ready
vmlinux 0xffffffff8a4ec580
Build id: 3.10.0-1160.21.1.el7.x86_64
0x170
4
scsi_dispatch_cmd
vmlinux 0xffffffff8a4e3e70
Build id: 3.10.0-1160.21.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff8a4ed59c
Build id: 3.10.0-1160.21.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff8a353809
Build id: 3.10.0-1160.21.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8a3538ca
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff8a357208
Build id: 3.10.0-1160.21.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff8a357674
Build id: 3.10.0-1160.21.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc03db2c4
Build id: 3.10.0-1160.21.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc03fd6c7
Build id: 3.10.0-1160.21.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc03dcc72
Build id: 3.10.0-1160.21.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc03fd6c7
Build id: 3.10.0-1160.21.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc03ff4d6
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc03ff65b
Build id: 3.10.0-1160.21.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc04007ba
Build id: 3.10.0-1160.21.1.el7.x86_64
0x23a
17
? console_unlock
vmlinux 0xffffffff8a09da06
Build id: 3.10.0-1160.21.1.el7.x86_64
0x216
18
xfs_log_worker
xfs 0xffffffffc0400896
Build id: 3.10.0-1160.21.1.el7.x86_64
0x36
19
process_one_work
vmlinux 0xffffffff8a0bddcf
Build id: 3.10.0-1160.21.1.el7.x86_64
0x17f
20
worker_thread
vmlinux 0xffffffff8a0beee6
Build id: 3.10.0-1160.21.1.el7.x86_64
0x126
21
? manage_workers.isra.26
vmlinux 0xffffffff8a0bedc0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2a0
22
kthread
vmlinux 0xffffffff8a0c5da1
Build id: 3.10.0-1160.21.1.el7.x86_64
0xd1
23
? insert_kthread_work
vmlinux 0xffffffff8a0c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
24
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a795df7
Build id: 3.10.0-1160.21.1.el7.x86_64
0x21
25
? insert_kthread_work
vmlinux 0xffffffff8a0c5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [mongod:1950]
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 21s! [kworker/9:2:25206]
Modules linked in: vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr sg joydev vmw_balloon parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect sysimgblt crc_t10dif fb_sys_fops crct10dif_generic ttm drm ata_piix crct10dif_pclmul crct10dif_common nfit libata crc32c_intel serio_raw libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 9 PID: 25206 Comm: kworker/9:2 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff8ee07d3da100 ti: ffff8ee1595bc000 task.ti: ffff8ee1595bc000
RIP: 0010:[<ffffffff8a78b495>]  [<ffffffff8a78b495>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8ee1595bfa30  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff8a3b27a3 RCX: ffffffff8ac34f40
RDX: ffffb0a081bd0000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8ee1595bfa30 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8edff7ea9800 R11: ffff8edff9d3b000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8ee07afed200 R15: ffffffff8a4eb239
FS:  0000000000000000(0000) GS:ffff8ee17fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f814ce6f000 CR3: 000000023e7de000 CR4: 00000000000407e0
Call Trace:
[<ffffffffc0179632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
[<ffffffff8a4ec580>] ? scsi_test_unit_ready+0x170/0x170
[<ffffffff8a4e3e70>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8a4ed59c>] scsi_request_fn+0x4ac/0x680
[<ffffffff8a353809>] __blk_run_queue+0x39/0x50
[<ffffffff8a3538ca>] queue_unplugged+0x2a/0xa0
[<ffffffff8a357208>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8a357674>] blk_finish_plug+0x14/0x40
[<ffffffffc03db2c4>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc03fd6c7>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc03dcc72>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc03fd6c7>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc03ff4d6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc03ff65b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc04007ba>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffff8a09da06>] ? console_unlock+0x216/0x4b0
[<ffffffffc0400896>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff8a0bddcf>] process_one_work+0x17f/0x440
[<ffffffff8a0beee6>] worker_thread+0x126/0x3c0
[<ffffffff8a0bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8a0c5da1>] kthread+0xd1/0xe0
[<ffffffff8a0c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8a0c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48 
Complete report #78111
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc02303be
Build id: 3.10.0-1160.11.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc02a971e
Build id: 3.10.0-1160.11.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc024b2e0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff862e3b90
Build id: 3.10.0-1160.11.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff862ed2bc
Build id: 3.10.0-1160.11.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff861536d9
Build id: 3.10.0-1160.11.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff8615379a
Build id: 3.10.0-1160.11.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff86156fd8
Build id: 3.10.0-1160.11.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff86157444
Build id: 3.10.0-1160.11.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc04c32c4
Build id: 3.10.0-1160.11.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc04e56c7
Build id: 3.10.0-1160.11.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc04c4c72
Build id: 3.10.0-1160.11.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc04e56c7
Build id: 3.10.0-1160.11.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc04e74d6
Build id: 3.10.0-1160.11.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc04e765b
Build id: 3.10.0-1160.11.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc04e87ba
Build id: 3.10.0-1160.11.1.el7.x86_64
0x23a
17
xfs_log_worker
xfs 0xffffffffc04e8896
Build id: 3.10.0-1160.11.1.el7.x86_64
0x36
18
process_one_work
vmlinux 0xffffffff85ebde9f
Build id: 3.10.0-1160.11.1.el7.x86_64
0x17f
19
worker_thread
vmlinux 0xffffffff85ebefb6
Build id: 3.10.0-1160.11.1.el7.x86_64
0x126
20
? manage_workers.isra.26
vmlinux 0xffffffff85ebee90
Build id: 3.10.0-1160.11.1.el7.x86_64
0x2a0
21
kthread
vmlinux 0xffffffff85ec5e71
Build id: 3.10.0-1160.11.1.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffff85ec5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffff86593df7
Build id: 3.10.0-1160.11.1.el7.x86_64
0x21
24
? insert_kthread_work
vmlinux 0xffffffff85ec5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/2:1:67381]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_CHECKSUM xt_addrtype ipt_MASQUERADE nf_nat_masquerade_ipv4 br_netfilter tun overlay(T) devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw vmw_balloon gf128mul glue_helper ablk_helper cryptd pcspkr joydev vmw_vmci sg i2c_piix4 parport_pc parport
ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit ata_piix drm libata libnvdimm mptspi scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common crc32c_intel mptbase e1000 serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 67381 Comm: kworker/2:1 Tainted: G        W    L ------------ T 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: xfs-sync/dm-2 xfs_log_worker [xfs]
task: ffff893cf7fba100 ti: ffff893d332bc000 task.ti: ffff893d332bc000
RIP: 0010:[<ffffffffc02303be>]  [<ffffffffc02303be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff893d332bf9c8  EFLAGS: 00000246
RAX: ffffb3f200d40000 RBX: ffff893c0a0db200 RCX: 0000000000000013
RDX: ffff893d71236c00 RSI: ffff893d71951000 RDI: 000000000000000e
RBP: ffff893d332bf9d8 R08: 0000000000000002 R09: 00000000b90315ff
R10: ffff893d71a83400 R11: ffff893d7040d6c0 R12: ffff893d594fdf80
R13: ffff893d332bf93c R14: ffffffff8615bffb R15: ffff893d332bf980
FS:  0000000000000000(0000) GS:ffff893d76680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0d7bad2d90 CR3: 00000001f3ba8000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc02a971e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc024b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff862e3b90>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff862ed2bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff861536d9>] __blk_run_queue+0x39/0x50
[<ffffffff8615379a>] queue_unplugged+0x2a/0xa0
[<ffffffff86156fd8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff86157444>] blk_finish_plug+0x14/0x40
[<ffffffffc04c32c4>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc04e56c7>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc04c4c72>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc04e56c7>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc04e74d6>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc04e765b>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc04e87ba>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffffc04e8896>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff85ebde9f>] process_one_work+0x17f/0x440
[<ffffffff85ebefb6>] worker_thread+0x126/0x3c0
[<ffffffff85ebee90>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff85ec5e71>] kthread+0xd1/0xe0
[<ffffffff85ec5da0>] ? insert_kthread_work+0x40/0x40
[<ffffffff86593df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff85ec5da0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 23 
Complete report #8182
Frame # Function Binary Source or offset Line
1
mpt_put_msg_frame
mptbase 0xffffffffc03483be
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5e
2
mptscsih_qcmd
mptscsih 0xffffffffc041371e
Build id: 3.10.0-1127.19.1.el7.x86_64
0x24e
3
mptspi_qcmd
mptspi 0xffffffffc04272e0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
4
scsi_dispatch_cmd
vmlinux 0xffffffff89ee2990
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb0
5
scsi_request_fn
vmlinux 0xffffffff89eec0bc
Build id: 3.10.0-1127.19.1.el7.x86_64
0x4ac
6
__blk_run_queue
vmlinux 0xffffffff89d527a9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x39
7
queue_unplugged
vmlinux 0xffffffff89d5286a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a
8
blk_flush_plug_list
vmlinux 0xffffffff89d560a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x1d8
9
blk_finish_plug
vmlinux 0xffffffff89d56514
Build id: 3.10.0-1127.19.1.el7.x86_64
0x14
10
_xfs_buf_ioapply
xfs 0xffffffffc05ee334
Build id: 3.10.0-1127.19.1.el7.x86_64
0x334
11
? xlog_bdstrat
xfs 0xffffffffc0610617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
12
__xfs_buf_submit
xfs 0xffffffffc05efce2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x72
13
xlog_bdstrat
xfs 0xffffffffc0610617
Build id: 3.10.0-1127.19.1.el7.x86_64
0x37
14
xlog_sync
xfs 0xffffffffc0612426
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2e6
15
xlog_state_release_iclog
xfs 0xffffffffc06125ab
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7b
16
xfs_log_force
xfs 0xffffffffc061370a
Build id: 3.10.0-1127.19.1.el7.x86_64
0x23a
17
xfs_log_worker
xfs 0xffffffffc06137e6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x36
18
process_one_work
vmlinux 0xffffffff89abe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
19
worker_thread
vmlinux 0xffffffff89abf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
20
? manage_workers.isra.26
vmlinux 0xffffffff89abf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
21
kthread
vmlinux 0xffffffff89ac6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
22
? insert_kthread_work
vmlinux 0xffffffff89ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
23
ret_from_fork_nospec_begin
vmlinux 0xffffffff8a192d1d
Build id: 3.10.0-1127.19.1.el7.x86_64
0x7
24
? insert_kthread_work
vmlinux 0xffffffff89ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [kworker/1:2:107639]
Modules linked in: eset_rtp(OE) binfmt_misc xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq iosf_mbi crc32_pclmul snd_seq_device ppdev snd_pcm ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon
snd_timer snd pcspkr joydev soundcore sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel nfit ata_piix libnvdimm serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 107639 Comm: kworker/1:2 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: xfs-sync/dm-0 xfs_log_worker [xfs]
task: ffff8999ccf462a0 ti: ffff8999b5470000 task.ti: ffff8999b5470000
RIP: 0010:[<ffffffffc03483be>]  [<ffffffffc03483be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff8999b54739c8  EFLAGS: 00000246
RAX: ffffa77840700000 RBX: ffff8999e9aa0210 RCX: 0000000000000007
RDX: ffff899a71bdac00 RSI: ffff899975933000 RDI: 000000000000000e
RBP: ffff8999b54739d8 R08: 0000000000000003 R09: 0000000034fa87ff
R10: ffff899975807800 R11: ffff8999e9a3b640 R12: ffff8999b540fd80
R13: ffff8999b547393c R14: ffffffff89d5b0cb R15: ffff8999b5473980
FS:  0000000000000000(0000) GS:ffff899a79640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002e0bc0b159d8 CR3: 00000000ac930000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc041371e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04272e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff89ee2990>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff89eec0bc>] scsi_request_fn+0x4ac/0x680
[<ffffffff89d527a9>] __blk_run_queue+0x39/0x50
[<ffffffff89d5286a>] queue_unplugged+0x2a/0xa0
[<ffffffff89d560a8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff89d56514>] blk_finish_plug+0x14/0x40
[<ffffffffc05ee334>] _xfs_buf_ioapply+0x334/0x460 [xfs]
[<ffffffffc0610617>] ? xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc05efce2>] __xfs_buf_submit+0x72/0x250 [xfs]
[<ffffffffc0610617>] xlog_bdstrat+0x37/0x70 [xfs]
[<ffffffffc0612426>] xlog_sync+0x2e6/0x3f0 [xfs]
[<ffffffffc06125ab>] xlog_state_release_iclog+0x7b/0xd0 [xfs]
[<ffffffffc061370a>] xfs_log_force+0x23a/0x2e0 [xfs]
[<ffffffffc06137e6>] xfs_log_worker+0x36/0x100 [xfs]
[<ffffffff89abe6bf>] process_one_work+0x17f/0x440
[<ffffffff89abf7d6>] worker_thread+0x126/0x3c0
[<ffffffff89abf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff89ac6691>] kthread+0xd1/0xe0
[<ffffffff89ac65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a192d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff89ac65c0>] ? insert_kthread_work+0x40/0x40
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 35