Complete report #1359681
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc00693be
Build id: 3.10.0-693.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc00d8729
Build id: 3.10.0-693.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc00e32b0
Build id: 3.10.0-693.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64
|
0xaa
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64
|
0x4df
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff811c4678
Build id: 3.10.0-693.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff811b235b
Build id: 3.10.0-693.el7.x86_64
|
0xadb
|
|
12
|
|
vmlinux |
0xffffffff81212f4b
Build id: 3.10.0-693.el7.x86_64
|
0x4b
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
|
0x154
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
|
0x35
|
|
15
|
page_fault
|
vmlinux |
0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [ora_scmn_orcl:2743]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev nfit sg libnvdimm parport_pc parport shpchp i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif pata_acpi crct10dif_generic vmwgfx ata_piix mptspi crct10dif_pclmul drm_kms_helper crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm
e1000 scsi_transport_spi crc32c_intel mptscsih libata serio_raw mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 2743 Comm: ora_scmn_orcl Not tainted 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff88021cb23f40 ti: ffff880223e3c000 task.ti: ffff880223e3c000
RIP: 0010:[<ffffffffc00693be>] [<ffffffffc00693be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff880223e3fb58 EFLAGS: 00000246
RAX: ffffc90000cc0000 RBX: ffff8802365318a0 RCX: 0000000000000008
RDX: ffff8800354bd200 RSI: ffff8802375af000 RDI: 000000000000000e
RBP: ffff880223e3fb68 R08: 0000000000000003 R09: ffff880035eb40d8
R10: ffff880221e1b800 R11: ffff880221e1b800 R12: ffff880005665e00
R13: ffff880223e3facc R14: ffffffff812ff13b R15: ffff880223e3fb10
FS: 00007f4175b923c0(0000) GS:ffff880236680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000018edc580 CR3: 00000001f3ef1000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8802375afd30 000000000000003c ffff880223e3fc10 ffffffffc00d8729
ffff8802375af008 0000000023e3fbd0 0000000000000008 ffff880221e1b800
0000000000000020 ffff8802375af188 0000006000000008 ffff880231ec2b30
Call Trace:
[<ffffffffc00d8729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00e32b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffff811c4678>] swapin_readahead+0xc8/0x110
[<ffffffff811b235b>] handle_mm_fault+0xadb/0xfa0
[<ffffffff81212f4b>] ? do_filp_open+0x4b/0xb0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 73 07
Complete report #240059
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffa00a73be
Build id: 3.10.0-229.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffa0076729
Build id: 3.10.0-229.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffa00912b0
Build id: 3.10.0-229.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff813f0a19
Build id: 3.10.0-229.el7.x86_64
|
0xb9
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff813f9b0f
Build id: 3.10.0-229.el7.x86_64
|
0x32f
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812aaee3
Build id: 3.10.0-229.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812aaf9a
Build id: 3.10.0-229.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812aedc8
Build id: 3.10.0-229.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812af1c4
Build id: 3.10.0-229.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff81194d48
Build id: 3.10.0-229.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff81182eff
Build id: 3.10.0-229.el7.x86_64
|
0x93f
|
|
12
|
__do_page_fault
|
vmlinux |
0xffffffff8160fe06
Build id: 3.10.0-229.el7.x86_64
|
0x156
|
|
13
|
|
vmlinux |
0xffffffff810d508e
Build id: 3.10.0-229.el7.x86_64
|
0xfe
|
BUG: soft lockup - CPU#1 stuck for 36s! [gnome-shell:3315]
Modules linked in: tcp_lp bnep bluetooth rfkill fuse ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter snd_ens1371 crc32_pclmul crc32c_intel snd_rawmidi snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq snd_seq_device aesni_intel lrw gf128mul glue_helper snd_pcm ppdev ablk_helper cryptd vmw_balloon snd_timer pcspkr serio_raw snd soundcore sg parport_pc parport vmw_vmci i2c_piix4 shpchp uinput ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif
pata_acpi vmwgfx drm_kms_helper e1000 ttm crct10dif_pclmul crct10dif_common mptspi scsi_transport_spi drm mptscsih mptbase ata_piix i2c_core libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 3315 Comm: gnome-shell Not tainted 3.10.0-229.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8800759c6660 ti: ffff88005cfe0000 task.ti: ffff88005cfe0000
RIP: 0010:[<ffffffffa00a73be>] [<ffffffffa00a73be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88005cfe3ac8 EFLAGS: 00000246
RAX: ffffc900083c0000 RBX: ffff88003bf27180 RCX: 000000000000000f
RDX: ffff8800362e8e00 RSI: ffff88003675e000 RDI: 000000000000000e
RBP: ffff88005cfe3ad8 R08: 0000000000000003 R09: 0000000060654fff
R10: ffff880075ad64c0 R11: 0000000000000000 R12: ffffffff812b358b
R13: ffff88005cfe3a90 R14: 0000000000000003 R15: ffff880078155218
FS: 00007f3336a53700(0000) GS:ffff88007c620000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f330c21cc00 CR3: 000000005cddc000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88003675ed30 0000000000000048 ffff88005cfe3b80 ffffffffa0076729
ffff88003675e008 00000000033e1708 000000000000000f ffff880075ad64c0
000000000000003c ffff88003675e188 000000600000000f ffff880076d82dd0
Call Trace:
[<ffffffffa0076729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa00912b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff813f0a19>] scsi_dispatch_cmd+0xb9/0x360
[<ffffffff813f9b0f>] scsi_request_fn+0x32f/0x570
[<ffffffff812aaee3>] __blk_run_queue+0x33/0x40
[<ffffffff812aaf9a>] queue_unplugged+0x2a/0xa0
[<ffffffff812aedc8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812af1c4>] blk_finish_plug+0x14/0x40
[<ffffffff81194d48>] swapin_readahead+0xc8/0x110
[<ffffffff81182eff>] handle_mm_fault+0x93f/0xd70
[<ffffffff8160fe06>] __do_page_fault+0x156/0x540
[<ffffffff810d508e>] ? do_futex+0xfe/0x5b0
Complete report #1228880
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc04423be
Build id: 3.10.0-862.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc046471e
Build id: 3.10.0-862.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc04102e0
Build id: 3.10.0-862.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff98e9b050
Build id: 3.10.0-862.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff98ea45bc
Build id: 3.10.0-862.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff98d18ad9
Build id: 3.10.0-862.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff98d18b9a
Build id: 3.10.0-862.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff98d1c358
Build id: 3.10.0-862.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff98d1c7c4
Build id: 3.10.0-862.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff98bda828
Build id: 3.10.0-862.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff98bc5777
Build id: 3.10.0-862.el7.x86_64
|
0x777
|
|
12
|
|
vmlinux |
0xffffffff98b033a0
Build id: 3.10.0-862.el7.x86_64
|
0x90
|
|
13
|
handle_mm_fault
|
vmlinux |
0xffffffff98bc747d
Build id: 3.10.0-862.el7.x86_64
|
0x39d
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffff9911a597
Build id: 3.10.0-862.el7.x86_64
|
0x197
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffff9911a925
Build id: 3.10.0-862.el7.x86_64
|
0x35
|
|
16
|
page_fault
|
vmlinux |
0xffffffff99116768
Build id: 3.10.0-862.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 57s! [HikariPool-1 ho:17020]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmhgfs(OE) bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq btusb snd_seq_device aesni_intel ppdev lrw gf128mul btrtl glue_helper btbcm ablk_helper btintel cryptd bluetooth snd_pcm snd_timer pcspkr joydev rfkill snd sg vmw_vmci soundcore nfit i2c_piix4 shpchp libnvdimm parport_pc parport ip_tables xfs libcrc32c
sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul drm crct10dif_common crc32c_intel ata_piix serio_raw libata mptspi scsi_transport_spi e1000 mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 17020 Comm: HikariPool-1 ho Kdump: loaded Tainted: G OEL ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff915d773f8fd0 ti: ffff915e37fc0000 task.ti: ffff915e37fc0000
RIP: 0010:[<ffffffffc04423be>] [<ffffffffc04423be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0018:ffff915e37fc3aa8 EFLAGS: 00010246
RAX: ffffb966c1c00000 RBX: ffff915d4e630aa0 RCX: 0000000000000001
RDX: ffff915d38961c00 RSI: ffff915b75fa2000 RDI: 000000000000000e
RBP: ffff915e37fc3ab8 R08: 0000000000000003 R09: 00000001139b2fff
R10: ffff915b75fa40d8 R11: ffff915d40246bc0 R12: ffff915e0e701500
R13: ffff915e37fc3a1c R14: ffffffff98d212cb R15: ffff915e37fc3a60
FS: 00007f6be6ff7700(0000) GS:ffff915e79540000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6cd0051820 CR3: 000000030f228000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc046471e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04102e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff98e9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff98ea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff98d18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff98d18b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff98d1c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff98d1c7c4>] blk_finish_plug+0x14/0x40
[<ffffffff98bda828>] swapin_readahead+0xc8/0x110
[<ffffffff98bc5777>] handle_pte_fault+0x777/0xc30
[<ffffffff98b033a0>] ? futex_wake+0x90/0x180
[<ffffffff98bc747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9911a597>] __do_page_fault+0x197/0x4f0
[<ffffffff9911a925>] do_page_fault+0x35/0x90
[<ffffffff99116768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 45
Complete report #1098045
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc03c83be
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc052c71e
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc045b2e0
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffffa129b4f0
Build id: 3.10.0-862.3.3.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffffa12a4a5c
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffffa1118f19
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffffa1118fda
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffffa111c798
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffffa111cc04
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffffa0fdab68
Build id: 3.10.0-862.3.3.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffffa0fc5ab7
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x777
|
|
12
|
handle_mm_fault
|
vmlinux |
0xffffffffa0fc77bd
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x39d
|
|
13
|
|
vmlinux |
0xffffffffa0ed849c
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x14c
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffffa151b5c7
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x197
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffffa151b955
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x35
|
|
16
|
page_fault
|
vmlinux |
0xffffffffa1517788
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 30s! [gnome-shell:50450]
Modules linked in: l2tp_ppp l2tp_netlink l2tp_core pppoe pppox ppp_generic slhc xt_multiport xt_policy dummy xt_nat veth tcp_lp fuse nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc coretemp
iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev snd_seq_midi snd_seq_midi_event snd_ens1371 cryptd snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore joydev pcspkr sg vmw_balloon vmw_vmci parport_pc parport i2c_piix4 shpchp ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi ata_piix mptscsih drm e1000 crct10dif_pclmul crct10dif_common libata i2c_core serio_raw crc32c_intel mptbase floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 50450 Comm: gnome-shell Kdump: loaded Tainted: G L ------------ 3.10.0-862.3.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
task: ffffa09b2c12dee0 ti: ffffa09b40fec000 task.ti: ffffa09b40fec000
RIP: 0010:[<ffffffffc03c83be>] [<ffffffffc03c83be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffffa09b40fefaa8 EFLAGS: 00000246
RAX: ffffbf9d00800000 RBX: ffffa09bf50e32a0 RCX: 000000000000000e
RDX: ffffa09af5616a00 RSI: ffffa09af51c8000 RDI: 000000000000000e
RBP: ffffa09b40fefab8 R08: 0000000000000003 R09: 0000000026a15fff
R10: ffffa09bf7fe60d8 R11: ffffa09bf3e4d340 R12: ffffa09af5632700
R13: ffffa09b40fefa1c R14: ffffffffa112170b R15: ffffa09b40fefa60
FS: 00007fbbc3b72ac0(0000) GS:ffffa09bfa600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000001e3475e0 CR3: 000000001ced0000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc052c71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc045b2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa129b4f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa12a4a5c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa1118f19>] __blk_run_queue+0x39/0x50
[<ffffffffa1118fda>] queue_unplugged+0x2a/0xa0
[<ffffffffa111c798>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa111cc04>] blk_finish_plug+0x14/0x40
[<ffffffffa0fdab68>] swapin_readahead+0xc8/0x110
[<ffffffffa0fc5ab7>] handle_pte_fault+0x777/0xc30
[<ffffffffa0fc77bd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa0ed849c>] ? update_curr+0x14c/0x1e0
[<ffffffffa151b5c7>] __do_page_fault+0x197/0x4f0
[<ffffffffa151b955>] do_page_fault+0x35/0x90
[<ffffffffa1517788>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 63 3d
Complete report #923387
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc00123be
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc00b9729
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc00ab2b0
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff814654da
Build id: 3.10.0-693.5.2.el7.x86_64
|
0xaa
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff8146e29f
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x4df
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812f7293
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812f734a
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812fa7d8
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812fabe4
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff811c45e8
Build id: 3.10.0-693.5.2.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff811b22cb
Build id: 3.10.0-693.5.2.el7.x86_64
|
0xadb
|
|
12
|
__do_page_fault
|
vmlinux |
0xffffffff816b0074
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x154
|
|
13
|
do_page_fault
|
vmlinux |
0xffffffff816b03a5
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x35
|
|
14
|
page_fault
|
vmlinux |
0xffffffff816ac5c8
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [abrt-applet:2251]
Modules linked in: tcp_lp binfmt_misc ext4 mbcache jbd2 fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmhgfs(OE) vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus iosf_mbi snd_seq crc32_pclmul ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw
gf128mul glue_helper ablk_helper cryptd snd_timer snd soundcore ppdev vmw_balloon sg nfit joydev pcspkr libnvdimm parport_pc parport vmw_vmci shpchp i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi scsi_transport_spi ata_piix mptscsih serio_raw libata e1000 mptbase i2c_core
CPU: 0 PID: 2251 Comm: abrt-applet Tainted: G OEL ------------ 3.10.0-693.5.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff88000aee0fd0 ti: ffff88000d390000 task.ti: ffff88000d390000
RIP: 0010:[<ffffffffc00123be>] [<ffffffffc00123be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88000d393b58 EFLAGS: 00000246
RAX: ffffc90000200000 RBX: ffff88000375b988 RCX: 0000000000000000
RDX: ffff880035537e00 RSI: ffff8800363ed000 RDI: 000000000000000e
RBP: ffff88000d393b68 R08: 0000000000000003 R09: 0000000021068fff
R10: ffff88003834dc00 R11: ffff88003834dc00 R12: ffff88000932ca80
R13: ffff88000d393acc R14: ffffffff812ff1fb R15: ffff88000d393b10
FS: 00007ff6fee47ac0(0000) GS:ffff88003be00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000055d3ca275280 CR3: 000000000d2e3000 CR4: 00000000000407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800363edd30 0000000000000060 ffff88000d393c10 ffffffffc00b9729
ffff8800363ed008 040000000d393bd0 0000000000000000 ffff88003834dc00
0000000000000000 ffff8800363ed188 0000006000000000 ffff88000000000b
Call Trace:
[<ffffffffc00b9729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00ab2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff814654da>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e29f>] scsi_request_fn+0x4df/0x680
[<ffffffff812f7293>] __blk_run_queue+0x33/0x40
[<ffffffff812f734a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa7d8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fabe4>] blk_finish_plug+0x14/0x40
[<ffffffff811c45e8>] swapin_readahead+0xc8/0x110
[<ffffffff811b22cb>] handle_mm_fault+0xadb/0xfa0
[<ffffffff816b0074>] __do_page_fault+0x154/0x450
[<ffffffff816b03a5>] do_page_fault+0x35/0x90
[<ffffffff816ac5c8>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 02
Complete report #626443
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc00cc3be
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc00af729
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc00622b0
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff8146782a
Build id: 3.10.0-693.17.1.el7.x86_64
|
0xaa
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff814705ef
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x4df
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812f96a3
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812f975a
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812fcbe8
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812fcff4
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff811c6718
Build id: 3.10.0-693.17.1.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff811b43fb
Build id: 3.10.0-693.17.1.el7.x86_64
|
0xadb
|
|
12
|
__do_page_fault
|
vmlinux |
0xffffffff816b37e4
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x154
|
|
13
|
do_page_fault
|
vmlinux |
0xffffffff816b3b15
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x35
|
|
14
|
|
vmlinux |
0xffffffff816afcda
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x13b
|
|
15
|
page_fault
|
vmlinux |
0xffffffff816af8f8
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [gnome-terminal-:2985]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables fuse tun ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg vmw_balloon joydev pcspkr vmw_vmci shpchp i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace
sunrpc xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel serio_raw mptspi scsi_transport_spi mptscsih mptbase ata_piix vmxnet3 libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 2985 Comm: gnome-terminal- Tainted: G L ------------ 3.10.0-693.17.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013
task: ffff8801377c8fd0 ti: ffff88007958c000 task.ti: ffff88007958c000
RIP: 0010:[<ffffffffc00cc3be>] [<ffffffffc00cc3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88007958fb58 EFLAGS: 00000246
RAX: ffffc90000780000 RBX: ffff8801122436a0 RCX: 0000000000000011
RDX: ffff880035cb6200 RSI: ffff880134fb4000 RDI: 000000000000000e
RBP: ffff88007958fb68 R08: 0000000000000003 R09: ffff880134aaa0d8
R10: ffff8800b3c6e700 R11: ffff8800b3c6e700 R12: ffffffff816b9bff
R13: ffffffff816b9c06 R14: ffffffff816b9c0d R15: ffffffff816b9c14
FS: 00007f0ed38d8a40(0000) GS:ffff88013fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0ecfec3a40 CR3: 00000000795b0000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc00af729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00622b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146782a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff814705ef>] scsi_request_fn+0x4df/0x680
[<ffffffff812f96a3>] __blk_run_queue+0x33/0x40
[<ffffffff812f975a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fcbe8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fcff4>] blk_finish_plug+0x14/0x40
[<ffffffff811c6718>] swapin_readahead+0xc8/0x110
[<ffffffff811b43fb>] handle_mm_fault+0xadb/0xfa0
[<ffffffff816b37e4>] __do_page_fault+0x154/0x450
[<ffffffff816b3b15>] do_page_fault+0x35/0x90
[<ffffffff816afcda>] ? error_swapgs+0x13b/0x18d
[<ffffffff816af8f8>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 a3 0d
Complete report #541610
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc00b43be
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc018e71e
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc019e2e0
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffffb6e9bb60
Build id: 3.10.0-862.9.1.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffffb6ea50cc
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffffb6d19519
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffffb6d195da
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffffb6d1cd98
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffffb6d1d204
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffffb6bdafe8
Build id: 3.10.0-862.9.1.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffffb6bc5f37
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x777
|
|
12
|
|
vmlinux |
0xffffffffb6b03b90
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x90
|
|
13
|
handle_mm_fault
|
vmlinux |
0xffffffffb6bc7c3d
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x39d
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffffb711b557
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x197
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffffb711b8e5
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x35
|
|
16
|
page_fault
|
vmlinux |
0xffffffffb7117758
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [mysqld:2407]
Modules linked in: tcp_lp fuse veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype xt_CHECKSUM iptable_mangle br_netfilter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec ac97_bus snd_seq aesni_intel lrw gf128mul snd_seq_device glue_helper ablk_helper ppdev snd_pcm cryptd vmw_balloon snd_timer pcspkr joydev snd vmw_vmci nfit sg soundcore i2c_piix4 shpchp libnvdimm parport_pc parport binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic sr_mod cdrom vmwgfx ata_generic mptspi pata_acpi drm_kms_helper scsi_transport_spi syscopyarea mptscsih sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul crct10dif_common drm e1000 mptbase crc32c_intel serio_raw libata i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2407 Comm: mysqld Kdump: loaded Tainted: G W L ------------ T 3.10.0-862.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff93903a51af70 ti: ffff93903a530000 task.ti: ffff93903a530000
RIP: 0010:[<ffffffffc00b43be>] [<ffffffffc00b43be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff93903a533aa8 EFLAGS: 00000246
RAX: ffffbb57c0980000 RBX: ffff93904936d7a0 RCX: 0000000000000021
RDX: ffff938ff567a200 RSI: ffff9390f196b000 RDI: 000000000000000e
RBP: ffff93903a533ab8 R08: 0000000000000003 R09: 000000012dbecfff
R10: ffff9390f190e0d8 R11: ffff939031104c40 R12: ffff9390462d1e00
R13: ffff93903a533a1c R14: ffffffffb6d21d0b R15: ffff93903a533a60
FS: 00007f33ebb1d700(0000) GS:ffff9390f96c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f34053c2728 CR3: 000000007a470000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc018e71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc019e2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffb6e9bb60>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffb6ea50cc>] scsi_request_fn+0x4cc/0x680
[<ffffffffb6d19519>] __blk_run_queue+0x39/0x50
[<ffffffffb6d195da>] queue_unplugged+0x2a/0xa0
[<ffffffffb6d1cd98>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffb6d1d204>] blk_finish_plug+0x14/0x40
[<ffffffffb6bdafe8>] swapin_readahead+0xc8/0x110
[<ffffffffb6bc5f37>] handle_pte_fault+0x777/0xc30
[<ffffffffb6b03b90>] ? futex_wake+0x90/0x180
[<ffffffffb6bc7c3d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb711b557>] __do_page_fault+0x197/0x4f0
[<ffffffffb711b8e5>] do_page_fault+0x35/0x90
[<ffffffffb7117758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 23 0c
Complete report #476615
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc01e03be
Build id: 3.10.0-1062.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc01fb71e
Build id: 3.10.0-1062.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc02322e0
Build id: 3.10.0-1062.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff914dc890
Build id: 3.10.0-1062.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff914e5efc
Build id: 3.10.0-1062.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff9134cf59
Build id: 3.10.0-1062.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff9134d01a
Build id: 3.10.0-1062.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff91350858
Build id: 3.10.0-1062.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff91350cc4
Build id: 3.10.0-1062.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff91205d31
Build id: 3.10.0-1062.el7.x86_64
|
0x191
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff911efe36
Build id: 3.10.0-1062.el7.x86_64
|
0xd66
|
|
12
|
|
vmlinux |
0xffffffff911102d8
Build id: 3.10.0-1062.el7.x86_64
|
0x1c8
|
|
13
|
handle_mm_fault
|
vmlinux |
0xffffffff911f200d
Build id: 3.10.0-1062.el7.x86_64
|
0x39d
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffff91786633
Build id: 3.10.0-1062.el7.x86_64
|
0x213
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffff91786955
Build id: 3.10.0-1062.el7.x86_64
|
0x35
|
|
16
|
page_fault
|
vmlinux |
0xffffffff91782768
Build id: 3.10.0-1062.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 33s! [java:71692]
Modules linked in: vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event crc32_pclmul ppdev ghash_clmulni_intel snd_ens1371 vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device btusb snd_pcm btrtl aesni_intel btbcm btintel bluetooth lrw gf128mul glue_helper ablk_helper cryptd snd_timer snd sg pcspkr soundcore joydev rfkill vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix libata syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm mptspi scsi_transport_spi mptscsih crc32c_intel mptbase e1000 nfit libnvdimm serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 71692 Comm: java Not tainted 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9c9f48afa0e0 ti: ffff9ca004530000 task.ti: ffff9ca004530000
RIP: 0010:[<ffffffffc01e03be>] [<ffffffffc01e03be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9ca004533a80 EFLAGS: 00000246
RAX: ffffaa53809c0000 RBX: ffff9c9f93382400 RCX: 0000000000000015
RDX: ffff9ca0329f7c00 RSI: ffff9ca032c2d000 RDI: 000000000000000e
RBP: ffff9ca004533a90 R08: 0000000000000002 R09: 000000008cfb3fff
R10: ffff9c9f3662d0d8 R11: ffff9ca00fca1f80 R12: 0000000100000001
R13: ffff9c9f1e883000 R14: ffff9ca0045339f4 R15: ffffffff913557ab
FS: 00007f760444f700(0000) GS:ffff9ca0396c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7608044fe0 CR3: 0000000054054000 CR4: 00000000003407e0
Call Trace:
[<ffffffffc01fb71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc02322e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff914dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff914e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff9134cf59>] __blk_run_queue+0x39/0x50
[<ffffffff9134d01a>] queue_unplugged+0x2a/0xa0
[<ffffffff91350858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff91350cc4>] blk_finish_plug+0x14/0x40
[<ffffffff91205d31>] swapin_readahead+0x191/0x210
[<ffffffff911efe36>] handle_pte_fault+0xd66/0xe20
[<ffffffff911102d8>] ? get_futex_key+0x1c8/0x2c0
[<ffffffff911f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff91786633>] __do_page_fault+0x213/0x500
[<ffffffff91786955>] do_page_fault+0x35/0x90
[<ffffffff91782768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 1e
Complete report #459335
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc02113be
Build id: 3.10.0-1062.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc026a71e
Build id: 3.10.0-1062.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc022c2e0
Build id: 3.10.0-1062.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff97adc890
Build id: 3.10.0-1062.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff97ae5efc
Build id: 3.10.0-1062.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff9794cf59
Build id: 3.10.0-1062.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff9794d01a
Build id: 3.10.0-1062.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff97950858
Build id: 3.10.0-1062.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff97950cc4
Build id: 3.10.0-1062.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff97805d31
Build id: 3.10.0-1062.el7.x86_64
|
0x191
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff977efe36
Build id: 3.10.0-1062.el7.x86_64
|
0xd66
|
|
12
|
handle_mm_fault
|
vmlinux |
0xffffffff977f200d
Build id: 3.10.0-1062.el7.x86_64
|
0x39d
|
|
13
|
|
vmlinux |
0xffffffff97d7eb25
Build id: 3.10.0-1062.el7.x86_64
|
0x345
|
|
14
|
|
vmlinux |
0xffffffff97d7eb31
Build id: 3.10.0-1062.el7.x86_64
|
0x351
|
|
15
|
__do_page_fault
|
vmlinux |
0xffffffff97d86633
Build id: 3.10.0-1062.el7.x86_64
|
0x213
|
|
16
|
do_page_fault
|
vmlinux |
0xffffffff97d86955
Build id: 3.10.0-1062.el7.x86_64
|
0x35
|
|
17
|
page_fault
|
vmlinux |
0xffffffff97d82768
Build id: 3.10.0-1062.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 21s! [GC Worker:6043]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 overlay(T) tun bridge stp llc devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus iosf_mbi crc32_pclmul ppdev snd_seq ghash_clmulni_intel snd_seq_device vmw_balloon snd_pcm aesni_intel lrw gf128mul
glue_helper ablk_helper sg cryptd snd_timer joydev pcspkr snd vmw_vmci i2c_piix4 soundcore parport_pc parport pcc_cpufreq ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix nfit crct10dif_pclmul crct10dif_common drm crc32c_intel mptspi libnvdimm scsi_transport_spi libata mptscsih serio_raw e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 6043 Comm: GC Worker Kdump: loaded Tainted: G L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff905556c41070 ti: ffff90554ffbc000 task.ti: ffff90554ffbc000
RIP: 0010:[<ffffffffc02113be>] [<ffffffffc02113be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff90554ffbfa80 EFLAGS: 00010246
RAX: ffffb54180980000 RBX: ffff90555f1a9000 RCX: 0000000000000009
RDX: ffff9055f164a800 RSI: ffff9055f5d21000 RDI: 000000000000000e
RBP: ffff90554ffbfa90 R08: 0000000000000002 R09: 000000010986dfff
R10: ffff9055f5d200d8 R11: ffff90552dd08380 R12: 0000000100000001
R13: ffff9055f6c33000 R14: ffff90554ffbf9f4 R15: ffffffff979557ab
FS: 00007f881e2d6640(0000) GS:ffff9055f9740000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000c6476f98 CR3: 00000000917c0000 CR4: 0000000000360fe0
Call Trace:
[<ffffffffc026a71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc022c2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff97adc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff97ae5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff9794cf59>] __blk_run_queue+0x39/0x50
[<ffffffff9794d01a>] queue_unplugged+0x2a/0xa0
[<ffffffff97950858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff97950cc4>] blk_finish_plug+0x14/0x40
[<ffffffff97805d31>] swapin_readahead+0x191/0x210
[<ffffffff977efe36>] handle_pte_fault+0xd66/0xe20
[<ffffffff977f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff97d7eb25>] ? __schedule+0x345/0x9c0
[<ffffffff97d7eb31>] ? __schedule+0x351/0x9c0
[<ffffffff97d86633>] __do_page_fault+0x213/0x500
[<ffffffff97d86955>] do_page_fault+0x35/0x90
[<ffffffff97d82768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 f3 21
Complete report #396637
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc02cc3be
Build id: 3.10.0-1062.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc031b71e
Build id: 3.10.0-1062.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc032f2e0
Build id: 3.10.0-1062.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff846dc890
Build id: 3.10.0-1062.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff846e5efc
Build id: 3.10.0-1062.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff8454cf59
Build id: 3.10.0-1062.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff8454d01a
Build id: 3.10.0-1062.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff84550858
Build id: 3.10.0-1062.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff84550cc4
Build id: 3.10.0-1062.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff84405d31
Build id: 3.10.0-1062.el7.x86_64
|
0x191
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff843efe36
Build id: 3.10.0-1062.el7.x86_64
|
0xd66
|
|
12
|
|
vmlinux |
0xffffffff842e48b8
Build id: 3.10.0-1062.el7.x86_64
|
0x148
|
|
13
|
handle_mm_fault
|
vmlinux |
0xffffffff843f200d
Build id: 3.10.0-1062.el7.x86_64
|
0x39d
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffff84986633
Build id: 3.10.0-1062.el7.x86_64
|
0x213
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffff84986955
Build id: 3.10.0-1062.el7.x86_64
|
0x35
|
|
16
|
page_fault
|
vmlinux |
0xffffffff84982768
Build id: 3.10.0-1062.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [oracle:3188]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ppdev ac97_bus iosf_mbi crc32_pclmul snd_seq ghash_clmulni_intel snd_seq_device snd_pcm vmw_balloon aesni_intel lrw gf128mul snd_timer snd glue_helper ablk_helper cryptd soundcore sg pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi
vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel nfit drm libnvdimm ata_piix libata serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3188 Comm: oracle Kdump: loaded Tainted: G L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff902c517ce2a0 ti: ffff902cf509c000 task.ti: ffff902cf509c000
RIP: 0010:[<ffffffffc02cc3be>] [<ffffffffc02cc3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff902cf509fa80 EFLAGS: 00000246
RAX: ffffad5d006c0000 RBX: ffff902c4b7ece60 RCX: 000000000000001b
RDX: ffff902cf1a5a200 RSI: ffff902bf5eb1000 RDI: 000000000000000e
RBP: ffff902cf509fa90 R08: 0000000000000003 R09: 0000000005fe7fff
R10: ffff902cf1fe10d8 R11: ffff902c076c0a80 R12: 0000000100000004
R13: ffff902cc2317180 R14: ffff902cf509f9f4 R15: ffffffff845557ab
FS: 00007f42ebab2740(0000) GS:ffff902cf9600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f42e9558478 CR3: 0000000133646000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc031b71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc032f2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff846dc890>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff846e5efc>] scsi_request_fn+0x4cc/0x680
[<ffffffff8454cf59>] __blk_run_queue+0x39/0x50
[<ffffffff8454d01a>] queue_unplugged+0x2a/0xa0
[<ffffffff84550858>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff84550cc4>] blk_finish_plug+0x14/0x40
[<ffffffff84405d31>] swapin_readahead+0x191/0x210
[<ffffffff843efe36>] handle_pte_fault+0xd66/0xe20
[<ffffffff842e48b8>] ? check_preempt_wakeup+0x148/0x250
[<ffffffff843f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff84986633>] __do_page_fault+0x213/0x500
[<ffffffff84986955>] do_page_fault+0x35/0x90
[<ffffffff84982768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 a3 2d
Complete report #320545
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc013c3be
Build id: 3.10.0-862.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc018b71e
Build id: 3.10.0-862.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc019c2e0
Build id: 3.10.0-862.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff9ee9b050
Build id: 3.10.0-862.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff9eea45bc
Build id: 3.10.0-862.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff9ed18ad9
Build id: 3.10.0-862.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff9ed18b9a
Build id: 3.10.0-862.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff9ed1c358
Build id: 3.10.0-862.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff9ed1c7c4
Build id: 3.10.0-862.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff9ebda828
Build id: 3.10.0-862.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff9ebc5777
Build id: 3.10.0-862.el7.x86_64
|
0x777
|
|
12
|
|
vmlinux |
0xffffffff9ebcf45d
Build id: 3.10.0-862.el7.x86_64
|
0x5cd
|
|
13
|
handle_mm_fault
|
vmlinux |
0xffffffff9ebc747d
Build id: 3.10.0-862.el7.x86_64
|
0x39d
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffff9f11a597
Build id: 3.10.0-862.el7.x86_64
|
0x197
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffff9f11a925
Build id: 3.10.0-862.el7.x86_64
|
0x35
|
|
16
|
|
vmlinux |
0xffffffff9f116ab6
Build id: 3.10.0-862.el7.x86_64
|
0xa7
|
|
17
|
page_fault
|
vmlinux |
0xffffffff9f116768
Build id: 3.10.0-862.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [gnome-shell:1716]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables xt_conntrack nf_nat nf_conntrack br_netfilter overlay(T) tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event edac_mce_amd crc32_pclmul ghash_clmulni_intel snd_ens1371 aesni_intel lrw snd_rawmidi gf128mul snd_ac97_codec glue_helper ablk_helper ac97_bus cryptd snd_seq snd_seq_device snd_pcm snd_timer ppdev snd soundcore nfit vmw_balloon libnvdimm pcspkr joydev parport_pc parport sg i2c_piix4 vmw_vmci shpchp xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata mptspi scsi_transport_spi serio_raw mptscsih e1000 mptbase i2c_core [last unloaded: ip_tables]
CPU: 0 PID: 1716 Comm: gnome-shell Tainted: G ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9742a41c0000 ti: ffff9742ab7b8000 task.ti: ffff9742ab7b8000
RIP: 0010:[<ffffffffc013c3be>] [<ffffffffc013c3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9742ab7bbaa8 EFLAGS: 00000246
RAX: ffffa79a80380000 RBX: ffff9742817f5288 RCX: 000000000000000a
RDX: ffff9742b5868200 RSI: ffff9742b5881000 RDI: 000000000000000e
RBP: ffff9742ab7bbab8 R08: 0000000000000003 R09: 000000003c892fff
R10: ffff9742b58860d8 R11: ffff97428a762300 R12: ffff9742b0244480
R13: ffff9742ab7bba1c R14: ffffffff9ed212cb R15: ffff9742ab7bba60
FS: 00007fdff3afbac0(0000) GS:ffff9742cc200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdfa2727000 CR3: 0000000024328000 CR4: 00000000000407f0
Call Trace:
[<ffffffffc018b71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc019c2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9ee9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9eea45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffff9ed18ad9>] __blk_run_queue+0x39/0x50
[<ffffffff9ed18b9a>] queue_unplugged+0x2a/0xa0
[<ffffffff9ed1c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff9ed1c7c4>] blk_finish_plug+0x14/0x40
[<ffffffff9ebda828>] swapin_readahead+0xc8/0x110
[<ffffffff9ebc5777>] handle_pte_fault+0x777/0xc30
[<ffffffff9ebcf45d>] ? change_protection+0x5cd/0x680
[<ffffffff9ebc747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9f11a597>] __do_page_fault+0x197/0x4f0
[<ffffffff9f11a925>] do_page_fault+0x35/0x90
[<ffffffff9f116ab6>] ? error_swapgs+0xa7/0xbd
[<ffffffff9f116768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 a3 14
Complete report #276393
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc02173be
Build id: 3.10.0-862.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc023971e
Build id: 3.10.0-862.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc01e52e0
Build id: 3.10.0-862.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffffa3c9b050
Build id: 3.10.0-862.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffffa3ca45bc
Build id: 3.10.0-862.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffffa3b18ad9
Build id: 3.10.0-862.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffffa3b18b9a
Build id: 3.10.0-862.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffffa3b1c358
Build id: 3.10.0-862.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffffa3b1c7c4
Build id: 3.10.0-862.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffffa39da828
Build id: 3.10.0-862.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffffa39c5777
Build id: 3.10.0-862.el7.x86_64
|
0x777
|
|
12
|
|
vmlinux |
0xffffffffa39033a0
Build id: 3.10.0-862.el7.x86_64
|
0x90
|
|
13
|
handle_mm_fault
|
vmlinux |
0xffffffffa39c747d
Build id: 3.10.0-862.el7.x86_64
|
0x39d
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffffa3f1a597
Build id: 3.10.0-862.el7.x86_64
|
0x197
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffffa3f1a925
Build id: 3.10.0-862.el7.x86_64
|
0x35
|
|
16
|
page_fault
|
vmlinux |
0xffffffffa3f16768
Build id: 3.10.0-862.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [java:3701]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec btusb joydev pcspkr btrtl btbcm btintel ac97_bus snd_seq bluetooth snd_seq_device snd_pcm snd_timer snd sg rfkill nfit soundcore libnvdimm vmw_vmci i2c_piix4 shpchp parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom vmwgfx ata_generic pata_acpi
sd_mod drm_kms_helper crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix serio_raw libata mptspi e1000 scsi_transport_spi mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3701 Comm: java Kdump: loaded Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9e3553949fa0 ti: ffff9e3558928000 task.ti: ffff9e3558928000
RIP: 0010:[<ffffffffc02173be>] [<ffffffffc02173be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9e355892baa8 EFLAGS: 00000246
RAX: ffffacbd40380000 RBX: ffff9e35219485a0 RCX: 000000000000001e
RDX: ffff9e3535a80400 RSI: ffff9e3535599000 RDI: 000000000000000e
RBP: ffff9e355892bab8 R08: 0000000000000003 R09: 0000000013a7dfff
R10: ffff9e353559c0d8 R11: ffff9e3577763800 R12: ffff9e3564214a80
R13: ffff9e355892ba1c R14: ffffffffa3b212cb R15: ffff9e355892ba60
FS: 00007fc1149c9700(0000) GS:ffff9e357b600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc1149c7940 CR3: 0000000078b9a000 CR4: 00000000003407f0
Call Trace:
[<ffffffffc023971e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01e52e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa3c9b050>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa3ca45bc>] scsi_request_fn+0x4cc/0x680
[<ffffffffa3b18ad9>] __blk_run_queue+0x39/0x50
[<ffffffffa3b18b9a>] queue_unplugged+0x2a/0xa0
[<ffffffffa3b1c358>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa3b1c7c4>] blk_finish_plug+0x14/0x40
[<ffffffffa39da828>] swapin_readahead+0xc8/0x110
[<ffffffffa39c5777>] handle_pte_fault+0x777/0xc30
[<ffffffffa39033a0>] ? futex_wake+0x90/0x180
[<ffffffffa39c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa3f1a597>] __do_page_fault+0x197/0x4f0
[<ffffffffa3f1a925>] do_page_fault+0x35/0x90
[<ffffffffa3f16768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 53 22
Complete report #212824
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc042d3be
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc041871e
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc047e2e0
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff8f2d1630
Build id: 3.10.0-957.12.2.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff8f2dabcc
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff8f143789
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff8f14384a
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff8f147078
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff8f1474e4
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff8efffb0b
Build id: 3.10.0-957.12.2.el7.x86_64
|
0xcb
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff8efe9be2
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x812
|
|
12
|
|
vmlinux |
0xffffffff8ef0cd58
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x1c8
|
|
13
|
handle_mm_fault
|
vmlinux |
0xffffffff8efec1fd
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x39d
|
|
14
|
__do_page_fault
|
vmlinux |
0xffffffff8f570603
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x203
|
|
15
|
do_page_fault
|
vmlinux |
0xffffffff8f570925
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x35
|
|
16
|
page_fault
|
vmlinux |
0xffffffff8f56c768
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 35s! [TaskSchedulerFo:30077]
Modules linked in: cdc_acm tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmhgfs(OE) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev pcspkr sg snd_seq_midi snd_seq_midi_event parport_pc parport snd_ens1371 snd_rawmidi snd_ac97_codec
ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi e1000 mptscsih mptbase ata_piix libata drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 30077 Comm: TaskSchedulerFo Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.12.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8e5445265140 ti: ffff8e52d3a84000 task.ti: ffff8e52d3a84000
RIP: 0010:[<ffffffffc042d3be>] [<ffffffffc042d3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8e52d3a87aa8 EFLAGS: 00000246
RAX: ffffad1cc0d40000 RBX: ffff8e525ca9b3a0 RCX: 000000000000000d
RDX: ffff8e52754d6600 RSI: ffff8e527561a000 RDI: 000000000000000e
RBP: ffff8e52d3a87ab8 R08: 0000000000000003 R09: 000000000413bfff
R10: ffff8e5275c130d8 R11: ffff8e52c02b1880 R12: ffff8e53f9a70480
R13: ffff8e52d3a87a1c R14: ffffffff8f14bfcb R15: ffff8e52d3a87a60
FS: 00007f861716b700(0000) GS:ffff8e5475600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f861d3ac9c8 CR3: 000000005adbc000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc041871e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc047e2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8f2d1630>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8f2dabcc>] scsi_request_fn+0x4cc/0x680
[<ffffffff8f143789>] __blk_run_queue+0x39/0x50
[<ffffffff8f14384a>] queue_unplugged+0x2a/0xa0
[<ffffffff8f147078>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8f1474e4>] blk_finish_plug+0x14/0x40
[<ffffffff8efffb0b>] swapin_readahead+0xcb/0x110
[<ffffffff8efe9be2>] handle_pte_fault+0x812/0xd10
[<ffffffff8ef0cd58>] ? get_futex_key+0x1c8/0x2c0
[<ffffffff8efec1fd>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8f570603>] __do_page_fault+0x203/0x4f0
[<ffffffff8f570925>] do_page_fault+0x35/0x90
[<ffffffff8f56c768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 b3 43
Complete report #203788
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffa009a3be
Build id: 3.10.0-327.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffa0080729
Build id: 3.10.0-327.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffa00be2b0
Build id: 3.10.0-327.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff81417b1a
Build id: 3.10.0-327.el7.x86_64
|
0xaa
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff81420aa1
Build id: 3.10.0-327.el7.x86_64
|
0x501
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812c73e3
Build id: 3.10.0-327.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812c749a
Build id: 3.10.0-327.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812cbd18
Build id: 3.10.0-327.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812cc124
Build id: 3.10.0-327.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff811a99c8
Build id: 3.10.0-327.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff81197512
Build id: 3.10.0-327.el7.x86_64
|
0xa42
|
|
12
|
__do_page_fault
|
vmlinux |
0xffffffff81640e22
Build id: 3.10.0-327.el7.x86_64
|
0x152
|
|
13
|
do_page_fault
|
vmlinux |
0xffffffff81641113
Build id: 3.10.0-327.el7.x86_64
|
0x23
|
|
14
|
page_fault
|
vmlinux |
0xffffffff8163d408
Build id: 3.10.0-327.el7.x86_64
|
0x28
|
BUG: soft lockup - CPU#2 stuck for 21s! [java:4033]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter fuse vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event coretemp snd_ens1371 snd_rawmidi snd_ac97_codec crc32_pclmul ghash_clmulni_intel ac97_bus aesni_intel lrw snd_seq snd_seq_device gf128mul snd_pcm snd_timer glue_helper ablk_helper snd soundcore cryptd pcspkr sg ppdev vmw_balloon parport_pc parport i2c_piix4 shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul
crct10dif_common vmwgfx drm_kms_helper ttm crc32c_intel drm serio_raw mptspi scsi_transport_spi mptscsih mptbase ata_piix libata e1000 i2c_core
CPU: 2 PID: 4033 Comm: java Tainted: G W L ------------ 3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff88022fecae00 ti: ffff88022dcac000 task.ti: ffff88022dcac000
RIP: 0010:[<ffffffffa009a3be>] [<ffffffffa009a3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88022dcafb70 EFLAGS: 00000246
RAX: ffffc90008cc0000 RBX: ffff88022dcafb40 RCX: 0000000000000012
RDX: ffff8800361d3e00 RSI: ffff88022d055000 RDI: 000000000000000e
RBP: ffff88022dcafb80 R08: 0000000000000002 R09: ffff8802326360d8
R10: ffff88022d20e300 R11: ffff88022d20e300 R12: 0000000100000001
R13: ffff880034e1c000 R14: ffff88022dcafae4 R15: ffffffff812d06bb
FS: 00007f59cfa4d700(0000) GS:ffff880236640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f59ec010ce0 CR3: 0000000231230000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff88022d055d30 000000000000003c ffff88022dcafc28 ffffffffa0080729
ffff88022d055008 040000002dcafbe8 0000000000000012 ffff88022d20e300
0000000000000048 ffff88022d055188 0000006000000012 ffff880035a82ef0
Call Trace:
[<ffffffffa0080729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffa00be2b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff81417b1a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff81420aa1>] scsi_request_fn+0x501/0x770
[<ffffffff812c73e3>] __blk_run_queue+0x33/0x40
[<ffffffff812c749a>] queue_unplugged+0x2a/0xa0
[<ffffffff812cbd18>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812cc124>] blk_finish_plug+0x14/0x40
[<ffffffff811a99c8>] swapin_readahead+0xc8/0x110
[<ffffffff81197512>] handle_mm_fault+0xa42/0xf50
[<ffffffff81640e22>] __do_page_fault+0x152/0x420
[<ffffffff81641113>] do_page_fault+0x23/0x80
[<ffffffff8163d408>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 83 0a
Complete report #170420
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc00503be
Build id: 3.10.0-693.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc0095729
Build id: 3.10.0-693.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc00492b0
Build id: 3.10.0-693.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64
|
0xaa
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64
|
0x4df
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff811c4678
Build id: 3.10.0-693.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff811b235b
Build id: 3.10.0-693.el7.x86_64
|
0xadb
|
|
12
|
|
vmlinux |
0xffffffff811df72c
Build id: 3.10.0-693.el7.x86_64
|
0x3c
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
|
0x154
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
|
0x35
|
|
15
|
page_fault
|
vmlinux |
0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [vmtoolsd:2423]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg nfit snd_seq_midi snd_seq_midi_event libnvdimm snd_ens1371 snd_rawmidi snd_ac97_codec parport_pc parport ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore i2c_piix4 vmw_vmci shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix libata serio_raw mptspi scsi_transport_spi e1000 mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 2423 Comm: vmtoolsd Not tainted 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff880012bd3f40 ti: ffff880010c28000 task.ti: ffff880010c28000
RIP: 0010:[<ffffffffc00503be>] [<ffffffffc00503be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff880010c2bb58 EFLAGS: 00000246
RAX: ffffc90000200000 RBX: ffff8800050f22a0 RCX: 0000000000000002
RDX: ffff880036250600 RSI: ffff8800358d4000 RDI: 000000000000000e
RBP: ffff880010c2bb68 R08: 0000000000000003 R09: 000000002e3e7fff
R10: ffff8800225e3800 R11: ffff8800225e3800 R12: ffff880008769680
R13: ffff880010c2bacc R14: ffffffff812ff13b R15: ffff880010c2bb10
FS: 00007fb3b7f05880(0000) GS:ffff88003c600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb3b7d082d9 CR3: 0000000010cff000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800358d4d30 0000000000000050 ffff880010c2bc10 ffffffffc0095729
ffff8800358d4008 0000000010c2bbd0 0000000000000002 ffff8800225e3800
0000000000000008 ffff8800358d4188 0000006000000002 ffff8800363c28f0
Call Trace:
[<ffffffffc0095729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00492b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffff811c4678>] swapin_readahead+0xc8/0x110
[<ffffffff811b235b>] handle_mm_fault+0xadb/0xfa0
[<ffffffff811df72c>] ? kmem_cache_alloc_trace+0x3c/0x200
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 05
Complete report #122920
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc00503be
Build id: 3.10.0-693.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc006b729
Build id: 3.10.0-693.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc00492b0
Build id: 3.10.0-693.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64
|
0xaa
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64
|
0x4df
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff811c4678
Build id: 3.10.0-693.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff811b235b
Build id: 3.10.0-693.el7.x86_64
|
0xadb
|
|
12
|
|
vmlinux |
0xffffffff810b1671
Build id: 3.10.0-693.el7.x86_64
|
0x31
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
|
0x154
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
|
0x35
|
|
15
|
page_fault
|
vmlinux |
0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 32s! [java:7860]
Modules linked in: binfmt_misc tcp_lp fuse vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit vmw_balloon pcspkr joydev libnvdimm sg parport_pc parport shpchp i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix e1000 libata mptspi scsi_transport_spi mptscsih serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 7860 Comm: java Tainted: G L ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff880131db2f70 ti: ffff88009050c000 task.ti: ffff88009050c000
RIP: 0010:[<ffffffffc00503be>] [<ffffffffc00503be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88009050fb58 EFLAGS: 00000246
RAX: ffffc90000700000 RBX: ffff880131e0e850 RCX: 0000000000000000
RDX: ffff880035681000 RSI: ffff8800356d2000 RDI: 000000000000000e
RBP: ffff88009050fb68 R08: 0000000000000003 R09: ffff880131c7a0d8
R10: ffff8800b3d4b2c0 R11: ffff8800b3d4b2c0 R12: ffff880090710180
R13: ffff88009050facc R14: ffffffff812ff13b R15: ffff88009050fb10
FS: 00007f9d68e52700(0000) GS:ffff880139680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000ee3f7000 CR3: 00000000b3d19000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800356d2d30 0000000000000024 ffff88009050fc10 ffffffffc006b729
ffff8800356d2008 000000009050fbd0 0000000000000000 ffff8800b3d4b2c0
0000000000000000 ffff8800356d2188 0000006000000000 ffff880034882830
Call Trace:
[<ffffffffc006b729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00492b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffff811c4678>] swapin_readahead+0xc8/0x110
[<ffffffff811b235b>] handle_mm_fault+0xadb/0xfa0
[<ffffffff810b1671>] ? remove_wait_queue+0x31/0x40
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 e3 05
Complete report #24743
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc041e3be
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc044071e
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc04d02e0
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff8a4a0140
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff8a4a96ac
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff8a31d5e9
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff8a31d6aa
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff8a320e68
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff8a3212d4
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff8a1de0e8
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff8a1c8a62
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x812
|
|
12
|
handle_mm_fault
|
vmlinux |
0xffffffff8a1caefd
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x39d
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffff8a720547
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x197
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffff8a7208d5
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x35
|
|
15
|
page_fault
|
vmlinux |
0xffffffff8a71c758
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 23s! [rtkit-daemon:999]
Modules linked in: uas usb_storage tcp_lp nls_utf8 isofs fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack 8021q garp mrp ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi
aesni_intel snd_ac97_codec lrw gf128mul glue_helper ablk_helper ac97_bus cryptd ppdev snd_seq vmw_balloon snd_seq_device snd_pcm snd_timer snd nfit soundcore pcspkr joydev sg libnvdimm parport_pc parport vmw_vmci shpchp i2c_piix4 sch_fq_codel ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel mptspi libata scsi_transport_spi e1000 mptscsih serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 999 Comm: rtkit-daemon Kdump: loaded Tainted: G OEL ------------ 3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8cab774b3f40 ti: ffff8cab7d57c000 task.ti: ffff8cab7d57c000
RIP: 0010:[<ffffffffc041e3be>] [<ffffffffc041e3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8cab7d57faa8 EFLAGS: 00000246
RAX: ffffab9b86600000 RBX: ffff8ca1473ed1a0 RCX: 0000000000000022
RDX: ffff8cab79397200 RSI: ffff8cab790e4000 RDI: 000000000000000e
RBP: ffff8cab7d57fab8 R08: 0000000000000003 R09: 00000001dab07fff
R10: ffff8c9ce92450d8 R11: ffff8ca8e3f3ebc0 R12: ffff8cab0da82d00
R13: ffff8cab7d57fa1c R14: ffffffff8a325ddb R15: ffff8cab7d57fa60
FS: 00007f07f451c700(0000) GS:ffff8cab7d840000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f07f7634780 CR3: 0000000fecb0a000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc044071e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc04d02e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff8a4a0140>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff8a4a96ac>] scsi_request_fn+0x4cc/0x680
[<ffffffff8a31d5e9>] __blk_run_queue+0x39/0x50
[<ffffffff8a31d6aa>] queue_unplugged+0x2a/0xa0
[<ffffffff8a320e68>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff8a3212d4>] blk_finish_plug+0x14/0x40
[<ffffffff8a1de0e8>] swapin_readahead+0xc8/0x110
[<ffffffff8a1c8a62>] handle_pte_fault+0x812/0xd10
[<ffffffff8a1caefd>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8a720547>] __do_page_fault+0x197/0x4f0
[<ffffffff8a7208d5>] do_page_fault+0x35/0x90
[<ffffffff8a71c758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 c3 42
Complete report #19386
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc036b3be
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc03e171e
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc03862e0
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffffba6d0fe0
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffffba6da57c
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffffba5435d9
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffffba54369a
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffffba546ec8
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffffba547334
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffffba3ff92b
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xcb
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffffba3e9a02
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x812
|
|
12
|
handle_mm_fault
|
vmlinux |
0xffffffffba3ec01d
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x39d
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffffba96f5e3
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x203
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffffba96f915
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x35
|
|
15
|
|
vmlinux |
0xffffffffba96ba96
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xa7
|
|
16
|
page_fault
|
vmlinux |
0xffffffffba96b758
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 107s! [miniserv.pl:9050]
Modules linked in: fuse xt_CHECKSUM xt_set xt_multiport ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip_set_hash_ip devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc coretemp crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd snd_seq_midi vmw_balloon snd_seq_midi_event pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec joydev ac97_bus
snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg floppy parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw pata_acpi nfit vmwgfx libnvdimm e1000 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih libata mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 9050 Comm: miniserv.pl Kdump: loaded Tainted: G L ------------ 3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8b9c51b00000 ti: ffff8b9bc89d0000 task.ti: ffff8b9bc89d0000
RIP: 0010:[<ffffffffc036b3be>] [<ffffffffc036b3be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8b9bc89d3aa8 EFLAGS: 00000246
RAX: ffffb7f640680000 RBX: ffff8b9c31974ea0 RCX: 0000000000000011
RDX: ffff8b9bf5694600 RSI: ffff8b9bf55c3000 RDI: 000000000000000e
RBP: ffff8b9bc89d3ab8 R08: 0000000000000003 R09: 0000000046eb2fff
R10: ffff8b9bf55350d8 R11: ffff8b9c4932ed80 R12: ffff8b9bc5d74a80
R13: ffff8b9bc89d3a1c R14: ffffffffba54be1b R15: ffff8b9bc89d3a60
FS: 00007f3a1f0e0740(0000) GS:ffff8b9c7b440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001812ee8 CR3: 0000000006b2e000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc03e171e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc03862e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffba6d0fe0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffba6da57c>] scsi_request_fn+0x4cc/0x680
[<ffffffffba5435d9>] __blk_run_queue+0x39/0x50
[<ffffffffba54369a>] queue_unplugged+0x2a/0xa0
[<ffffffffba546ec8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffba547334>] blk_finish_plug+0x14/0x40
[<ffffffffba3ff92b>] swapin_readahead+0xcb/0x110
[<ffffffffba3e9a02>] handle_pte_fault+0x812/0xd10
[<ffffffffba3ec01d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffba96f5e3>] __do_page_fault+0x203/0x500
[<ffffffffba96f915>] do_page_fault+0x35/0x90
[<ffffffffba96ba96>] ? error_swapgs+0xa7/0xbd
[<ffffffffba96b758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 93 37
Complete report #13344
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc01433be
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc016571e
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc00f82e0
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff9aa9aff0
Build id: 3.10.0-862.2.3.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff9aaa455c
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff9a918a79
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff9a918b3a
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff9a91c2f8
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff9a91c764
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff9a7da7c8
Build id: 3.10.0-862.2.3.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff9a7c5717
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x777
|
|
12
|
handle_mm_fault
|
vmlinux |
0xffffffff9a7c741d
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x39d
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffff9ad1a557
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x197
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffff9ad1a8e5
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x35
|
|
15
|
|
vmlinux |
0xffffffff9ad16aa6
Build id: 3.10.0-862.2.3.el7.x86_64
|
0xa7
|
|
16
|
page_fault
|
vmlinux |
0xffffffff9ad16758
Build id: 3.10.0-862.2.3.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [/opt/zimbra/com:499]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev parport_pc nfit parport libnvdimm sg shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw e1000 mptspi scsi_transport_spi mptscsih mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 499 Comm: /opt/zimbra/com Kdump: loaded Tainted: G W L ------------ 3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9dea5e749fa0 ti: ffff9dea6bb2c000 task.ti: ffff9dea6bb2c000
RIP: 0010:[<ffffffffc01433be>] [<ffffffffc01433be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff9dea6bb2faa8 EFLAGS: 00000246
RAX: ffffba7700e80000 RBX: ffff9de8e68734a0 RCX: 0000000000000001
RDX: ffff9de8f5c04e00 RSI: ffff9dea752cc000 RDI: 000000000000000e
RBP: ffff9dea6bb2fab8 R08: 0000000000000003 R09: 0000000113f95fff
R10: ffff9dea752c80d8 R11: ffff9de902f63500 R12: ffff9de906725800
R13: ffff9dea6bb2fa1c R14: ffffffff9a92126b R15: ffff9dea6bb2fa60
FS: 00007fbe58da7740(0000) GS:ffff9dea7fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000005834786 CR3: 00000001a0312000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc016571e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc00f82e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff9aa9aff0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff9aaa455c>] scsi_request_fn+0x4cc/0x680
[<ffffffff9a918a79>] __blk_run_queue+0x39/0x50
[<ffffffff9a918b3a>] queue_unplugged+0x2a/0xa0
[<ffffffff9a91c2f8>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff9a91c764>] blk_finish_plug+0x14/0x40
[<ffffffff9a7da7c8>] swapin_readahead+0xc8/0x110
[<ffffffff9a7c5717>] handle_pte_fault+0x777/0xc30
[<ffffffff9a7c741d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9ad1a557>] __do_page_fault+0x197/0x4f0
[<ffffffff9ad1a8e5>] do_page_fault+0x35/0x90
[<ffffffff9ad16aa6>] ? error_swapgs+0xa7/0xbd
[<ffffffff9ad16758>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 13 15
Complete report #10251
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc00813be
Build id: 3.10.0-693.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc00b9729
Build id: 3.10.0-693.el7.x86_64
|
0x249
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc00a12b0
Build id: 3.10.0-693.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff8146540a
Build id: 3.10.0-693.el7.x86_64
|
0xaa
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff8146e1cf
Build id: 3.10.0-693.el7.x86_64
|
0x4df
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff812f71d3
Build id: 3.10.0-693.el7.x86_64
|
0x33
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff812f728a
Build id: 3.10.0-693.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff812fa718
Build id: 3.10.0-693.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff812fab24
Build id: 3.10.0-693.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff811c4678
Build id: 3.10.0-693.el7.x86_64
|
0xc8
|
|
11
|
handle_mm_fault
|
vmlinux |
0xffffffff811b235b
Build id: 3.10.0-693.el7.x86_64
|
0xadb
|
|
12
|
__do_page_fault
|
vmlinux |
0xffffffff816affb4
Build id: 3.10.0-693.el7.x86_64
|
0x154
|
|
13
|
do_page_fault
|
vmlinux |
0xffffffff816b02e5
Build id: 3.10.0-693.el7.x86_64
|
0x35
|
|
14
|
page_fault
|
vmlinux |
0xffffffff816ac508
Build id: 3.10.0-693.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [4_scheduler:1512]
Modules linked in: nls_utf8 isofs fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 iosf_mbi snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq aesni_intel ppdev lrw gf128mul glue_helper ablk_helper snd_seq_device cryptd vmw_balloon snd_pcm pcspkr snd_timer
joydev snd soundcore nfit libnvdimm parport_pc parport sg vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih mptbase crc32c_intel e1000 serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 1512 Comm: 4_scheduler Tainted: G L ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff88005fbceeb0 ti: ffff88006240c000 task.ti: ffff88006240c000
RIP: 0010:[<ffffffffc00813be>] [<ffffffffc00813be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff88006240fb58 EFLAGS: 00000246
RAX: ffffc90000380000 RBX: ffff880018b4f6a0 RCX: 0000000000000015
RDX: ffff880035beba00 RSI: ffff880035bfe000 RDI: 000000000000000e
RBP: ffff88006240fb68 R08: 0000000000000003 R09: 00000000390b1fff
R10: ffff88003595ea00 R11: ffff88003595ea00 R12: ffff88005dde5500
R13: ffff88006240facc R14: ffffffff812ff13b R15: ffff88006240fb10
FS: 00007f77939bd700(0000) GS:ffff88007b6c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7819287490 CR3: 0000000076f81000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880035bfed30 0000000000000018 ffff88006240fc10 ffffffffc00b9729
ffff880035bfe008 000000006240fbd0 0000000000000015 ffff88003595ea00
0000000000000054 ffff880035bfe188 0000006000000015 ffff880034fc3010
Call Trace:
[<ffffffffc00b9729>] mptscsih_qcmd+0x249/0x820 [mptscsih]
[<ffffffffc00a12b0>] mptspi_qcmd+0x50/0xe0 [mptspi]
[<ffffffff8146540a>] scsi_dispatch_cmd+0xaa/0x230
[<ffffffff8146e1cf>] scsi_request_fn+0x4df/0x680
[<ffffffff812f71d3>] __blk_run_queue+0x33/0x40
[<ffffffff812f728a>] queue_unplugged+0x2a/0xa0
[<ffffffff812fa718>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff812fab24>] blk_finish_plug+0x14/0x40
[<ffffffff811c4678>] swapin_readahead+0xc8/0x110
[<ffffffff811b235b>] handle_mm_fault+0xadb/0xfa0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 f3 08
Complete report #6243
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc01c33be
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc01e671e
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc01de2e0
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffff960d1700
Build id: 3.10.0-957.21.3.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffff960dac9c
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffff95f43849
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffff95f4390a
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffff95f47148
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffff95f475b4
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffff95dffb8b
Build id: 3.10.0-957.21.3.el7.x86_64
|
0xcb
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffff95de9c62
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x812
|
|
12
|
handle_mm_fault
|
vmlinux |
0xffffffff95dec27d
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x39d
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffff96370603
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x203
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffff96370925
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x35
|
|
15
|
page_fault
|
vmlinux |
0xffffffff9636c768
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 21s! [in:imjournal:1927]
Modules linked in: tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon pcspkr sg joydev vmw_vmci i2c_piix4 parport_pc parport
ip_tables xfs libcrc32c vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper ata_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm pata_acpi drm nfit mptspi scsi_transport_spi e1000 mptscsih ata_piix libnvdimm libata crct10dif_pclmul crct10dif_common mptbase crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 1927 Comm: in:imjournal Kdump: loaded Tainted: G W L ------------ 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff94c9355db0c0 ti: ffff94cd1aea8000 task.ti: ffff94cd1aea8000
RIP: 0010:[<ffffffffc01c33be>] [<ffffffffc01c33be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff94cd1aeabaa8 EFLAGS: 00000246
RAX: ffffaaa401c00000 RBX: ffff94cca36e7bd0 RCX: 0000000000000025
RDX: ffff94cd2a306800 RSI: ffff94cd2acef000 RDI: 000000000000000e
RBP: ffff94cd1aeabab8 R08: 0000000000000003 R09: 000000041cf18fff
R10: ffff94cd2afc50d8 R11: ffff94cb363af640 R12: ffff94ca9fa7d080
R13: ffff94cd1aeaba1c R14: ffffffff95f4c09b R15: ffff94cd1aeaba60
FS: 00007f2bab1a2700(0000) GS:ffff94cd3fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2ba47c76e0 CR3: 000000042577a000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc01e671e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc01de2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffff960d1700>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffff960dac9c>] scsi_request_fn+0x4cc/0x680
[<ffffffff95f43849>] __blk_run_queue+0x39/0x50
[<ffffffff95f4390a>] queue_unplugged+0x2a/0xa0
[<ffffffff95f47148>] blk_flush_plug_list+0x1d8/0x230
[<ffffffff95f475b4>] blk_finish_plug+0x14/0x40
[<ffffffff95dffb8b>] swapin_readahead+0xcb/0x110
[<ffffffff95de9c62>] handle_pte_fault+0x812/0xd10
[<ffffffff95dec27d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff96370603>] __do_page_fault+0x203/0x4f0
[<ffffffff96370925>] do_page_fault+0x35/0x90
[<ffffffff9636c768>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 13 1d
Complete report #4579
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mpt_put_msg_frame
|
mptbase |
0xffffffffc02023be
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x5e
|
|
2
|
mptscsih_qcmd
|
mptscsih |
0xffffffffc021d71e
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x24e
|
|
3
|
mptspi_qcmd
|
mptspi |
0xffffffffc026c2e0
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x50
|
|
4
|
scsi_dispatch_cmd
|
vmlinux |
0xffffffffa689b4f0
Build id: 3.10.0-862.3.3.el7.x86_64
|
0xb0
|
|
5
|
scsi_request_fn
|
vmlinux |
0xffffffffa68a4a5c
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x4cc
|
|
6
|
__blk_run_queue
|
vmlinux |
0xffffffffa6718f19
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x39
|
|
7
|
queue_unplugged
|
vmlinux |
0xffffffffa6718fda
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x2a
|
|
8
|
blk_flush_plug_list
|
vmlinux |
0xffffffffa671c798
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x1d8
|
|
9
|
blk_finish_plug
|
vmlinux |
0xffffffffa671cc04
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x14
|
|
10
|
swapin_readahead
|
vmlinux |
0xffffffffa65dab68
Build id: 3.10.0-862.3.3.el7.x86_64
|
0xc8
|
|
11
|
handle_pte_fault
|
vmlinux |
0xffffffffa65c5ab7
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x777
|
|
12
|
handle_mm_fault
|
vmlinux |
0xffffffffa65c77bd
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x39d
|
|
13
|
__do_page_fault
|
vmlinux |
0xffffffffa6b1b5c7
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x197
|
|
14
|
do_page_fault
|
vmlinux |
0xffffffffa6b1b955
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x35
|
|
15
|
page_fault
|
vmlinux |
0xffffffffa6b17788
Build id: 3.10.0-862.3.3.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 44s! [rtkit-daemon:5444]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr sg nfit parport_pc parport libnvdimm vmw_vmci i2c_piix4 shpchp ip_tables xfs libcrc32c
sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptspi e1000 scsi_transport_spi ahci mptscsih libahci mptbase vmwgfx ata_generic pata_acpi drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 5444 Comm: rtkit-daemon Kdump: loaded Tainted: G L ------------ 3.10.0-862.3.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
task: ffff8a8e76b30fd0 ti: ffff8a8e76b3c000 task.ti: ffff8a8e76b3c000
RIP: 0010:[<ffffffffc02023be>] [<ffffffffc02023be>] mpt_put_msg_frame+0x5e/0x80 [mptbase]
RSP: 0000:ffff8a8e76b3faa8 EFLAGS: 00000246
RAX: ffffb17680400000 RBX: ffff8a8e7acfdea0 RCX: 0000000000000004
RDX: ffff8a8e35c1ea00 RSI: ffff8a8e35f6c000 RDI: 000000000000000e
RBP: ffff8a8e76b3fab8 R08: 0000000000000003 R09: 000000004ff7bfff
R10: ffff8a8e35f6d0d8 R11: ffff8a8e7af90a80 R12: ffff8a8e61cc4c00
R13: ffff8a8e76b3fa1c R14: ffffffffa672170b R15: ffff8a8e76b3fa60
FS: 00007f0c0ddc2700(0000) GS:ffff8a8e7fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0c10edf2b0 CR3: 000000007a3cc000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc021d71e>] mptscsih_qcmd+0x24e/0x830 [mptscsih]
[<ffffffffc026c2e0>] mptspi_qcmd+0x50/0xf0 [mptspi]
[<ffffffffa689b4f0>] scsi_dispatch_cmd+0xb0/0x240
[<ffffffffa68a4a5c>] scsi_request_fn+0x4cc/0x680
[<ffffffffa6718f19>] __blk_run_queue+0x39/0x50
[<ffffffffa6718fda>] queue_unplugged+0x2a/0xa0
[<ffffffffa671c798>] blk_flush_plug_list+0x1d8/0x230
[<ffffffffa671cc04>] blk_finish_plug+0x14/0x40
[<ffffffffa65dab68>] swapin_readahead+0xc8/0x110
[<ffffffffa65c5ab7>] handle_pte_fault+0x777/0xc30
[<ffffffffa65c77bd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa6b1b5c7>] __do_page_fault+0x197/0x4f0
[<ffffffffa6b1b955>] do_page_fault+0x35/0x90
[<ffffffffa6b17788>] page_fault+0x28/0x30
Code: 8b 96 68 01 00 00 0f b7 c8 44 03 a6 b0 01 00 00 44 8b 04 8a 45 09 c4 f6 86 e0 00 00 00 04 75 10 48 8b 83 e8 00 00 00 44 89 60 40 <5b> 41 5c 5d c3 48 8d 76 08 0f b7 c8 44 89 e2 48 c7 c7 78 03 21