Info

Permalink
Function
mpt_fault_reset_work
First occurrence
2020-09-25
Last occurrence
2023-01-04
State
NEW
Components
kernel
Quality
-19

Statistics

Operating system Count
CentOS 7 511
Architecture Count
x86_64 511
Related packages Count
kernel 511
 0:3.10.0-1160.31.1.el7 52
 0:3.10.0-1160.62.1.el7 50
 0:3.10.0-957.el7 40
 0:3.10.0-1160.el7 37
 0:3.10.0-1062.9.1.el7 33
 0:3.10.0-1062.el7 33
 0:3.10.0-1160.42.2.el7 28
 0:3.10.0-957.12.2.el7 27
 0:3.10.0-862.el7 26
 0:3.10.0-1127.19.1.el7 24
 0:3.10.0-1160.36.2.el7 23
 0:3.10.0-1160.6.1.el7 14
 0:3.10.0-957.27.2.el7 13
 0:3.10.0-1160.88.1.el7 13
 0:3.10.0-1160.71.1.el7 12
 0:3.10.0-1127.el7 11
 0:3.10.0-1160.45.1.el7 10
 0:3.10.0-957.10.1.el7 8
 0:3.10.0-1160.49.1.el7 6
 0:3.10.0-1062.18.1.el7 5
 0:3.10.0-1160.76.1.el7 5
 0:3.10.0-1160.81.1.el7 5
 0:3.10.0-1127.18.2.el7 4
 0:3.10.0-957.1.3.el7 4
 0:3.10.0-957.21.3.el7 4
 0:3.10.0-1160.11.1.el7 3
 0:3.10.0-1160.15.2.el7 3
 0:3.10.0-957.5.1.el7 2
 0:3.10.0-1160.99.1.el7 2
 0:3.10.0-1160.53.1.el7 2
 0:3.10.0-1160.95.1.el7 2
 0:3.10.0-1062.12.1.el7 1
 0:3.10.0-1062.1.2.el7 1
 0:3.10.0-1127.10.1.el7 1
 0:3.10.0-1062.4.1.el7 1
 0:3.10.0-1160.92.1.el7 1
 0:3.10.0-1160.25.1.el7 1
 0:3.10.0-1160.21.1.el7 1
 0:3.10.0-957.12.1.el7 1
 0:3.10.0-1160.66.1.el7 1
 0:3.10.0-1160.108.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #17031
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc040ffa9
Build id: 3.10.0-957.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffffa356770f
Build id: 3.10.0-957.el7.x86_64
0x3ff
3
process_one_work
vmlinux 0xffffffffa2eb9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffa2ebade6
Build id: 3.10.0-957.el7.x86_64
0x126
5
? manage_workers.isra.25
vmlinux 0xffffffffa2ebacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffa2ec1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffa2ec1b60
Build id: 3.10.0-957.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffa3574c37
Build id: 3.10.0-957.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffa2ec1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [kworker/u256:0:6]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock bnep sunrpc coretemp snd_seq_midi iosf_mbi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon pcspkr snd_ens1371 btusb btrtl btbcm btintel snd_rawmidi snd_ac97_codec bluetooth ac97_bus snd_seq snd_seq_device snd_pcm rfkill snd_timer snd joydev soundcore sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif
crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx ata_generic pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata e1000 mptspi scsi_transport_spi mptscsih drm_panel_orientation_quirks mptbase nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 6 Comm: kworker/u256:0 Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8ae677c65140 ti: ffff8ae677d44000 task.ti: ffff8ae677d44000
RIP: 0010:[<ffffffffc040ffa9>]  [<ffffffffc040ffa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8ae677d47d90  EFLAGS: 00000202
RAX: ffffa621431c0000 RBX: ffff8ae535fc9f00 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff8ae9a79f3d60
RBP: ffff8ae677d47e18 R08: ffff8ae9a79f3d68 R09: d458ec6cb21f3d60
R10: d458ec6cb21f3d60 R11: 0000000000000000 R12: ffff8ae677d47d50
R13: 0000000000000001 R14: ffff8ae9ab65ac00 R15: ffff8ae677c651a8
FS:  0000000000000000(0000) GS:ffff8ae9ab640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8a7f239024 CR3: 00000003b7e12000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa356770f>] ? __schedule+0x3ff/0x890
[<ffffffffa2eb9d4f>] process_one_work+0x17f/0x440
[<ffffffffa2ebade6>] worker_thread+0x126/0x3c0
[<ffffffffa2ebacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa2ec1c31>] kthread+0xd1/0xe0
[<ffffffffa2ec1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa3574c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa2ec1b60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #926214
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02371a9
Build id: 3.10.0-1160.108.1.el7.x86_64
0xd9
2
? __switch_to
vmlinux 0xffffffff8902c5ee
Build id: 3.10.0-1160.108.1.el7.x86_64
0xce
3
process_one_work
vmlinux 0xffffffff890c32ef
Build id: 3.10.0-1160.108.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff890c4436
Build id: 3.10.0-1160.108.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff890c4310
Build id: 3.10.0-1160.108.1.el7.x86_64
0x2b0
6
kthread
vmlinux 0xffffffff890cb621
Build id: 3.10.0-1160.108.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff890cb550
Build id: 3.10.0-1160.108.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff897c51f7
Build id: 3.10.0-1160.108.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff890cb550
Build id: 3.10.0-1160.108.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [kworker/u256:2:16256]
Modules linked in: tcp_lp bmhook(OE) tmhook(OE) dsa_filter(POE) dsa_filter_hook(OE) binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_hda_codec_generic snd_hda_intel snd_hda_codec sb_edac iosf_mbi snd_hda_core crc32_pclmul snd_hwdep ghash_clmulni_intel snd_seq ppdev snd_seq_device aesni_intel snd_pcm lrw gf128mul glue_helper snd_timer vmw_balloon ablk_helper cryptd snd pcspkr joydev sg soundcore
parport_pc parport vmw_vmci i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper nfit syscopyarea sysfillrect sysimgblt libnvdimm fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel drm libata mptspi e1000 scsi_transport_spi mptscsih serio_raw mptbase drm_panel_orientation_quirks fuse [last unloaded: bmhook]
CPU: 1 PID: 16256 Comm: kworker/u256:2 Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-1160.108.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff958ab2dfe300 ti: ffff958aad320000 task.ti: ffff958aad320000
RIP: 0010:[<ffffffffc02371a9>]  [<ffffffffc02371a9>] mpt_fault_reset_work+0xd9/0x48a [mptbase]
RSP: 0018:ffff958aad323d90  EFLAGS: 00000202
RAX: ffffae99c0cc0000 RBX: ffff958ab2dfe368 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff958bb107ad60
RBP: ffff958aad323e18 R08: ffff958bb107ad68 R09: a9d0d16cfb87ad60
R10: a9d0d16cfb87ad60 R11: ffff958b650dc6c0 R12: ffff958aad323d28
R13: 00003ee9c2de2a27 R14: ffffffff890eab04 R15: ffff958aad323d28
FS:  0000000000000000(0000) GS:ffff958bb5640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f597c4bb5a0 CR3: 00000002346f2000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8902c5ee>] ? __switch_to+0xce/0x590
[<ffffffff890c32ef>] process_one_work+0x17f/0x440
[<ffffffff890c4436>] worker_thread+0x126/0x3c0
[<ffffffff890c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff890cb621>] kthread+0xd1/0xe0
[<ffffffff890cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff897c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff890cb550>] ? insert_kthread_work+0x40/0x40
Code: 85 3f 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #913194
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02b8fa9
Build id: 3.10.0-1160.el7.x86_64
0xc9
2
process_one_work
vmlinux 0xffffffffab4bdc4f
Build id: 3.10.0-1160.el7.x86_64
0x17f
3
worker_thread
vmlinux 0xffffffffab4bed66
Build id: 3.10.0-1160.el7.x86_64
0x126
4
? manage_workers.isra.26
vmlinux 0xffffffffab4bec40
Build id: 3.10.0-1160.el7.x86_64
0x2a0
5
kthread
vmlinux 0xffffffffab4c5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
6
? insert_kthread_work
vmlinux 0xffffffffab4c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
7
ret_from_fork_nospec_begin
vmlinux 0xffffffffabb93ddd
Build id: 3.10.0-1160.el7.x86_64
0x7
8
? insert_kthread_work
vmlinux 0xffffffffab4c5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 267s! [kworker/u256:0:38480]
Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm dns_resolver ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc kvm_intel kvm irqbypass nfit libnvdimm crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr ppdev sg vmw_balloon parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata crc32c_intel e1000 mptspi serio_raw scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 38480 Comm: kworker/u256:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff889ca39a4200 ti: ffff889dcc844000 task.ti: ffff889dcc844000
RIP: 0010:[<ffffffffc02b8fa9>]  [<ffffffffc02b8fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff889dcc847d90  EFLAGS: 00000202
RAX: ffffb65381c00000 RBX: 0000000000000202 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff88a0ac60ed60
RBP: ffff889dcc847e18 R08: ffff88a0ac60ed68 R09: dd7cd71cf6e0ed60
R10: dd7cd71cf6e0ed60 R11: 0000000000000000 R12: ffffffffab5ca550
R13: ffffffffab4bbf2b R14: ffff889dcc847d28 R15: ffffffffab4aecb8
FS:  0000000000000000(0000) GS:ffff88a0ad700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff685639000 CR3: 0000000024568000 CR4: 00000000003607e0
Call Trace:
[<ffffffffab4bdc4f>] process_one_work+0x17f/0x440
[<ffffffffab4bed66>] worker_thread+0x126/0x3c0
[<ffffffffab4bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffab4c5c21>] kthread+0xd1/0xe0
[<ffffffffab4c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffffabb93ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffab4c5b50>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #801661
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0344fa9
Build id: 3.10.0-957.el7.x86_64
0xc9
2
? x2apic_send_IPI_mask
vmlinux 0xffffffffa1c61253
Build id: 3.10.0-957.el7.x86_64
0x13
3
? try_to_wake_up
vmlinux 0xffffffffa1cd64f0
Build id: 3.10.0-957.el7.x86_64
0x190
4
process_one_work
vmlinux 0xffffffffa1cb9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffffa1cbade6
Build id: 3.10.0-957.el7.x86_64
0x126
6
? manage_workers.isra.25
vmlinux 0xffffffffa1cbacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffffa1cc1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffffa1cc1b60
Build id: 3.10.0-957.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffa2374c37
Build id: 3.10.0-957.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffa1cc1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 62s! [kworker/u256:0:95926]
Modules linked in: vmw_vsock_vmci_transport vsock vfat fat sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport tpm_crb vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm e1000e ahci ata_piix mptsas libahci drm scsi_transport_sas mptscsih ptp libata mptbase pps_core nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 95926 Comm: kworker/u256:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.17369862.B64.2012240522 12/24/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8b8b4f25d140 ti: ffff8b88fe5dc000 task.ti: ffff8b88fe5dc000
RIP: 0010:[<ffffffffc0344fa9>]  [<ffffffffc0344fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8b88fe5dfd90  EFLAGS: 00000202
RAX: ffff9b60c31b8000 RBX: ffff8b88fe5dfda8 RCX: 0000000000000008
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff8b90bed7fd60
RBP: ffff8b88fe5dfe18 R08: ffffffffa2413f08 R09: 0000000000000002
R10: 0000000014fac000 R11: 0000000000001800 R12: ffff8b88fe5dfd50
R13: 0000000000000001 R14: ffff8b90bd79ac00 R15: ffff8b8b4f25d1a8
FS:  0000000000000000(0000) GS:ffff8b90bd600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f347c002770 CR3: 000000082be10000 CR4: 00000000001607f0
Call Trace:
[<ffffffffa1c61253>] ? x2apic_send_IPI_mask+0x13/0x20
[<ffffffffa1cd64f0>] ? try_to_wake_up+0x190/0x390
[<ffffffffa1cb9d4f>] process_one_work+0x17f/0x440
[<ffffffffa1cbade6>] worker_thread+0x126/0x3c0
[<ffffffffa1cbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa1cc1c31>] kthread+0xd1/0xe0
[<ffffffffa1cc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa2374c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa1cc1b60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #790865
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0123fa9
Build id: 3.10.0-1160.el7.x86_64
0xc9
2
? sched_clock_cpu
vmlinux 0xffffffff9ecde185
Build id: 3.10.0-1160.el7.x86_64
0x85
3
? try_to_wake_up
vmlinux 0xffffffff9ecda830
Build id: 3.10.0-1160.el7.x86_64
0x190
4
process_one_work
vmlinux 0xffffffff9ecbdc4f
Build id: 3.10.0-1160.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffff9ecbed66
Build id: 3.10.0-1160.el7.x86_64
0x126
6
? manage_workers.isra.26
vmlinux 0xffffffff9ecbec40
Build id: 3.10.0-1160.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffff9ecc5c21
Build id: 3.10.0-1160.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff9ecc5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff9f393ddd
Build id: 3.10.0-1160.el7.x86_64
0x7
10
? insert_kthread_work
vmlinux 0xffffffff9ecc5b50
Build id: 3.10.0-1160.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [kworker/u256:3:59015]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ext4 snd_ens1371 snd_rawmidi mbcache snd_ac97_codec jbd2 ac97_bus snd_seq crc32_pclmul ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw snd_timer ppdev gf128mul snd glue_helper ablk_helper soundcore cryptd sg joydev pcspkr vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c
sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx nfit drm_kms_helper libnvdimm syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata drm mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih mptbase e1000 crc32c_intel serio_raw drm_panel_orientation_quirks fuse
CPU: 1 PID: 59015 Comm: kworker/u256:3 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8d7dd711d280 ti: ffff8d7ffc738000 task.ti: ffff8d7ffc738000
RIP: 0010:[<ffffffffc0123fa9>]  [<ffffffffc0123fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8d7ffc73bd90  EFLAGS: 00000202
RAX: ffffaf2bc3500000 RBX: ffff8d7ffc73bda8 RCX: ffff8d79d9bd2120
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff8d79dcf8dd60
RBP: ffff8d7ffc73be18 R08: 0000000000000002 R09: ca1819966778dd60
R10: ca1819966778dd60 R11: 0000000000000246 R12: ffff8d7ffc73bd50
R13: 0000000000000001 R14: ffff8d809d65ad40 R15: ffff8d7dd711d2e8
FS:  0000000000000000(0000) GS:ffff8d809d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000841c30 CR3: 00000002988d2000 CR4: 0000000000760fe0
PKRU: 00000000
Call Trace:
[<ffffffff9ecde185>] ? sched_clock_cpu+0x85/0xc0
[<ffffffff9ecda830>] ? try_to_wake_up+0x190/0x390
[<ffffffff9ecbdc4f>] process_one_work+0x17f/0x440
[<ffffffff9ecbed66>] worker_thread+0x126/0x3c0
[<ffffffff9ecbec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9ecc5c21>] kthread+0xd1/0xe0
[<ffffffff9ecc5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff9f393ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff9ecc5b50>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #674612
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc016a1a9
Build id: 3.10.0-1160.88.1.el7.x86_64
0xd9
2
? try_to_wake_up
vmlinux 0xffffffff988e0e92
Build id: 3.10.0-1160.88.1.el7.x86_64
0x192
3
process_one_work
vmlinux 0xffffffff988c32ef
Build id: 3.10.0-1160.88.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff988c4436
Build id: 3.10.0-1160.88.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff988c4310
Build id: 3.10.0-1160.88.1.el7.x86_64
0x2b0
6
kthread
vmlinux 0xffffffff988cb621
Build id: 3.10.0-1160.88.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff988cb550
Build id: 3.10.0-1160.88.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff98fc51dd
Build id: 3.10.0-1160.88.1.el7.x86_64
0x7
9
? insert_kthread_work
vmlinux 0xffffffff988cb550
Build id: 3.10.0-1160.88.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 29s! [kworker/u256:3:84572]
Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype mfe_fileaccess_100714877(OE) cbsensor_2_5_1230541(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter mfe_aac_100714877(OE) vmw_vsock_vmci_transport vsock ppdev vmw_balloon joydev sg vmw_vmci i2c_piix4 pcspkr parport_pc parport nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c
sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic crct10dif_common pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit mptspi ata_piix scsi_transport_spi libata libnvdimm mptscsih mptbase serio_raw crc32c_intel vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse br_netfilter bridge stp llc overlay(T)
CPU: 5 PID: 84572 Comm: kworker/u256:3 Tainted: G           OEL ------------ T 3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8fbaa430c200 ti: ffff8fc31f548000 task.ti: ffff8fc31f548000
RIP: 0010:[<ffffffffc016a1a9>]  [<ffffffffc016a1a9>] mpt_fault_reset_work+0xd9/0x48a [mptbase]
RSP: 0018:ffff8fc31f54bd90  EFLAGS: 00000202
RAX: ffffb9abc6300000 RBX: ffff8fc45c0aba58 RCX: dead000000000200
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff8fc4950bbd60
RBP: ffff8fc31f54be18 R08: ffff8fc4950bbd68 R09: c0ed3b959f8bbd60
R10: c0ed3b959f8bbd60 R11: ffffffffc07ea160 R12: 000000011070e007
R13: ffffffff98a8def7 R14: ffff8fc31f54bd68 R15: 0000000000002000
FS:  0000000000000000(0000) GS:ffff8fc493740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000001b80031d000 CR3: 0000000785010000 CR4: 00000000000007e0
Call Trace:
[<ffffffff988e0e92>] ? try_to_wake_up+0x192/0x3a0
[<ffffffff988c32ef>] process_one_work+0x17f/0x440
[<ffffffff988c4436>] worker_thread+0x126/0x3c0
[<ffffffff988c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff988cb621>] kthread+0xd1/0xe0
[<ffffffff988cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff98fc51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff988cb550>] ? insert_kthread_work+0x40/0x40
Code: 85 3f 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #527348
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02d2fa9
Build id: 3.10.0-1127.18.2.el7.x86_64
0xc9
2
process_one_work
vmlinux 0xffffffff898be6bf
Build id: 3.10.0-1127.18.2.el7.x86_64
0x17f
3
worker_thread
vmlinux 0xffffffff898bf7d6
Build id: 3.10.0-1127.18.2.el7.x86_64
0x126
4
? manage_workers.isra.26
vmlinux 0xffffffff898bf6b0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x2a0
5
kthread
vmlinux 0xffffffff898c6691
Build id: 3.10.0-1127.18.2.el7.x86_64
0xd1
6
? insert_kthread_work
vmlinux 0xffffffff898c65c0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x40
7
ret_from_fork_nospec_begin
vmlinux 0xffffffff89f92d37
Build id: 3.10.0-1127.18.2.el7.x86_64
0x21
8
? insert_kthread_work
vmlinux 0xffffffff898c65c0
Build id: 3.10.0-1127.18.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [kworker/u16:2:14684]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg vmw_balloon pcspkr joydev vmw_vmci parport_pc i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit libata crct10dif_pclmul crct10dif_common mptspi libnvdimm crc32c_intel scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 14684 Comm: kworker/u16:2 Not tainted 3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8ec2bbada0e0 ti: ffff8ec3e2874000 task.ti: ffff8ec3e2874000
RIP: 0010:[<ffffffffc02d2fa9>]  [<ffffffffc02d2fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8ec3e2877d90  EFLAGS: 00000202
RAX: ffffb0f483640000 RBX: 0000000000000202 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff8ec89a7d2d60
RBP: ffff8ec3e2877e18 R08: ffff8ec89a7d2d68 R09: c4dd24d3e4fd2d60
R10: c4dd24d3e4fd2d60 R11: 0000000000000000 R12: ffffffff899ca9d0
R13: ffffffff898bc99b R14: ffff8ec3e2877d28 R15: ffffffff898af728
FS:  0000000000000000(0000) GS:ffff8ec8bfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007faeccef8098 CR3: 00000001b2a10000 CR4: 00000000003607e0
Call Trace:
[<ffffffff898be6bf>] process_one_work+0x17f/0x440
[<ffffffff898bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff898bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff898c6691>] kthread+0xd1/0xe0
[<ffffffff898c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff89f92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff898c65c0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #494101
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc00e3fa9
Build id: 3.10.0-1160.36.2.el7.x86_64
0xc9
2
process_one_work
vmlinux 0xffffffffaaabde6f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x17f
3
worker_thread
vmlinux 0xffffffffaaabef86
Build id: 3.10.0-1160.36.2.el7.x86_64
0x126
4
? manage_workers.isra.26
vmlinux 0xffffffffaaabee60
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2a0
5
kthread
vmlinux 0xffffffffaaac5e41
Build id: 3.10.0-1160.36.2.el7.x86_64
0xd1
6
? system_call_after_swapgs
vmlinux 0xffffffffab195ec9
Build id: 3.10.0-1160.36.2.el7.x86_64
0x96
7
? insert_kthread_work
vmlinux 0xffffffffaaac5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffab195df7
Build id: 3.10.0-1160.36.2.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffaaac5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 28s! [xfsaild/dm-0:430]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [kworker/u8:0:25096]
Modules linked in: veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack overlay(T) binfmt_misc vmw_vsock_vmci_transport vsock sunrpc coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon pcspkr joydev parport_pc sg parport vmw_vmci i2c_piix4 br_netfilter bridge stp llc ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi e1000 mptscsih mptbase crct10dif_pclmul crct10dif_common crc32c_intel serio_raw floppy drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod fuse
CPU: 0 PID: 25096 Comm: kworker/u8:0 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/07/2011
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8db0bfdd1080 ti: ffff8daf12be8000 task.ti: ffff8daf12be8000
RIP: 0010:[<ffffffffc00e3fa9>]  [<ffffffffc00e3fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8daf12bebd90  EFLAGS: 00000202
RAX: ffff9b8700f80000 RBX: 0000000000000202 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff8db0bba1bd60
RBP: ffff8daf12bebe18 R08: ffff8db0bba1bd68 R09: c93c9f29c621bd60
R10: c93c9f29c621bd60 R11: 0000000000000000 R12: ffffffffaabcb100
R13: ffffffffaaabc14b R14: ffff8daf12bebd28 R15: ffffffffaaaaeea8
FS:  0000000000000000(0000) GS:ffff8db0bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000b1707831000 CR3: 0000000230ef6000 CR4: 00000000000007f0
Call Trace:
[<ffffffffaaabde6f>] process_one_work+0x17f/0x440
[<ffffffffaaabef86>] worker_thread+0x126/0x3c0
[<ffffffffaaabee60>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffaaac5e41>] kthread+0xd1/0xe0
[<ffffffffab195ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffaaac5d70>] ? insert_kthread_work+0x40/0x40
[<ffffffffab195df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaaac5d70>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #467082
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc00fdfa9
Build id: 3.10.0-957.10.1.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffffa456878f
Build id: 3.10.0-957.10.1.el7.x86_64
0x3ff
3
process_one_work
vmlinux 0xffffffffa3eb9d8f
Build id: 3.10.0-957.10.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffa3ebae26
Build id: 3.10.0-957.10.1.el7.x86_64
0x126
5
? manage_workers.isra.25
vmlinux 0xffffffffa3ebad00
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffa3ec1c71
Build id: 3.10.0-957.10.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffa3ec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffa4575c37
Build id: 3.10.0-957.10.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffa3ec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 31s! [nessusd:3369]
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 31s! [kworker/u16:1:30095]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13707(E) xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables devlink ip6table_filter ip6_tables ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack libcrc32c br_netfilter bridge stp llc overlay(T) falcon_lsm_pinned_13601(E) vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev parport_pc sg parport vmw_vmci i2c_piix4 nfsd nfs_acl lockd grace auth_rpcgss binfmt_misc sunrpc
ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata nfit crct10dif_pclmul crct10dif_common libnvdimm mptspi crc32c_intel scsi_transport_spi serio_raw mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy fuse [last unloaded: falcon_kal]
CPU: 7 PID: 30095 Comm: kworker/u16:1 Kdump: loaded Tainted: P            EL ------------ T 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8d415a22b0c0 ti: ffff8d3e2124c000 task.ti: ffff8d3e2124c000
RIP: 0010:[<ffffffffc00fdfa9>]  [<ffffffffc00fdfa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8d3e2124fd90  EFLAGS: 00000202
RAX: ffffb34f83640000 RBX: 0000000000000000 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff8d41ffdadd60
RBP: ffff8d3e2124fe18 R08: ffff8d41ffdadd68 R09: caf78dd68a5add60
R10: caf78dd68a5add60 R11: 7fffffffffffffff R12: ffff8d3e2124fd50
R13: 0000000000000001 R14: ffff8d41ffddac00 R15: ffff8d415a22b128
FS:  0000000000000000(0000) GS:ffff8d41ffdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f13a8e23000 CR3: 000000071c7a2000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa456878f>] ? __schedule+0x3ff/0x890
[<ffffffffa3eb9d8f>] process_one_work+0x17f/0x440
[<ffffffffa3ebae26>] worker_thread+0x126/0x3c0
[<ffffffffa3ebad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa3ec1c71>] kthread+0xd1/0xe0
[<ffffffffa3ec1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa4575c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa3ec1ba0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #430337
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0109fa9
Build id: 3.10.0-1160.62.1.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffffb1adac82
Build id: 3.10.0-1160.62.1.el7.x86_64
0x192
3
process_one_work
vmlinux 0xffffffffb1abdfdf
Build id: 3.10.0-1160.62.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffb1abf0f6
Build id: 3.10.0-1160.62.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffffb1abefd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffb1ac5fb1
Build id: 3.10.0-1160.62.1.el7.x86_64
0xd1
7
? system_call_after_swapgs
vmlinux 0xffffffffb2199ec9
Build id: 3.10.0-1160.62.1.el7.x86_64
0x96
8
? insert_kthread_work
vmlinux 0xffffffffb1ac5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffb2199df7
Build id: 3.10.0-1160.62.1.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffb1ac5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/u4:0:15443]
Modules linked in: tcp_diag udp_diag inet_diag xt_nat xt_mark xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables bridge stp llc ip_set vmw_vsock_vmci_transport vsock sunrpc ppdev coretemp vmw_balloon joydev pcspkr parport_pc sg parport i2c_piix4 vmw_vmci binfmt_misc xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm serio_raw mptspi scsi_transport_spi libata e1000 mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash
dm_log dm_mod fuse [last unloaded: ip_tables]
CPU: 0 PID: 15443 Comm: kworker/u4:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff924b7c10a100 ti: ffff924a8e3a8000 task.ti: ffff924a8e3a8000
RIP: 0010:[<ffffffffc0109fa9>]  [<ffffffffc0109fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff924a8e3abd90  EFLAGS: 00000202
RAX: ffff9ec840f40000 RBX: fffffffffffb3f70 RCX: 0000000000000000
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff924a75d57d60
RBP: ffff924a8e3abe18 R08: ffffffffb221e640 R09: ffff924b7c10a100
R10: 000000000000bc39 R11: 0000000000000400 R12: ffffffffb1ae5a71
R13: ffff924a8e3abd40 R14: 0000000000000001 R15: ffff924b8641acc0
FS:  0000000000000000(0000) GS:ffff924b86400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0cea28a000 CR3: 00000000a868a000 CR4: 00000000000407f0
Call Trace:
[<ffffffffb1adac82>] ? try_to_wake_up+0x192/0x390
[<ffffffffb1abdfdf>] process_one_work+0x17f/0x440
[<ffffffffb1abf0f6>] worker_thread+0x126/0x3c0
[<ffffffffb1abefd0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb1ac5fb1>] kthread+0xd1/0xe0
[<ffffffffb2199ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb1ac5ee0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb2199df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb1ac5ee0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #419674
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0363fa9
Build id: 3.10.0-1160.45.1.el7.x86_64
0xc9
2
? __switch_to
vmlinux 0xffffffff9ac2b59e
Build id: 3.10.0-1160.45.1.el7.x86_64
0xce
3
process_one_work
vmlinux 0xffffffff9acbde8f
Build id: 3.10.0-1160.45.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff9acbefa6
Build id: 3.10.0-1160.45.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff9acbee80
Build id: 3.10.0-1160.45.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff9acc5e61
Build id: 3.10.0-1160.45.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff9acc5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff9b395df7
Build id: 3.10.0-1160.45.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff9acc5d90
Build id: 3.10.0-1160.45.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [kworker/u256:3:14671]
Modules linked in: tcp_lp nls_utf8 isofs br_netfilter overlay(T) tun devlink bridge stp llc ip_set vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac snd_ens1371 iosf_mbi snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq ppdev snd_seq_device aesni_intel snd_pcm lrw gf128mul glue_helper ablk_helper snd_timer cryptd snd vmw_balloon soundcore joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common nfit crc32c_intel ata_piix libnvdimm libata mptspi scsi_transport_spi serio_raw e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod fuse [last unloaded: ip_tables]
CPU: 6 PID: 14671 Comm: kworker/u256:3 Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9c531902a100 ti: ffff9c520165c000 task.ti: ffff9c520165c000
RIP: 0010:[<ffffffffc0363fa9>]  [<ffffffffc0363fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0000:ffff9c520165fd90  EFLAGS: 00000202
RAX: ffffad2101c40000 RBX: ffff9c531902a168 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff9c54a89d2d60
RBP: ffff9c520165fe18 R08: ffff9c54a89d2d68 R09: 8eacf9dff31d2d60
R10: 8eacf9dff31d2d60 R11: 0000000000000000 R12: ffff9c520165fd28
R13: 000039032d61bf8c R14: ffffffff9ace482c R15: ffff9c520165fd28
FS:  0000000000000000(0000) GS:ffff9c562d780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2d90e72000 CR3: 00000002a8810000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9ac2b59e>] ? __switch_to+0xce/0x580
[<ffffffff9acbde8f>] process_one_work+0x17f/0x440
[<ffffffff9acbefa6>] worker_thread+0x126/0x3c0
[<ffffffff9acbee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9acc5e61>] kthread+0xd1/0xe0
[<ffffffff9acc5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffff9b395df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9acc5d90>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #369104
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc024dfa9
Build id: 3.10.0-1160.42.2.el7.x86_64
0xc9
2
? __switch_to
vmlinux 0xffffffffb282b59e
Build id: 3.10.0-1160.42.2.el7.x86_64
0xce
3
process_one_work
vmlinux 0xffffffffb28bde8f
Build id: 3.10.0-1160.42.2.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffb28befa6
Build id: 3.10.0-1160.42.2.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffffb28bee80
Build id: 3.10.0-1160.42.2.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffb28c5e61
Build id: 3.10.0-1160.42.2.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffb28c5d90
Build id: 3.10.0-1160.42.2.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffb2f95ddd
Build id: 3.10.0-1160.42.2.el7.x86_64
0x7
9
? insert_kthread_work
vmlinux 0xffffffffb28c5d90
Build id: 3.10.0-1160.42.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [postmaster:4884]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/u256:0:54484]
Modules linked in: talpa_vfshook(OE) talpa_pedconnector(OE) talpa_pedevice(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables tcp_diag inet_diag nfsv3 nfs_acl nfs lockd grace fscache cmac arc4 md4 nls_utf8 cifs ccm dns_resolver bridge stp llc binfmt_misc ip_set vmw_vsock_vmci_transport vsock ppdev iosf_mbi vmw_balloon crc32_pclmul ghash_clmulni_intel aesni_intel
lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 auth_rpcgss sunrpc xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix mptspi serio_raw scsi_transport_spi mptscsih libata vmxnet3 mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: talpa_syscallhook]
CPU: 0 PID: 54484 Comm: kworker/u256:0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.42.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff917278040000 ti: ffff9171f58d0000 task.ti: ffff9171f58d0000
RIP: 0010:[<ffffffffc024dfa9>]  [<ffffffffc024dfa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9171f58d3d90  EFLAGS: 00000202
RAX: ffffa070431c0000 RBX: ffff917278040068 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff917667f0bd60
RBP: ffff9171f58d3e18 R08: ffff917667f0bd68 R09: ba25f1b37270bd60
R10: ba25f1b37270bd60 R11: ffff9171c2afcfc0 R12: ffff9171f58d3d28
R13: 0004d2a2100eb613 R14: ffffffffb28e482c R15: ffff9171f58d3d28
FS:  0000000000000000(0000) GS:ffff91766b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc7e0db7790 CR3: 00000001d5d64000 CR4: 00000000007607f0
PKRU: 00000000
Call Trace:
[<ffffffffb282b59e>] ? __switch_to+0xce/0x580
[<ffffffffb28bde8f>] process_one_work+0x17f/0x440
[<ffffffffb28befa6>] worker_thread+0x126/0x3c0
[<ffffffffb28bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb28c5e61>] kthread+0xd1/0xe0
[<ffffffffb28c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffb2f95ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffb28c5d90>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #340487
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0245fa9
Build id: 3.10.0-957.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffffb1ad64f0
Build id: 3.10.0-957.el7.x86_64
0x190
3
process_one_work
vmlinux 0xffffffffb1ab9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffb1abade6
Build id: 3.10.0-957.el7.x86_64
0x126
5
? manage_workers.isra.25
vmlinux 0xffffffffb1abacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffb1ac1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffb1ac1b60
Build id: 3.10.0-957.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffb2174c37
Build id: 3.10.0-957.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffb1ac1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u256:1:58]
Modules linked in: veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) overlay(T) sunrpc snd_seq_midi snd_seq_midi_event coretemp snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel snd_seq lrw gf128mul glue_helper ablk_helper snd_seq_device snd_pcm cryptd snd_timer snd vmw_balloon joydev soundcore pcspkr parport_pc parport sg i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic
pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi drm e1000 libata scsi_transport_spi mptscsih mptbase nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 58 Comm: kworker/u256:1 Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8d7bf644d140 ti: ffff8d7bf64ec000 task.ti: ffff8d7bf64ec000
RIP: 0010:[<ffffffffc0245fa9>]  [<ffffffffc0245fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8d7bf64efd90  EFLAGS: 00000202
RAX: ffffa3ca00d00000 RBX: ffff8d7bfa41ab80 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff8d7bf5ed7d60
RBP: ffff8d7bf64efe18 R08: ffff8d7bf5ed7d68 R09: ca0fb5c6006d7d60
R10: ca0fb5c6006d7d60 R11: 0000000000000000 R12: 00000000f64efd50
R13: ffffffffb1ae11ed R14: ffff8d7bf64efd38 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8d7bfa400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f30bb387780 CR3: 00000000a32be000 CR4: 00000000003607f0
Call Trace:
[<ffffffffb1ad64f0>] ? try_to_wake_up+0x190/0x390
[<ffffffffb1ab9d4f>] process_one_work+0x17f/0x440
[<ffffffffb1abade6>] worker_thread+0x126/0x3c0
[<ffffffffb1abacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffb1ac1c31>] kthread+0xd1/0xe0
[<ffffffffb1ac1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffb2174c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb1ac1b60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #317923
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0329fa9
Build id: 3.10.0-957.12.1.el7.x86_64
0xc9
2
? sched_clock_cpu
vmlinux 0xffffffff948d9f15
Build id: 3.10.0-957.12.1.el7.x86_64
0x85
3
? try_to_wake_up
vmlinux 0xffffffff948d6530
Build id: 3.10.0-957.12.1.el7.x86_64
0x190
4
process_one_work
vmlinux 0xffffffff948b9d8f
Build id: 3.10.0-957.12.1.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffff948bae26
Build id: 3.10.0-957.12.1.el7.x86_64
0x126
6
? manage_workers.isra.25
vmlinux 0xffffffff948bad00
Build id: 3.10.0-957.12.1.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffff948c1c71
Build id: 3.10.0-957.12.1.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff948c1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff94f75bf7
Build id: 3.10.0-957.12.1.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff948c1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 37s! [kworker/1:1:17722]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 37s! [kworker/u8:0:17781]
Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev coretemp crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common
crc32c_intel serio_raw vmxnet3 ata_generic pata_acpi vmwgfx floppy drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptsas scsi_transport_sas drm ata_piix mptscsih mptbase libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 17781 Comm: kworker/u8:0 Kdump: loaded Tainted: G             L ------------   3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffffa0e4f60330c0 ti: ffffa0e4e4f3c000 task.ti: ffffa0e4e4f3c000
RIP: 0010:[<ffffffffc0329fa9>]  [<ffffffffc0329fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffffa0e4e4f3fd90  EFLAGS: 00000202
RAX: ffffb0c900c70000 RBX: ffffa0e4e4f3fda8 RCX: ffffa0e4effc9060
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffffa0e4ffda9d60
RBP: ffffa0e4e4f3fe18 R08: 0000000000000003 R09: ffffffff958e9c00
R10: 0000000007000000 R11: 0000000000000400 R12: ffffa0e4e4f3fd50
R13: 0000000000000001 R14: ffffa0e4ffd1ac00 R15: ffffa0e4f6033128
FS:  0000000000000000(0000) GS:ffffa0e4ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffc752e2010 CR3: 00000000b7e5e000 CR4: 00000000000007e0
Call Trace:
[<ffffffff948d9f15>] ? sched_clock_cpu+0x85/0xc0
[<ffffffff948d6530>] ? try_to_wake_up+0x190/0x390
[<ffffffff948b9d8f>] process_one_work+0x17f/0x440
[<ffffffff948bae26>] worker_thread+0x126/0x3c0
[<ffffffff948bad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff948c1c71>] kthread+0xd1/0xe0
[<ffffffff948c1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffff94f75bf7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff948c1ba0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #307371
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc027cf89
Build id: 3.10.0-862.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffff9211291c
Build id: 3.10.0-862.el7.x86_64
0x41c
3
process_one_work
vmlinux 0xffffffff91ab2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff91ab3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
5
? manage_workers.isra.24
vmlinux 0xffffffff91ab39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff91abae31
Build id: 3.10.0-862.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff91abad60
Build id: 3.10.0-862.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff9211f61d
Build id: 3.10.0-862.el7.x86_64
0x7
9
? insert_kthread_work
vmlinux 0xffffffff91abad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/u256:3:7002]
Modules linked in: nls_utf8 isofs rfcomm fuse tun devlink ip_set nfnetlink bridge stp llc bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev joydev vmw_balloon pcspkr btusb btrtl btbcm btintel bluetooth snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd sg soundcore nfit rfkill vmw_vmci i2c_piix4 libnvdimm shpchp parport_pc parport xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm drm serio_raw mptspi scsi_transport_spi ata_piix mptscsih e1000
libata mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 7002 Comm: kworker/u256:3 Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff97b9ce900000 ti: ffff97b9f6b94000 task.ti: ffff97b9f6b94000
RIP: 0010:[<ffffffffc027cf89>]  [<ffffffffc027cf89>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff97b9f6b97d90  EFLAGS: 00000202
RAX: ffffae66c0700000 RBX: ffff97bab9658b80 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff97bab1820d60
RBP: ffff97b9f6b97e18 R08: ffff97bab1820d68 R09: a114d1b27c020d60
R10: a114d1b27c020d60 R11: 0000000000000000 R12: 0000004db5872200
R13: ffffffff91ad865c R14: ffff97b9f6b97d40 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff97bab9640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f52ec639d00 CR3: 0000000134e30000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9211291c>] ? __schedule+0x41c/0xa20
[<ffffffff91ab2dff>] process_one_work+0x17f/0x440
[<ffffffff91ab3ac6>] worker_thread+0x126/0x3c0
[<ffffffff91ab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff91abae31>] kthread+0xd1/0xe0
[<ffffffff91abad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff9211f61d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff91abad60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #290774
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc015cfa9
Build id: 3.10.0-1160.21.1.el7.x86_64
0xc9
2
? x2apic_send_IPI_mask
vmlinux 0xffffffff90e63d03
Build id: 3.10.0-1160.21.1.el7.x86_64
0x13
3
? try_to_wake_up
vmlinux 0xffffffff90edae00
Build id: 3.10.0-1160.21.1.el7.x86_64
0x190
4
process_one_work
vmlinux 0xffffffff90ebddcf
Build id: 3.10.0-1160.21.1.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffff90ebeee6
Build id: 3.10.0-1160.21.1.el7.x86_64
0x126
6
? manage_workers.isra.26
vmlinux 0xffffffff90ebedc0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffff90ec5da1
Build id: 3.10.0-1160.21.1.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff90ec5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff91595ddd
Build id: 3.10.0-1160.21.1.el7.x86_64
0x7
10
? insert_kthread_work
vmlinux 0xffffffff90ec5cd0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u256:2:99366]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun binfmt_misc devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper snd_ens1371 snd_rawmidi snd_ac97_codec cryptd ac97_bus snd_seq snd_seq_device snd_pcm
snd_timer snd sg pcspkr soundcore vmw_balloon joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix nfit serio_raw libnvdimm e1000 libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 99366 Comm: kworker/u256:2 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff887dd05b5280 ti: ffff887de9830000 task.ti: ffff887de9830000
RIP: 0010:[<ffffffffc015cfa9>]  [<ffffffffc015cfa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff887de9833d90  EFLAGS: 00000202
RAX: ffffadd600780000 RBX: ffff887de9833da8 RCX: 0000000000000002
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff887df5ef3d60
RBP: ffff887de9833e18 R08: ffffffff91613a48 R09: ffffffff91f0abc0
R10: 00000000e7000000 R11: 0000000000000400 R12: ffff887de9833d50
R13: 0000000000000001 R14: ffff887e3b61ad40 R15: ffff887dd05b52e8
FS:  0000000000000000(0000) GS:ffff887e3b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005654c008604d CR3: 0000000012aa8000 CR4: 00000000003607f0
Call Trace:
[<ffffffff90e63d03>] ? x2apic_send_IPI_mask+0x13/0x20
[<ffffffff90edae00>] ? try_to_wake_up+0x190/0x390
[<ffffffff90ebddcf>] process_one_work+0x17f/0x440
[<ffffffff90ebeee6>] worker_thread+0x126/0x3c0
[<ffffffff90ebedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff90ec5da1>] kthread+0xd1/0xe0
[<ffffffff90ec5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff91595ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff90ec5cd0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #233856
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc024bfa9
Build id: 3.10.0-1160.36.2.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffff840dab12
Build id: 3.10.0-1160.36.2.el7.x86_64
0x192
3
process_one_work
vmlinux 0xffffffff840bde6f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff840bef86
Build id: 3.10.0-1160.36.2.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff840bee60
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff840c5e41
Build id: 3.10.0-1160.36.2.el7.x86_64
0xd1
7
? system_call_after_swapgs
vmlinux 0xffffffff84795ec9
Build id: 3.10.0-1160.36.2.el7.x86_64
0x96
8
? insert_kthread_work
vmlinux 0xffffffff840c5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff84795df7
Build id: 3.10.0-1160.36.2.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff840c5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 30s! [kworker/0:2:29173]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 30s! [kworker/u4:0:28223]
Modules linked in: ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack vmw_vsock_vmci_transport vsock sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul pcspkr glue_helper ablk_helper cryptd vmw_balloon sg joydev i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel serio_raw mptspi e1000 scsi_transport_spi mptscsih ata_piix mptbase libata drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 28223 Comm: kworker/u4:0 Tainted: G             L ------------   3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8803dd5e3180 ti: ffff8803d4fa4000 task.ti: ffff8803d4fa4000
RIP: 0010:[<ffffffffc024bfa9>]  [<ffffffffc024bfa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8803d4fa7d90  EFLAGS: 00000202
RAX: ffffae2980440000 RBX: 0000000000004e17 RCX: 0000000000000000
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff8803f5dadd60
RBP: ffff8803d4fa7e18 R08: ffffffff8481e640 R09: ffff8803dd5e3180
R10: dfefb098805add60 R11: ffff880419ea4000 R12: ffffffff840e592a
R13: ffff8803d4fa7d40 R14: 0000000000000001 R15: ffff88041fd1acc0
FS:  0000000000000000(0000) GS:ffff88041fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffb8bda7000 CR3: 0000000056a5a000 CR4: 00000000000407e0
Call Trace:
[<ffffffff840dab12>] ? try_to_wake_up+0x192/0x390
[<ffffffff840bde6f>] process_one_work+0x17f/0x440
[<ffffffff840bef86>] worker_thread+0x126/0x3c0
[<ffffffff840bee60>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff840c5e41>] kthread+0xd1/0xe0
[<ffffffff84795ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff840c5d70>] ? insert_kthread_work+0x40/0x40
[<ffffffff84795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff840c5d70>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #230123
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc025ef89
Build id: 3.10.0-862.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffffad51291c
Build id: 3.10.0-862.el7.x86_64
0x41c
3
process_one_work
vmlinux 0xffffffffaceb2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffaceb3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
5
? manage_workers.isra.24
vmlinux 0xffffffffaceb39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffacebae31
Build id: 3.10.0-862.el7.x86_64
0xd1
7
? finish_task_switch
vmlinux 0xffffffffacec9664
Build id: 3.10.0-862.el7.x86_64
0x54
8
? insert_kthread_work
vmlinux 0xffffffffacebad60
Build id: 3.10.0-862.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffad51f637
Build id: 3.10.0-862.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffacebad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 35s! [ksoftirqd/1:14]
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u16:2:15772]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs ip6table_nat nf_conntrack_ipv6 lockd nf_defrag_ipv6 grace nf_nat_ipv6 fscache xt_set xt_mark ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipport ip_set_hash_ipportip ip_set dummy xt_comment xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables devlink ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat br_netfilter bridge stp llc overlay(T) nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper
cryptd pcspkr sg joydev vmw_balloon parport_pc parport i2c_piix4 shpchp vmw_vmci sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptspi scsi_transport_spi libata mptscsih mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 15772 Comm: kworker/u16:2 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff95b9ffef3f40 ti: ffff95b9cd3bc000 task.ti: ffff95b9cd3bc000
RIP: 0010:[<ffffffffc025ef89>]  [<ffffffffc025ef89>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff95b9cd3bfd90  EFLAGS: 00000202
RAX: ffffa55701bc0000 RBX: ffff95b5f66f4400 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff95b9eca09d60
RBP: ffff95b9cd3bfe18 R08: ffff95b9eca09d68 R09: a917e33777209d60
R10: a917e33777209d60 R11: 0000000000000000 R12: ffff95b5f6612e00
R13: ffffffffaced865c R14: ffff95b9cd3bfd40 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff95b9ffcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000c010441000 CR3: 00000003f260e000 CR4: 00000000000007e0
Call Trace:
[<ffffffffad51291c>] ? __schedule+0x41c/0xa20
[<ffffffffaceb2dff>] process_one_work+0x17f/0x440
[<ffffffffaceb3ac6>] worker_thread+0x126/0x3c0
[<ffffffffaceb39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffacebae31>] kthread+0xd1/0xe0
[<ffffffffacec9664>] ? finish_task_switch+0x54/0x170
[<ffffffffacebad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffad51f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffacebad60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #230122
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02d3f89
Build id: 3.10.0-862.el7.x86_64
0xc9
2
? x2apic_send_IPI_mask
vmlinux 0xffffffffab45e753
Build id: 3.10.0-862.el7.x86_64
0x13
3
? try_to_wake_up
vmlinux 0xffffffffab4cebfc
Build id: 3.10.0-862.el7.x86_64
0x18c
4
process_one_work
vmlinux 0xffffffffab4b2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffffab4b3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
6
? manage_workers.isra.24
vmlinux 0xffffffffab4b39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffffab4bae31
Build id: 3.10.0-862.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffffab4bad60
Build id: 3.10.0-862.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffabb1f637
Build id: 3.10.0-862.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffab4bad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 38s! [kworker/u16:3:11359]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 lockd nf_nat_ipv6 grace fscache xt_set xt_mark ip_set_hash_ipportip ip_set_hash_ipport ip_set_bitmap_port ip_set_hash_ipportnet ip_set dummy xt_comment nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle overlay(T) ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper
cryptd joydev pcspkr ppdev vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci shpchp sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix libahci crct10dif_pclmul mptspi crct10dif_common scsi_transport_spi crc32c_intel libata mptscsih serio_raw i2c_core vmxnet3 mptbase dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 11359 Comm: kworker/u16:3 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff883dd3214f10 ti: ffff883cd1110000 task.ti: ffff883cd1110000
RIP: 0010:[<ffffffffc02d3f89>]  [<ffffffffc02d3f89>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0000:ffff883cd1113d90  EFLAGS: 00000202
RAX: ffff9aff01cc0000 RBX: ffff883cd1113da8 RCX: 0000000000000038
RDX: 0000000024000000 RSI: 0000000000000046 RDI: ffff883eac0fdd60
RBP: ffff883cd1113e18 R08: ffffffffabc138e0 R09: ffffffffac4c7040
R10: 000000001be78000 R11: 0000000000000400 R12: ffff883eab812a00
R13: ffffffffab4d865c R14: ffff883cd1113d40 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff883ebfd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f9a50123c38 CR3: 000000022954a000 CR4: 00000000000007e0
Call Trace:
[<ffffffffab45e753>] ? x2apic_send_IPI_mask+0x13/0x20
[<ffffffffab4cebfc>] ? try_to_wake_up+0x18c/0x350
[<ffffffffab4b2dff>] process_one_work+0x17f/0x440
[<ffffffffab4b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffab4b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffab4bae31>] kthread+0xd1/0xe0
[<ffffffffab4bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffabb1f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffab4bad60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #216719
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc032efa9
Build id: 3.10.0-957.el7.x86_64
0xc9
2
process_one_work
vmlinux 0xffffffff83cb9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
3
worker_thread
vmlinux 0xffffffff83cbade6
Build id: 3.10.0-957.el7.x86_64
0x126
4
? manage_workers.isra.25
vmlinux 0xffffffff83cbacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
5
kthread
vmlinux 0xffffffff83cc1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
6
? insert_kthread_work
vmlinux 0xffffffff83cc1b60
Build id: 3.10.0-957.el7.x86_64
0x40
7
ret_from_fork_nospec_begin
vmlinux 0xffffffff84374c37
Build id: 3.10.0-957.el7.x86_64
0x21
8
? insert_kthread_work
vmlinux 0xffffffff83cc1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 247s! [kworker/u256:1:77918]
Modules linked in: nfnetlink_queue nfnetlink_log igb_uio(OE) uio tcp_lp nls_utf8 isofs rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper snd_ens1371 ablk_helper cryptd snd_rawmidi
snd_ac97_codec ac97_bus snd_seq btusb btrtl vmw_balloon snd_seq_device snd_pcm btbcm btintel snd_timer bluetooth snd joydev pcspkr rfkill soundcore sg parport_pc parport vmw_vmci i2c_piix4 sch_fq_codel fuse ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm e1000 mptspi libata scsi_transport_spi mptscsih nfit mptbase drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 77918 Comm: kworker/u256:1 Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff91f55daeb0c0 ti: ffff91f56caf0000 task.ti: ffff91f56caf0000
RIP: 0010:[<ffffffffc032efa9>]  [<ffffffffc032efa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff91f56caf3d90  EFLAGS: 00000202
RAX: ffffbbcd009c0000 RBX: 0000000000000202 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff91f4f5715d60
RBP: ffff91f56caf3e18 R08: ffff91f4f5715d68 R09: b82bbc2f7ff15d60
R10: b82bbc2f7ff15d60 R11: 0000000000000000 R12: ffffffff83dc2830
R13: ffffffff83cb7f5b R14: ffff91f56caf3d28 R15: ffffffff83caae88
FS:  0000000000000000(0000) GS:ffff91f5f9640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdb6990f069 CR3: 00000000b5de6000 CR4: 00000000003607e0
Call Trace:
[<ffffffff83cb9d4f>] process_one_work+0x17f/0x440
[<ffffffff83cbade6>] worker_thread+0x126/0x3c0
[<ffffffff83cbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff83cc1c31>] kthread+0xd1/0xe0
[<ffffffff83cc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff84374c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff83cc1b60>] ? insert_kthread_work+0x40/0x40
Complete report #204479
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc015dfa9
Build id: 3.10.0-1160.31.1.el7.x86_64
0xc9
2
? __switch_to
vmlinux 0xffffffff8b62b59e
Build id: 3.10.0-1160.31.1.el7.x86_64
0xce
3
process_one_work
vmlinux 0xffffffff8b6bde5f
Build id: 3.10.0-1160.31.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff8b6bef76
Build id: 3.10.0-1160.31.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff8b6bee50
Build id: 3.10.0-1160.31.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff8b6c5e31
Build id: 3.10.0-1160.31.1.el7.x86_64
0xd1
7
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ec9
Build id: 3.10.0-1160.31.1.el7.x86_64
0x96
8
? insert_kthread_work
vmlinux 0xffffffff8b6c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff8bd95df7
Build id: 3.10.0-1160.31.1.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff8b6c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 262s! [cmdb_hostserver:130094]
sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel mptspi libata scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 67247 Comm: kworker/u256:1 Tainted: G             L ------------ T 3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1506250318 06/25/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9e5a33b90000 ti: ffff9e547b9c8000 task.ti: ffff9e547b9c8000
RIP: 0010:[<ffffffffc015dfa9>]  [<ffffffffc015dfa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9e547b9cbd90  EFLAGS: 00000202
RAX: ffffaf7643500000 RBX: ffff9e5a33b90068 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff9e5bf9d24d60
RBP: ffff9e547b9cbe18 R08: ffff9e5bf9d24d68 R09: 868fb712c4524d60
R10: 868fb712c4524d60 R11: 0000000000000293 R12: ffff9e547b9cbd28
R13: 00002f18ae7542a2 R14: ffffffff8b6e452c R15: ffff9e547b9cbd28
FS:  0000000000000000(0000) GS:ffff9e5bfd640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa5cf601000 CR3: 000000082da10000 CR4: 00000000000007e0
Call Trace:
[<ffffffff8b62b59e>] ? __switch_to+0xce/0x580
[<ffffffff8b6bde5f>] process_one_work+0x17f/0x440
[<ffffffff8b6bef76>] worker_thread+0x126/0x3c0
[<ffffffff8b6bee50>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8b6c5e31>] kthread+0xd1/0xe0
[<ffffffff8bd95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff8b6c5d60>] ? insert_kthread_work+0x40/0x40
[<ffffffff8bd95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8b6c5d60>] ? insert_kthread_work+0x40/0x40
Complete report #194951
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02a6fa9
Build id: 3.10.0-1160.25.1.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffff974dab32
Build id: 3.10.0-1160.25.1.el7.x86_64
0x192
3
process_one_work
vmlinux 0xffffffff974bddcf
Build id: 3.10.0-1160.25.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff974beee6
Build id: 3.10.0-1160.25.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff974bedc0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff974c5da1
Build id: 3.10.0-1160.25.1.el7.x86_64
0xd1
7
? system_call_after_swapgs
vmlinux 0xffffffff97b95ec9
Build id: 3.10.0-1160.25.1.el7.x86_64
0x96
8
? insert_kthread_work
vmlinux 0xffffffff974c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff97b95df7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff974c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 37s! [kworker/u256:1:116201]
Modules linked in: tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink event_collector_2_1_364486(OE) talpa_vfshook(OE) talpa_pedconnector(OE) talpa_pedevice(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) vmw_vsock_vmci_transport vsock sunrpc kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx
drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul crct10dif_common sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm serio_raw libata mptspi scsi_transport_spi mptscsih mptbase vmxnet3 nfit libnvdimm drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 116201 Comm: kworker/u256:1 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9d203314b180 ti: ffff9d1e1c3e4000 task.ti: ffff9d1e1c3e4000
RIP: 0010:[<ffffffffc02a6fa9>]  [<ffffffffc02a6fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9d1e1c3e7d90  EFLAGS: 00000202
RAX: ffffc35540f80000 RBX: ffffffff974e73b8 RCX: 0000000000000000
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff9d1e35fe8d60
RBP: ffff9d1e1c3e7e18 R08: ffffffff97c1e640 R09: ffff9d1e3596e300
R10: 0000000006c00000 R11: 0000000000000400 R12: 0000000000000000
R13: ffff9d203561acc0 R14: ffff9d203561acc0 R15: ffff9d203561ad40
FS:  0000000000000000(0000) GS:ffff9d2035640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff271b21000 CR3: 00000000ba6bc000 CR4: 00000000000007e0
Call Trace:
[<ffffffff974dab32>] ? try_to_wake_up+0x192/0x390
[<ffffffff974bddcf>] process_one_work+0x17f/0x440
[<ffffffff974beee6>] worker_thread+0x126/0x3c0
[<ffffffff974bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff974c5da1>] kthread+0xd1/0xe0
[<ffffffff97b95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff974c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff97b95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff974c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #191022
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0115fa9
Build id: 3.10.0-1160.31.1.el7.x86_64
0xc9
2
? __switch_to
vmlinux 0xffffffffa122b59e
Build id: 3.10.0-1160.31.1.el7.x86_64
0xce
3
process_one_work
vmlinux 0xffffffffa12bde5f
Build id: 3.10.0-1160.31.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffa12bef76
Build id: 3.10.0-1160.31.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffffa12bee50
Build id: 3.10.0-1160.31.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffa12c5e31
Build id: 3.10.0-1160.31.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffa12c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffa1995df7
Build id: 3.10.0-1160.31.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffa12c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/u8:1:8100]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables vsock_diag tcp_diag udp_diag inet_diag unix_diag tun devlink bridge stp llc ip_set vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 nfsd nfs_acl lockd grace auth_rpcgss
sunrpc xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm vmxnet3 mptspi drm ata_piix scsi_transport_spi mptscsih nfit mptbase libata libnvdimm drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ip_tables]
CPU: 0 PID: 8100 Comm: kworker/u8:1 Kdump: loaded Not tainted 3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9df47e6c4200 ti: ffff9df3545e0000 task.ti: ffff9df3545e0000
RIP: 0010:[<ffffffffc0115fa9>]  [<ffffffffc0115fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9df3545e3d90  EFLAGS: 00000202
RAX: ffffb4c341bc0000 RBX: ffff9df47e6c4268 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff9df4eabbfd60
RBP: ffff9df3545e3e18 R08: ffff9df4eabbfd68 R09: 882bf304f53bfd60
R10: 882bf304f53bfd60 R11: 0000000000000000 R12: ffff9df3545e3d28
R13: 0000042c6cd405af R14: ffffffffa12e452c R15: ffff9df3545e3d28
FS:  0000000000000000(0000) GS:ffff9df4ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc5c4a02000 CR3: 000000041ff94000 CR4: 00000000000407f0
Call Trace:
[<ffffffffa122b59e>] ? __switch_to+0xce/0x580
[<ffffffffa12bde5f>] process_one_work+0x17f/0x440
[<ffffffffa12bef76>] worker_thread+0x126/0x3c0
[<ffffffffa12bee50>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa12c5e31>] kthread+0xd1/0xe0
[<ffffffffa12c5d60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa1995df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa12c5d60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #174688
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0406f89
Build id: 3.10.0-862.el7.x86_64
0xc9
2
? x2apic_send_IPI_mask
vmlinux 0xffffffffb9a5e753
Build id: 3.10.0-862.el7.x86_64
0x13
3
process_one_work
vmlinux 0xffffffffb9ab2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffb9ab3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
5
? manage_workers.isra.24
vmlinux 0xffffffffb9ab39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffb9abae31
Build id: 3.10.0-862.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffb9abad60
Build id: 3.10.0-862.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffba11f637
Build id: 3.10.0-862.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffb9abad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kworker/u16:3:12091]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nfsv4 dns_resolver nfs fscache rpcsec_gss_krb5 nf_conntrack_netlink br_netfilter xt_addrtype xt_set xt_mark ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set nfnetlink dummy xt_comment xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 overlay(T) nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr sg joydev
vmw_balloon parport_pc i2c_piix4 parport vmw_vmci shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci crct10dif_pclmul ata_piix crct10dif_common mptspi crc32c_intel scsi_transport_spi libata serio_raw mptscsih vmxnet3 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 12091 Comm: kworker/u16:3 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff914a2c4deeb0 ti: ffff91467d540000 task.ti: ffff91467d540000
RIP: 0010:[<ffffffffc0406f89>]  [<ffffffffc0406f89>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff91467d543d90  EFLAGS: 00000202
RAX: ffffaeb781c00000 RBX: ffff91467d543da8 RCX: 0000000000000038
RDX: 0000000024000000 RSI: 0000000000000046 RDI: ffff914a3fdacd60
RBP: ffff91467d543e18 R08: ffffffffba213660 R09: ffffffffbaac7040
R10: 000000002ea69370 R11: 0000000000000400 R12: ffff914a2c4deeb0
R13: ffffffffb9ad865c R14: ffff91467d543d40 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff914a3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007efc68cdc000 CR3: 00000004217be000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb9a5e753>] ? x2apic_send_IPI_mask+0x13/0x20
[<ffffffffb9ab2dff>] process_one_work+0x17f/0x440
[<ffffffffb9ab3ac6>] worker_thread+0x126/0x3c0
[<ffffffffb9ab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffb9abae31>] kthread+0xd1/0xe0
[<ffffffffb9abad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffba11f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb9abad60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #174533
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02a4f89
Build id: 3.10.0-862.el7.x86_64
0xc9
2
? ttwu_do_activate.constprop.93
vmlinux 0xffffffffb08cba9d
Build id: 3.10.0-862.el7.x86_64
0x5d
3
? try_to_wake_up
vmlinux 0xffffffffb08cebfc
Build id: 3.10.0-862.el7.x86_64
0x18c
4
process_one_work
vmlinux 0xffffffffb08b2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffffb08b3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
6
? manage_workers.isra.24
vmlinux 0xffffffffb08b39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffffb08bae31
Build id: 3.10.0-862.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffffb08bad60
Build id: 3.10.0-862.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffb0f1f637
Build id: 3.10.0-862.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffb08bad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 50s! [java:1625]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 33s! [kworker/u16:3:18649]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 grace fscache nf_conntrack_netlink br_netfilter xt_addrtype xt_set overlay(T) xt_mark xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 ip_set_hash_ipportip ip_set_bitmap_port xt_conntrack ip_set_hash_ipportnet ip_set_hash_ipport ip_set ipt_REJECT nf_reject_ipv4 nfnetlink dummy xt_comment iptable_nat nf_nat_ipv4 nf_nat tun bridge stp llc ebtable_filter ebtables nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs ip6table_filter ip6_tables nf_conntrack iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev sg
ablk_helper cryptd vmw_balloon pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci shpchp sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_generic cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci ata_piix crct10dif_pclmul drm crct10dif_common mptspi scsi_transport_spi serio_raw crc32c_intel libata mptscsih vmxnet3 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 18649 Comm: kworker/u16:3 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8e89c5be8fd0 ti: ffff8e882a7e4000 task.ti: ffff8e882a7e4000
RIP: 0010:[<ffffffffc02a4f89>]  [<ffffffffc02a4f89>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8e882a7e7d90  EFLAGS: 00000202
RAX: ffffb58541bc0000 RBX: 000000002a7e7d30 RCX: 0000000000000000
RDX: 0000000024000000 RSI: 0000000000000046 RDI: ffff8e89ec0a7d60
RBP: ffff8e882a7e7e18 R08: 0000000000000000 R09: ffff8e89c5be8fd0
R10: 000000028497f1b0 R11: 0000000000000400 R12: ffff8e882a7e7d48
R13: 0000000000000001 R14: ffff8e89ffc58b00 R15: ffff8e89c5be8fd0
FS:  0000000000000000(0000) GS:ffff8e89ffc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f49cc44242c CR3: 0000000180608000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb08cba9d>] ? ttwu_do_activate.constprop.93+0x5d/0x70
[<ffffffffb08cebfc>] ? try_to_wake_up+0x18c/0x350
[<ffffffffb08b2dff>] process_one_work+0x17f/0x440
[<ffffffffb08b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffb08b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffb08bae31>] kthread+0xd1/0xe0
[<ffffffffb08bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffb0f1f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb08bad60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #114415
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02b5fa9
Build id: 3.10.0-1127.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffff8f785922
Build id: 3.10.0-1127.el7.x86_64
0x402
3
process_one_work
vmlinux 0xffffffff8f0be6bf
Build id: 3.10.0-1127.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff8f0bf7d6
Build id: 3.10.0-1127.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff8f0bf6b0
Build id: 3.10.0-1127.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff8f0c6691
Build id: 3.10.0-1127.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff8f0c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff8f792d37
Build id: 3.10.0-1127.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff8f0c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u256:0:48207]
Modules linked in: qingteng(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock iosf_mbi snd_seq_midi kvm_intel snd_seq_midi_event kvm irqbypass crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq aesni_intel lrw snd_seq_device gf128mul ppdev glue_helper ablk_helper snd_pcm vmw_balloon cryptd snd_timer pcspkr joydev snd soundcore sg parport_pc
parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif ata_generic pata_acpi vmwgfx crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi drm nfit e1000 serio_raw mptscsih mptbase libata libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 48207 Comm: kworker/u256:0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9b74766cc1c0 ti: ffff9b74436b8000 task.ti: ffff9b74436b8000
RIP: 0010:[<ffffffffc02b5fa9>]  [<ffffffffc02b5fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0000:ffff9b74436bbd90  EFLAGS: 00000202
RAX: ffffa740403c0000 RBX: ffff9b74436bbd58 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff9b747574cd60
RBP: ffff9b74436bbe18 R08: ffff9b747574cd68 R09: 922dc5a13ff4cd60
R10: 922dc5a13ff4cd60 R11: fffff72ac032fd00 R12: ffff9b74436bbd50
R13: 0000000000000001 R14: ffff9b74bb61ad40 R15: ffff9b74766cc228
FS:  0000000000000000(0000) GS:ffff9b74bb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f27abaac000 CR3: 0000000078904000 CR4: 00000000003607f0
Call Trace:
[<ffffffff8f785922>] ? __schedule+0x402/0x840
[<ffffffff8f0be6bf>] process_one_work+0x17f/0x440
[<ffffffff8f0bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff8f0bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8f0c6691>] kthread+0xd1/0xe0
[<ffffffff8f0c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8f792d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8f0c65c0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #112603
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0408f89
Build id: 3.10.0-862.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffff876cebfc
Build id: 3.10.0-862.el7.x86_64
0x18c
3
process_one_work
vmlinux 0xffffffff876b2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff876b3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
5
? manage_workers.isra.24
vmlinux 0xffffffff876b39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff876bae31
Build id: 3.10.0-862.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff876bad60
Build id: 3.10.0-862.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff87d1f637
Build id: 3.10.0-862.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff876bad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [kworker/u8:0:21890]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_security iptable_raw ip_set nfnetlink tcp_diag inet_diag binfmt_misc tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc shpchp parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c
sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi drm crct10dif_pclmul scsi_transport_spi crct10dif_common libata e1000 crc32c_intel mptscsih mptbase serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 21890 Comm: kworker/u8:0 Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9c4ced152f70 ti: ffff9c4cb029c000 task.ti: ffff9c4cb029c000
RIP: 0010:[<ffffffffc0408f89>]  [<ffffffffc0408f89>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9c4cb029fd90  EFLAGS: 00000202
RAX: ffffb0e7c0400000 RBX: 0000000000000000 RCX: dead000000000200
RDX: 0000000024000000 RSI: 0000000000000046 RDI: ffff9c4cee779d60
RBP: ffff9c4cb029fe18 R08: ffff9c4cee779d68 R09: 8ecbe26e78f79d60
R10: 8ecbe26e78f79d60 R11: 0000000000000000 R12: ffff9c4c801ecf10
CPU: 1 PID: 12643 Comm: WebExtensions Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9c4cc6006eb0 ti: ffff9c4c839a8000 task.ti: ffff9c4c839a8000
RIP: 0033:[<00007f7c09a25bed>]  [<00007f7c09a25bed>] 0x7f7c09a25bed
RSP: 002b:00007fffc391a5b0  EFLAGS: 00010293
RAX: 0000000000000001 RBX: 00007fcfa35957b8 RCX: ffffffffffffffff
RDX: 00000000ffffffff RSI: 0000000000000003 RDI: 00007f7bea09a680
RBP: 00007f7c08e3e510 R08: 0000000000000003 R09: 0000000000000000
R10: 00007f7c08eef428 R11: 0000000000000293 R12: ffff9c4cf1c861e8
R13: 0000000000002710 R14: 00000000000000d1 R15: 00007fcfa35957b8
FS:  00007f7c0ad6e780(0000) GS:ffff9c4cf1c80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa471990000 CR3: 0000000003936000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 25s! [xfsaild/dm-1:639]
R13: ffffffff876d865c R14: ffff9c4cb029fd40 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff9c4cf1d80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f457d8ce5f0 CR3: 000000004bfaa000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff876cebfc>] ? try_to_wake_up+0x18c/0x350
[<ffffffff876b2dff>] process_one_work+0x17f/0x440
[<ffffffff876b3ac6>] worker_thread+0x126/0x3c0
[<ffffffff876b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff876bae31>] kthread+0xd1/0xe0
[<ffffffff876bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff87d1f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff876bad60>] ? insert_kthread_work+0x40/0x40
Complete report #95636
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0381fa9
Build id: 3.10.0-957.1.3.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffffb816778f
Build id: 3.10.0-957.1.3.el7.x86_64
0x3ff
3
process_one_work
vmlinux 0xffffffffb7ab9d4f
Build id: 3.10.0-957.1.3.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffb7abade6
Build id: 3.10.0-957.1.3.el7.x86_64
0x126
5
? manage_workers.isra.25
vmlinux 0xffffffffb7abacc0
Build id: 3.10.0-957.1.3.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffb7ac1c31
Build id: 3.10.0-957.1.3.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffb7ac1b60
Build id: 3.10.0-957.1.3.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffb8174c37
Build id: 3.10.0-957.1.3.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffb7ac1b60
Build id: 3.10.0-957.1.3.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [kworker/u16:2:5328]
Modules linked in: xt_addrtype ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfnetlink_log xt_nat veth binfmt_misc br_netfilter ip_set nfnetlink bridge stp llc iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack vmw_vsock_vmci_transport overlay(T) vsock sb_edac coretemp iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs
libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi mptscsih vmxnet3 libata mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: iptable_filter]
CPU: 1 PID: 5328 Comm: kworker/u16:2 Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffffa0502377d140 ti: ffffa04fd2958000 task.ti: ffffa04fd2958000
RIP: 0010:[<ffffffffc0381fa9>]  [<ffffffffc0381fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffffa04fd295bd90  EFLAGS: 00000202
RAX: ffffb20143680000 RBX: ffffa04e3119ea00 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffffa0525c6fbd60
RBP: ffffa04fd295be18 R08: ffffa0525c6fbd68 R09: 7eb62e9366efbd60
R10: 7eb62e9366efbd60 R11: 0000000000000000 R12: ffffa04fd295bd50
R13: 0000000000000001 R14: ffffa0527fc5ac00 R15: ffffa0502377d1a8
FS:  0000000000000000(0000) GS:ffffa0527fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f24349f2000 CR3: 000000062b082000 CR4: 00000000000607e0
Call Trace:
[<ffffffffb816778f>] ? __schedule+0x3ff/0x890
[<ffffffffb7ab9d4f>] process_one_work+0x17f/0x440
[<ffffffffb7abade6>] worker_thread+0x126/0x3c0
[<ffffffffb7abacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffb7ac1c31>] kthread+0xd1/0xe0
[<ffffffffb7ac1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffb8174c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb7ac1b60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #72650
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0140fa9
Build id: 3.10.0-1160.11.1.el7.x86_64
0xc9
2
? finish_task_switch
vmlinux 0xffffffff8b2d4def
Build id: 3.10.0-1160.11.1.el7.x86_64
0x14f
3
? __schedule
vmlinux 0xffffffff8b9868ef
Build id: 3.10.0-1160.11.1.el7.x86_64
0x3af
4
process_one_work
vmlinux 0xffffffff8b2bde9f
Build id: 3.10.0-1160.11.1.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffff8b2befb6
Build id: 3.10.0-1160.11.1.el7.x86_64
0x126
6
? manage_workers.isra.26
vmlinux 0xffffffff8b2bee90
Build id: 3.10.0-1160.11.1.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffff8b2c5e71
Build id: 3.10.0-1160.11.1.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff8b2c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff8b993df7
Build id: 3.10.0-1160.11.1.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff8b2c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/u4:1:13973]
Modules linked in: binfmt_misc vmw_vsock_vmci_transport vsock sunrpc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd pcspkr vmw_balloon sg joydev vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih crc32c_intel mptbase serio_raw vmxnet3 nfit libnvdimm drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 13973 Comm: kworker/u4:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9646b4bfa100 ti: ffff964623b44000 task.ti: ffff964623b44000
RIP: 0010:[<ffffffffc0140fa9>]  [<ffffffffc0140fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff964623b47d90  EFLAGS: 00000202
RAX: ffffa1edc08c0000 RBX: ffff964623b47d58 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff9645b5425d60
RBP: ffff964623b47e18 R08: ffff9645b5425d68 R09: a6e8c13c3fc25d60
R10: a6e8c13c3fc25d60 R11: 0000000000000000 R12: ffff964623b47d50
R13: 0000000000000001 R14: ffff9646bfd1ad40 R15: ffff9646b4bfa168
FS:  0000000000000000(0000) GS:ffff9646bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f280ca6b000 CR3: 0000000035e8e000 CR4: 00000000001607f0
Call Trace:
[<ffffffff8b2d4def>] ? finish_task_switch+0x14f/0x1c0
[<ffffffff8b9868ef>] ? __schedule+0x3af/0x860
[<ffffffff8b2bde9f>] process_one_work+0x17f/0x440
[<ffffffff8b2befb6>] worker_thread+0x126/0x3c0
[<ffffffff8b2bee90>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8b2c5e71>] kthread+0xd1/0xe0
[<ffffffff8b2c5da0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8b993df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8b2c5da0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #62513
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc00e2fa9
Build id: 3.10.0-1062.18.1.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffffb8edb340
Build id: 3.10.0-1062.18.1.el7.x86_64
0x190
3
process_one_work
vmlinux 0xffffffffb8ebe34f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffb8ebf466
Build id: 3.10.0-1062.18.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffffb8ebf340
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffb8ec6321
Build id: 3.10.0-1062.18.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffb8ec6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffb958dd37
Build id: 3.10.0-1062.18.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffb8ec6250
Build id: 3.10.0-1062.18.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [kworker/u256:2:56096]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt crc32c_intel fb_sys_fops ttm
serio_raw ata_piix drm mptspi scsi_transport_spi libata mptscsih vmxnet3 mptbase drm_panel_orientation_quirks
CPU: 2 PID: 56096 Comm: kworker/u256:2 Kdump: loaded Tainted: G             L ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8babcced62a0 ti: ffff8bacd96a8000 task.ti: ffff8bacd96a8000
RIP: 0010:[<ffffffffc00e2fa9>]  [<ffffffffc00e2fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8bacd96abd90  EFLAGS: 00000202
RAX: ffffab2501940000 RBX: ffff8bafed69ac80 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff8bafe9146d60
RBP: ffff8bacd96abe18 R08: ffff8bafe9146d68 R09: d13fe75e33946d60
R10: d13fe75e33946d60 R11: 0000000000000000 R12: 00000000d96abd50
R13: ffffffffb8ee5c01 R14: ffff8bacd96abd38 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8bafed680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb14c024000 CR3: 00000004190d4000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb8edb340>] ? try_to_wake_up+0x190/0x390
[<ffffffffb8ebe34f>] process_one_work+0x17f/0x440
[<ffffffffb8ebf466>] worker_thread+0x126/0x3c0
[<ffffffffb8ebf340>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb8ec6321>] kthread+0xd1/0xe0
[<ffffffffb8ec6250>] ? insert_kthread_work+0x40/0x40
[<ffffffffb958dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb8ec6250>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #24199
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc044bf89
Build id: 3.10.0-862.el7.x86_64
0xc9
2
? ttwu_do_activate.constprop.93
vmlinux 0xffffffffaf0cba9d
Build id: 3.10.0-862.el7.x86_64
0x5d
3
? try_to_wake_up
vmlinux 0xffffffffaf0cebfc
Build id: 3.10.0-862.el7.x86_64
0x18c
4
process_one_work
vmlinux 0xffffffffaf0b2dff
Build id: 3.10.0-862.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffffaf0b3ac6
Build id: 3.10.0-862.el7.x86_64
0x126
6
? manage_workers.isra.24
vmlinux 0xffffffffaf0b39a0
Build id: 3.10.0-862.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffffaf0bae31
Build id: 3.10.0-862.el7.x86_64
0xd1
8
? SyS_futex
vmlinux 0xffffffffaf1064b0
Build id: 3.10.0-862.el7.x86_64
0x80
9
? insert_kthread_work
vmlinux 0xffffffffaf0bad60
Build id: 3.10.0-862.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffaf71f637
Build id: 3.10.0-862.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffaf0bad60
Build id: 3.10.0-862.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 27s! [kworker/u256:1:11377]
Modules linked in: tcp_lp fuse xt_multiport xt_set iptable_raw ip_set_hash_ip ip_set_hash_net ip_set veth nf_conntrack_netlink nfnetlink xt_nat ip_vs_sh ip_vs_wrr ip_vs_rr tcp_diag inet_diag ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle lp ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 cfg80211 tun bridge rfkill stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event coretemp iosf_mbi kvm_intel snd_ens1371 kvm snd_rawmidi irqbypass snd_ac97_codec crc32_pclmul ac97_bus ghash_clmulni_intel ppdev
snd_seq snd_seq_device aesni_intel vmw_balloon snd_pcm lrw gf128mul glue_helper ablk_helper cryptd snd_timer snd joydev pcspkr soundcore sg vmw_vmci i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix mptspi scsi_transport_spi libata serio_raw mptscsih mptbase e1000 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 11377 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9466f3635ee0 ti: ffff9466bfabc000 task.ti: ffff9466bfabc000
RIP: 0010:[<ffffffffc044bf89>]  [<ffffffffc044bf89>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9466bfabfd90  EFLAGS: 00000202
RAX: ffffb10d83200000 RBX: 00000000bfabfd30 RCX: 0000000000000000
RDX: 0000000024000000 RSI: 0000000000000046 RDI: ffff94691fecad60
RBP: ffff9466bfabfe18 R08: 0000000000000000 R09: ffff9466f3635ee0
R10: ae5b14b66a6cad60 R11: 0000000000000000 R12: ffff9466bfabfd48
R13: 0000000000000001 R14: ffff9469256d8b00 R15: ffff9466f3635ee0
FS:  0000000000000000(0000) GS:ffff9469256c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00003dff773a9000 CR3: 0000000439e0e000 CR4: 00000000003607e0
Call Trace:
[<ffffffffaf0cba9d>] ? ttwu_do_activate.constprop.93+0x5d/0x70
[<ffffffffaf0cebfc>] ? try_to_wake_up+0x18c/0x350
[<ffffffffaf0b2dff>] process_one_work+0x17f/0x440
[<ffffffffaf0b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffaf0b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffaf0bae31>] kthread+0xd1/0xe0
[<ffffffffaf1064b0>] ? SyS_futex+0x80/0x180
[<ffffffffaf0bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf71f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaf0bad60>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #22608
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc021efa9
Build id: 3.10.0-1127.19.1.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffff848db6d0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x190
3
process_one_work
vmlinux 0xffffffff848be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff848bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff848bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff848c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff848c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff84f92d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff848c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 299s! [kworker/u8:4:11536]
Modules linked in: xt_multiport xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables tcp_lp fuse tun devlink bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg vmw_balloon joydev pcspkr parport_pc parport i2c_piix4 vmw_vmci xfs libcrc32c sr_mod sd_mod
cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata drm serio_raw mptspi scsi_transport_spi mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 11536 Comm: kworker/u8:4 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9d3417225230 ti: ffff9d34b6e5c000 task.ti: ffff9d34b6e5c000
RIP: 0010:[<ffffffffc021efa9>]  [<ffffffffc021efa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9d34b6e5fd90  EFLAGS: 00000202
RAX: ffffb88d41000000 RBX: 0000000000041d0c RCX: 0000000000000000
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff9d34bfd2fd60
RBP: ffff9d34b6e5fe18 R08: ffffffff8501e540 R09: ffff9d3417225230
R10: 000000000000bb93 R11: 0000000000000400 R12: ffffffff848e5fca
R13: ffff9d34b6e5fd40 R14: 0000000000000001 R15: ffff9d34bfc9acc0
FS:  0000000000000000(0000) GS:ffff9d34bfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f56b50ba000 CR3: 0000000187df2000 CR4: 00000000000407e0
Call Trace:
[<ffffffff848db6d0>] ? try_to_wake_up+0x190/0x390
[<ffffffff848be6bf>] process_one_work+0x17f/0x440
[<ffffffff848bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff848bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff848c6691>] kthread+0xd1/0xe0
[<ffffffff848c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff84f92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff848c65c0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #22270
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02a4fa9
Build id: 3.10.0-1160.6.1.el7.x86_64
0xc9
2
? finish_task_switch
vmlinux 0xffffffff940d4b9f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x14f
3
? __schedule
vmlinux 0xffffffff94786d4f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x3af
4
process_one_work
vmlinux 0xffffffff940bdc4f
Build id: 3.10.0-1160.6.1.el7.x86_64
0x17f
5
worker_thread
vmlinux 0xffffffff940bed66
Build id: 3.10.0-1160.6.1.el7.x86_64
0x126
6
? manage_workers.isra.26
vmlinux 0xffffffff940bec40
Build id: 3.10.0-1160.6.1.el7.x86_64
0x2a0
7
kthread
vmlinux 0xffffffff940c5c21
Build id: 3.10.0-1160.6.1.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffff940c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffff94794df7
Build id: 3.10.0-1160.6.1.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffff940c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 38s! [ksoftirqd/2:19]
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 38s! [kworker/u128:2:7094]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_hda_codec_generic sb_edac snd_hda_intel coretemp snd_hda_codec snd_hda_core iosf_mbi crc32_pclmul snd_hwdep ghash_clmulni_intel snd_seq aesni_intel snd_seq_device lrw gf128mul snd_pcm glue_helper ppdev
ablk_helper vmw_balloon cryptd snd_timer snd joydev pcspkr sg soundcore parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi libata crc32c_intel serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks floppy fuse
CPU: 6 PID: 7094 Comm: kworker/u128:2 Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff96303c22b180 ti: ffff963040c9c000 task.ti: ffff963040c9c000
RIP: 0010:[<ffffffffc02a4fa9>]  [<ffffffffc02a4fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff963040c9fd90  EFLAGS: 00000202
RAX: ffffa9e900700000 RBX: ffff963040c9fd58 RCX: dead000000000200
RDX: 0000000024000000 RSI: 6b00000000000000 RDI: ffff96307bcf6d60
RBP: ffff963040c9fe18 R08: ffff96307bcf6d68 R09: a73da6f2c64f6d60
R10: a73da6f2c64f6d60 R11: 0000000000000400 R12: ffff963040c9fd50
R13: 0000000000000001 R14: ffff96307b59ad40 R15: ffff96303c22b1e8
FS:  0000000000000000(0000) GS:ffff96307b580000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f819d5ad6d0 CR3: 0000000053210000 CR4: 00000000000607e0
Call Trace:
[<ffffffff940d4b9f>] ? finish_task_switch+0x14f/0x1c0
[<ffffffff94786d4f>] ? __schedule+0x3af/0x860
[<ffffffff940bdc4f>] process_one_work+0x17f/0x440
[<ffffffff940bed66>] worker_thread+0x126/0x3c0
[<ffffffff940bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff940c5c21>] kthread+0xd1/0xe0
[<ffffffff940c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff94794df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff940c5b50>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #14141
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc01cffa9
Build id: 3.10.0-957.21.3.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffff96368ca2
Build id: 3.10.0-957.21.3.el7.x86_64
0x652
3
process_one_work
vmlinux 0xffffffff95cb9ebf
Build id: 3.10.0-957.21.3.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff95cbaf56
Build id: 3.10.0-957.21.3.el7.x86_64
0x126
5
? manage_workers.isra.25
vmlinux 0xffffffff95cbae30
Build id: 3.10.0-957.21.3.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff95cc1da1
Build id: 3.10.0-957.21.3.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff95cc1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff96375c37
Build id: 3.10.0-957.21.3.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff95cc1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u16:0:31836]
Modules linked in: tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon pcspkr sg joydev vmw_vmci i2c_piix4 parport_pc parport
ip_tables xfs libcrc32c vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper ata_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm pata_acpi drm nfit mptspi scsi_transport_spi e1000 mptscsih ata_piix libnvdimm libata crct10dif_pclmul crct10dif_common mptbase crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 31836 Comm: kworker/u16:0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff94cc7ca1b0c0 ti: ffff94cb10ef8000 task.ti: ffff94cb10ef8000
RIP: 0010:[<ffffffffc01cffa9>]  [<ffffffffc01cffa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff94cb10efbd90  EFLAGS: 00000202
RAX: ffffaaa401c00000 RBX: 0000054400000202 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff94cd2acefd60
RBP: ffff94cb10efbe18 R08: ffff94cd2acefd68 R09: accae991354efd60
R10: accae991354efd60 R11: 0000000000000000 R12: ffff94cb10efbd50
R13: 0000000000000001 R14: ffff94cd3fc9ac00 R15: ffff94cc7ca1b128
FS:  0000000000000000(0000) GS:ffff94cd3fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055bfb77a4200 CR3: 0000000425e04000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff96368ca2>] ? __schedule+0x652/0x860
[<ffffffff95cb9ebf>] process_one_work+0x17f/0x440
[<ffffffff95cbaf56>] worker_thread+0x126/0x3c0
[<ffffffff95cbae30>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff95cc1da1>] kthread+0xd1/0xe0
[<ffffffff95cc1cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffff96375c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff95cc1cd0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #13978
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc02c4fa9
Build id: 3.10.0-1127.19.1.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffff85adb6d0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x190
3
process_one_work
vmlinux 0xffffffff85abe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff85abf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff85abf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff85ac6691
Build id: 3.10.0-1127.19.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff86192d37
Build id: 3.10.0-1127.19.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff85ac65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/u64:1:15339]
Modules linked in: arc4 md4 nls_utf8 cifs ccm dns_resolver fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel ata_piix serio_raw drm libata mptspi scsi_transport_spi nfit libnvdimm mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmfsrtx_bridge_helper]
CPU: 0 PID: 15339 Comm: kworker/u64:1 Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff92a4c4439070 ti: ffff92a4c447c000 task.ti: ffff92a4c447c000
RIP: 0010:[<ffffffffc02c4fa9>]  [<ffffffffc02c4fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff92a4c447fd90  EFLAGS: 00000202
RAX: ffffae5d407c0000 RBX: ffffffff86c2b280 RCX: dead000000000200
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff92a573abad60
RBP: ffff92a4c447fe18 R08: ffff92a573abad68 R09: b569c3f6be2bad60
R10: b569c3f6be2bad60 R11: ffff92a573b1b6c0 R12: 0000000185ae5fe6
R13: 00000000004cc54b R14: ffffffff85acc112 R15: ffff92a4c447fd38
FS:  0000000000000000(0000) GS:ffff92a57ae00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb155ec1628 CR3: 000000005b9d2000 CR4: 00000000000007f0
Call Trace:
[<ffffffff85adb6d0>] ? try_to_wake_up+0x190/0x390
[<ffffffff85abe6bf>] process_one_work+0x17f/0x440
[<ffffffff85abf7d6>] worker_thread+0x126/0x3c0
[<ffffffff85abf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff85ac6691>] kthread+0xd1/0xe0
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff86192d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff85ac65c0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #11269
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc040ffa9
Build id: 3.10.0-1127.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffffa1f85922
Build id: 3.10.0-1127.el7.x86_64
0x402
3
process_one_work
vmlinux 0xffffffffa18be6bf
Build id: 3.10.0-1127.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffa18bf7d6
Build id: 3.10.0-1127.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffffa18bf6b0
Build id: 3.10.0-1127.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffa18c6691
Build id: 3.10.0-1127.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffa18c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffa1f92d37
Build id: 3.10.0-1127.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffa18c65c0
Build id: 3.10.0-1127.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [kworker/u8:0:6129]
Modules linked in: tcp_lp fuse ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev i2c_piix4 sg vmw_vmci parport_pc parport xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic vmwgfx pata_acpi drm_kms_helper
syscopyarea sysfillrect sysimgblt crct10dif_pclmul fb_sys_fops crct10dif_common ttm crc32c_intel mptspi scsi_transport_spi serio_raw drm ata_piix mptscsih libata mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 6129 Comm: kworker/u8:0 Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff9b85c08f20e0 ti: ffff9b85ca454000 task.ti: ffff9b85ca454000
RIP: 0010:[<ffffffffc040ffa9>]  [<ffffffffc040ffa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff9b85ca457d90  EFLAGS: 00000202
RAX: ffffac87c0ec0000 RBX: ffff9b86e31a6100 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff9b86e3841d60
RBP: ffff9b85ca457e18 R08: ffff9b86e3841d68 R09: 91e40d766e041d60
R10: 91e40d766e041d60 R11: 0000000000000100 R12: ffff9b85ca457d50
R13: 0000000000000001 R14: ffff9b86ffc1ad40 R15: ffff9b85c08f2148
FS:  0000000000000000(0000) GS:ffff9b86ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9125e77000 CR3: 000000010a102000 CR4: 00000000001607f0
Call Trace:
[<ffffffffa1f85922>] ? __schedule+0x402/0x840
[<ffffffffa18be6bf>] process_one_work+0x17f/0x440
[<ffffffffa18bf7d6>] worker_thread+0x126/0x3c0
[<ffffffffa18bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa18c6691>] kthread+0xd1/0xe0
[<ffffffffa18c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa1f92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa18c65c0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #4849
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc010efa9
Build id: 3.10.0-957.12.2.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffffa2cd6820
Build id: 3.10.0-957.12.2.el7.x86_64
0x190
3
process_one_work
vmlinux 0xffffffffa2cb9e3f
Build id: 3.10.0-957.12.2.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffffa2cbaed6
Build id: 3.10.0-957.12.2.el7.x86_64
0x126
5
? manage_workers.isra.25
vmlinux 0xffffffffa2cbadb0
Build id: 3.10.0-957.12.2.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffffa2cc1d21
Build id: 3.10.0-957.12.2.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffffa2cc1c50
Build id: 3.10.0-957.12.2.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffffa3375c37
Build id: 3.10.0-957.12.2.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffffa2cc1c50
Build id: 3.10.0-957.12.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 45s! [kworker/u4:1:1506]
Modules linked in: ppdev vmw_balloon coretemp joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom crc32c_intel serio_raw vmxnet3 nfit libnvdimm ata_generic pata_acpi floppy vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi mptscsih ata_piix mptbase libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1506 Comm: kworker/u4:1 Kdump: loaded Tainted: G             L ------------   3.10.0-957.12.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffffa023b63d2080 ti: ffffa02391b00000 task.ti: ffffa02391b00000
RIP: 0010:[<ffffffffc010efa9>]  [<ffffffffc010efa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffffa02391b03d90  EFLAGS: 00000202
RAX: ffffad1d80840000 RBX: ffffa02391b03da8 RCX: ffffffffa3800000
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffffa0233600fd60
RBP: ffffa02391b03e18 R08: 0000000000000000 R09: ffffffffa3ce9c00
R10: 0000000007000000 R11: 0000000000000400 R12: ffffa02391b03d50
R13: 0000000000000001 R14: ffffa0243fd1ac00 R15: ffffa023b63d20e8
FS:  0000000000000000(0000) GS:ffffa0243fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc00986afac CR3: 00000000b810c000 CR4: 00000000000007e0
Call Trace:
[<ffffffffa2cd6820>] ? try_to_wake_up+0x190/0x390
[<ffffffffa2cb9e3f>] process_one_work+0x17f/0x440
[<ffffffffa2cbaed6>] worker_thread+0x126/0x3c0
[<ffffffffa2cbadb0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa2cc1d21>] kthread+0xd1/0xe0
[<ffffffffa2cc1c50>] ? insert_kthread_work+0x40/0x40
[<ffffffffa3375c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa2cc1c50>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #3464
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc0252fa9
Build id: 3.10.0-1062.9.1.el7.x86_64
0xc9
2
? __schedule
vmlinux 0xffffffff853805a2
Build id: 3.10.0-1062.9.1.el7.x86_64
0x402
3
process_one_work
vmlinux 0xffffffff84cbe21f
Build id: 3.10.0-1062.9.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff84cbf336
Build id: 3.10.0-1062.9.1.el7.x86_64
0x126
5
? manage_workers.isra.26
vmlinux 0xffffffff84cbf210
Build id: 3.10.0-1062.9.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff84cc61f1
Build id: 3.10.0-1062.9.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff84cc6120
Build id: 3.10.0-1062.9.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff8538dd37
Build id: 3.10.0-1062.9.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff84cc6120
Build id: 3.10.0-1062.9.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 29s! [kworker/u16:2:3361]
Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel
lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm nfit crct10dif_pclmul libata mptspi crct10dif_common crc32c_intel libnvdimm scsi_transport_spi serio_raw mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 3361 Comm: kworker/u16:2 Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff97c69a3fc1c0 ti: ffff97c6cee10000 task.ti: ffff97c6cee10000
RIP: 0010:[<ffffffffc0252fa9>]  [<ffffffffc0252fa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0000:ffff97c6cee13d90  EFLAGS: 00000202
RAX: ffffa5f6c35c0000 RBX: ffff97c2ad7e0500 RCX: dead000000000200
RDX: 0000000024000000 RSI: eb00000000000000 RDI: ffff97c6da9e1d60
RBP: ffff97c6cee13e18 R08: ffff97c6da9e1d68 R09: a0e42d4e651e1d60
R10: a0e42d4e651e1d60 R11: 0000000000000002 R12: ffff97c6cee13d50
R13: 0000000000000001 R14: ffff97c6ffc9ad00 R15: ffff97c69a3fc228
FS:  0000000000000000(0000) GS:ffff97c6ffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb4ccb07000 CR3: 0000000809b18000 CR4: 00000000001607e0
Call Trace:
[<ffffffff853805a2>] ? __schedule+0x402/0x840
[<ffffffff84cbe21f>] process_one_work+0x17f/0x440
[<ffffffff84cbf336>] worker_thread+0x126/0x3c0
[<ffffffff84cbf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff84cc61f1>] kthread+0xd1/0xe0
[<ffffffff84cc6120>] ? insert_kthread_work+0x40/0x40
[<ffffffff8538dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff84cc6120>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac 
Complete report #2087
Frame # Function Binary Source or offset Line
1
mpt_fault_reset_work
mptbase 0xffffffffc01dbfa9
Build id: 3.10.0-957.10.1.el7.x86_64
0xc9
2
? try_to_wake_up
vmlinux 0xffffffff85ed6530
Build id: 3.10.0-957.10.1.el7.x86_64
0x190
3
process_one_work
vmlinux 0xffffffff85eb9d8f
Build id: 3.10.0-957.10.1.el7.x86_64
0x17f
4
worker_thread
vmlinux 0xffffffff85ebae26
Build id: 3.10.0-957.10.1.el7.x86_64
0x126
5
? manage_workers.isra.25
vmlinux 0xffffffff85ebad00
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a0
6
kthread
vmlinux 0xffffffff85ec1c71
Build id: 3.10.0-957.10.1.el7.x86_64
0xd1
7
? insert_kthread_work
vmlinux 0xffffffff85ec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
8
ret_from_fork_nospec_begin
vmlinux 0xffffffff86575c37
Build id: 3.10.0-957.10.1.el7.x86_64
0x21
9
? insert_kthread_work
vmlinux 0xffffffff85ec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [kworker/u8:1:25121]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc i2c_piix4 parport vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw
 ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ata_piix mptspi scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 25121 Comm: kworker/u8:1 Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: mpt_poll_0 mpt_fault_reset_work [mptbase]
task: ffff8a772927c100 ti: ffff8a7153b78000 task.ti: ffff8a7153b78000
RIP: 0010:[<ffffffffc01dbfa9>]  [<ffffffffc01dbfa9>] mpt_fault_reset_work+0xc9/0x47a [mptbase]
RSP: 0018:ffff8a7153b7bd90  EFLAGS: 00000202
RAX: ffff97a58ce00000 RBX: 000000000000103c RCX: 0000000000000000
RDX: 0000000024000000 RSI: 0000000000000096 RDI: ffff8a774e291d60
RBP: ffff8a7153b7be18 R08: 0000000000000000 R09: ffff8a772927c100
R10: d62251d2d8a91d60 R11: 0000000000000000 R12: ffffffff85ee12b8
R13: ffff8a7153b7bd40 R14: 0000000000000001 R15: ffff8a77cfd9ab80
FS:  0000000000000000(0000) GS:ffff8a77cfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4d46c3e000 CR3: 0000001efdae4000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 [<ffffffff85ed6530>] ? try_to_wake_up+0x190/0x390
 [<ffffffff85eb9d8f>] process_one_work+0x17f/0x440
 [<ffffffff85ebae26>] worker_thread+0x126/0x3c0
 [<ffffffff85ebad00>] ? manage_workers.isra.25+0x2a0/0x2a0
 [<ffffffff85ec1c71>] kthread+0xd1/0xe0
 [<ffffffff85ec1ba0>] ? insert_kthread_work+0x40/0x40
 [<ffffffff86575c37>] ret_from_fork_nospec_begin+0x21/0x21
 [<ffffffff85ec1ba0>] ? insert_kthread_work+0x40/0x40
Code: 45 d0 65 48 33 04 25 28 00 00 00 0f 85 33 02 00 00 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 48 8b 87 88 f3 ff ff 8b 10 <89> d0 25 00 00 00 f0 3d 00 00 00 f0 89 87 b4 f3 ff ff 0f 84 ac