Info

Permalink
Function
arcmsr_get_firmware_spec
First occurrence
2020-11-23
Last occurrence
2024-03-13
State
NEW
Components
kernel
Quality
-23

Statistics

Operating system Count
CentOS 7 465
Architecture Count
x86_64 465
Related packages Count
kernel 465
 0:3.10.0-1160.15.2.el7 381
 0:3.10.0-957.21.2.el7 48
 0:3.10.0-957.el7 13
 0:3.10.0-693.2.2.el7 9
 0:3.10.0-957.12.1.el7 8
 0:3.10.0-862.11.6.el7 3
 0:3.10.0-1160.80.1.el7 2
 0:3.10.0-957.27.2.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #929490
Frame # Function Binary Source or offset Line
1
arcmsr_get_firmware_spec
arcmsr 0xffffffffc00dd452
Build id: 3.10.0-1160.15.2.el7.x86_64
0x82
2
arcmsr_bus_reset
arcmsr 0xffffffffc00ddc67
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2e7
3
? put_prev_entity
vmlinux 0xffffffffaaae6101
Build id: 3.10.0-1160.15.2.el7.x86_64
0x31
4
scsi_try_bus_reset
vmlinux 0xffffffffaaee6256
Build id: 3.10.0-1160.15.2.el7.x86_64
0x46
5
scsi_eh_ready_devs
vmlinux 0xffffffffaaee81c1
Build id: 3.10.0-1160.15.2.el7.x86_64
0x771
6
scsi_error_handler
vmlinux 0xffffffffaaee999c
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56c
7
? scsi_eh_get_sense
vmlinux 0xffffffffaaee9430
Build id: 3.10.0-1160.15.2.el7.x86_64
0x250
8
kthread
vmlinux 0xffffffffaaac5e11
Build id: 3.10.0-1160.15.2.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffaaac5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffab194df7
Build id: 3.10.0-1160.15.2.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffaaac5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [scsi_eh_0:453]
Modules linked in: tcp_lp ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_CHECKSUM iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sunrpc xfs dm_mirror dm_region_hash dm_log dm_mod sb_edac intel_powerclamp coretemp intel_rapl iTCO_wdt iTCO_vendor_support iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_i801 lpc_ich ioatdma hpilo hpwdt sg dca ipmi_si wmi ipmi_devintf ipmi_msghandler pcc_cpufreq acpi_power_meter ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt
bnx2x fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel mpt2sas tg3 hpsa mdio libcrc32c ptp drm_panel_orientation_quirks raid_class arcmsr scsi_transport_sas pps_core fuse
CPU: 8 PID: 453 Comm: scsi_eh_0 Kdump: loaded Not tainted 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: HP ProLiant DL360 Gen9, BIOS P89 07/11/2014
task: ffff9770777f8000 ti: ffff977077b80000 task.ti: ffff977077b80000
RIP: 0010:[<ffffffffc00dd452>]  [<ffffffffc00dd452>] arcmsr_get_firmware_spec+0x82/0x5b0 [arcmsr]
RSP: 0018:ffff977077b83ca0  EFLAGS: 00000246
RAX: ffffbb5c43720000 RBX: 00000000c840579a RCX: ffffbb5c437200bc
RDX: 0000000000000000 RSI: ffffbb5c43722244 RDI: ffff9770777f6d90
RBP: ffff977077b83d10 R08: ffff977077b80000 R09: 0000000000000001
R10: 0000000000000001 R11: 7fffffffffffffff R12: ffffffffaaaae09e
R13: ffff977077b83c38 R14: 0000000000000282 R15: 0000000101b60ecb
FS:  0000000000000000(0000) GS:ffff97736fa00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8cf93d9000 CR3: 000000025c210000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc00ddc67>] arcmsr_bus_reset+0x2e7/0x5d0 [arcmsr]
[<ffffffffaaae6101>] ? put_prev_entity+0x31/0x400
[<ffffffffaaee6256>] scsi_try_bus_reset+0x46/0x100
[<ffffffffaaee81c1>] scsi_eh_ready_devs+0x771/0xc60
[<ffffffffaaee999c>] scsi_error_handler+0x56c/0x8b0
[<ffffffffaaee9430>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffffaaac5e11>] kthread+0xd1/0xe0
[<ffffffffaaac5d40>] ? insert_kthread_work+0x40/0x40
[<ffffffffab194df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaaac5d40>] ? insert_kthread_work+0x40/0x40
Code: 00 00 48 89 45 a8 4c 8d a0 3c 22 00 00 48 89 7d 98 48 89 75 b0 49 89 f7 8b 50 34 83 ca 0d 89 50 34 48 8d 88 bc 00 00 00 90 8b 11 <85> d2 79 fa ba 01 00 00 00 48 8b 45 a8 89 90 b0 00 00 00 b2 08 
Complete report #613418
Frame # Function Binary Source or offset Line
1
arcmsr_get_firmware_spec
arcmsr 0xffffffffc005d442
Build id: 3.10.0-693.2.2.el7.x86_64
0x82
2
arcmsr_bus_reset
arcmsr 0xffffffffc005dc77
Build id: 3.10.0-693.2.2.el7.x86_64
0x2e7
3
scsi_try_bus_reset
vmlinux 0xffffffff81467983
Build id: 3.10.0-693.2.2.el7.x86_64
0x43
4
scsi_eh_ready_devs
vmlinux 0xffffffff81469731
Build id: 3.10.0-693.2.2.el7.x86_64
0x771
5
scsi_error_handler
vmlinux 0xffffffff8146aecc
Build id: 3.10.0-693.2.2.el7.x86_64
0x56c
6
? scsi_eh_get_sense
vmlinux 0xffffffff8146a960
Build id: 3.10.0-693.2.2.el7.x86_64
0x250
7
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.2.2.el7.x86_64
0xcf
8
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.2.2.el7.x86_64
0x40
9
ret_from_fork
vmlinux 0xffffffff816b4f58
Build id: 3.10.0-693.2.2.el7.x86_64
0x58
10
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.2.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#17 stuck for 22s! [scsi_eh_0:424]
Modules linked in: rpcsec_gss_krb5 binfmt_misc bonding ib_isert iscsi_target_mod target_core_mod ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core sb_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw joydev gf128mul glue_helper ablk_helper pcspkr cryptd ipmi_ssif iTCO_wdt iTCO_vendor_support mei_me i2c_i801 sg mei ioatdma ipmi_si lpc_ich ipmi_devintf shpchp wmi ipmi_msghandler nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c mlx4_en sd_mod crc_t10dif crct10dif_generic mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci ttm libsas drm mlx4_core ahci igb scsi_transport_sas libahci libata crct10dif_pclmul crct10dif_common crc32c_intel ptp
pps_core dca i2c_algo_bit arcmsr i2c_core devlink dm_mirror dm_region_hash dm_log dm_mod
CPU: 17 PID: 424 Comm: scsi_eh_0 Tainted: G             L ------------   3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Supermicro X9DRW/X9DRW, BIOS 3.2 01/15/2015
task: ffff88085db8cf10 ti: ffff88085cb94000 task.ti: ffff88085cb94000
RIP: 0010:[<ffffffffc005d442>]  [<ffffffffc005d442>] arcmsr_get_firmware_spec+0x82/0x5d0 [arcmsr]
RSP: 0018:ffff88085cb97cb0  EFLAGS: 00000246
RAX: ffffc90006b60000 RBX: ffff88085cb97c70 RCX: ffffc90006b600bc
RDX: 0000000000000000 RSI: ffffc90006b62244 RDI: ffff88085cb92d90
RBP: ffff88085cb97d10 R08: ffff88085cb94000 R09: 0000000000000097
R10: 0000000000030969 R11: 0148c6ea6d80c5c0 R12: 0000000003bc1987
R13: 0000000000000286 R14: ffffffff8109927e R15: ffff88085cb97c38
FS:  0000000000000000(0000) GS:ffff88085fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3bbcdf3000 CR3: 00000000019f2000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 ffff88085cb92d90 ffff88085cb92d84 ffffc90006b60000
ffffc90006b62244 0000000000000000 0000000003bc1987 ffff88085cb90000
000000000000000d ffff88085cb90740 ffffc90006b600f8 ffffc90006b60000
Call Trace:
[<ffffffffc005dc77>] arcmsr_bus_reset+0x2e7/0x5d0 [arcmsr]
[<ffffffff81467983>] scsi_try_bus_reset+0x43/0x100
[<ffffffff81469731>] scsi_eh_ready_devs+0x771/0xc60
[<ffffffff8146aecc>] scsi_error_handler+0x56c/0x8b0
[<ffffffff8146a960>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f58>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 00 00 48 89 45 b8 4c 8d a0 3c 22 00 00 48 89 7d a8 48 89 75 c0 49 89 f7 8b 50 34 83 ca 0d 89 50 34 48 8d 88 bc 00 00 00 90 8b 11 <85> d2 79 fa ba 01 00 00 00 48 8b 45 b8 89 90 b0 00 00 00 b2 08 
Complete report #613457
Frame # Function Binary Source or offset Line
1
arcmsr_get_firmware_spec
arcmsr 0xffffffffc03fc522
Build id: 3.10.0-1160.80.1.el7.x86_64
0x82
2
arcmsr_bus_reset
arcmsr 0xffffffffc03fcd37
Build id: 3.10.0-1160.80.1.el7.x86_64
0x2e7
3
? set_next_entity
vmlinux 0xffffffffb0ee88bc
Build id: 3.10.0-1160.80.1.el7.x86_64
0x3c
4
scsi_try_bus_reset
vmlinux 0xffffffffb1304f56
Build id: 3.10.0-1160.80.1.el7.x86_64
0x46
5
scsi_eh_ready_devs
vmlinux 0xffffffffb1306f91
Build id: 3.10.0-1160.80.1.el7.x86_64
0x771
6
scsi_error_handler
vmlinux 0xffffffffb13087ac
Build id: 3.10.0-1160.80.1.el7.x86_64
0x56c
7
? scsi_eh_get_sense
vmlinux 0xffffffffb1308240
Build id: 3.10.0-1160.80.1.el7.x86_64
0x250
8
kthread
vmlinux 0xffffffffb0ecb4d1
Build id: 3.10.0-1160.80.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffb0ecb400
Build id: 3.10.0-1160.80.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffb15c51f7
Build id: 3.10.0-1160.80.1.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffb0ecb400
Build id: 3.10.0-1160.80.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [scsi_eh_0:256]
Modules linked in: xt_policy drbg ansi_cprng authenc xfrm6_mode_tunnel xfrm4_mode_tunnel wireguard(E) ip6_udp_tunnel udp_tunnel rpcsec_gss_krb5 binfmt_misc vmnet(OE) ppdev parport_pc parport tun bridge stp llc vmw_vsock_vmci_transport vsock ebtable_filter ebtables ip6table_filter devlink ip6_tables vmw_vmci xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 vmmon(OE) ah4 af_key ipt_REJECT nf_reject_ipv4 xt_multiport xt_conntrack iptable_filter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack nct6775 hwmon_vid vfat fat dm_mirror dm_region_hash dm_log dm_mod ext4 mbcache jbd2 intel_powerclamp coretemp intel_rapl kvm_intel kvm irqbypass crc32_pclmul mei_wdt ghash_clmulni_intel aesni_intel lrw joydev gf128mul
snd_soc_rt5640 glue_helper ablk_helper cryptd snd_soc_ssm4567 snd_soc_rl6231 pcspkr snd_soc_core snd_compress snd_seq snd_seq_device sg snd_pcm snd_timer snd nuvoton_cir soundcore rc_core regmap_i2c mei_me snd_soc_sst_acpi snd_soc_acpi i2c_designware_platform snd_soc_acpi_intel_match i2c_designware_core mei acpi_pad nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic i915 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci drm libahci mvsas e1000e libsas scsi_transport_sas crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw ptp sdhci_acpi pps_core sdhci video arcmsr drm_panel_orientation_quirks mmc_core i2c_hid iosf_mbi fuse
CPU: 3 PID: 256 Comm: scsi_eh_0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.80.1.el7.x86_64 #1
Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./H97 Pro4, BIOS P2.10 03/07/2018
task: ffff8fec39c12100 ti: ffff8fec39680000 task.ti: ffff8fec39680000
RIP: 0010:[<ffffffffc03fc522>]  [<ffffffffc03fc522>] arcmsr_get_firmware_spec+0x82/0x5b0 [arcmsr]
RSP: 0018:ffff8fec39683ca0  EFLAGS: 00000246
RAX: ffffb776c3740000 RBX: 00000000af14dacf RCX: ffffb776c37400bc
RDX: 0000000000000000 RSI: ffffb776c3742244 RDI: ffff8fec3967ed90
RBP: ffff8fec39683d10 R08: ffff8fec39680000 R09: 0000000000000001
R10: 0000000000000001 R11: 00112ad93c2ef0c0 R12: ffffffffb0eb2dbe
R13: ffff8fec39683c38 R14: 0000000000000282 R15: 000000021ffb8216
FS:  0000000000000000(0000) GS:ffff8fec5eb80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f46aefbc3ad CR3: 0000000315210000 CR4: 00000000001607e0
Call Trace:
[<ffffffffc03fcd37>] arcmsr_bus_reset+0x2e7/0x5d0 [arcmsr]
[<ffffffffb0ee88bc>] ? set_next_entity+0x3c/0xe0
[<ffffffffb1304f56>] scsi_try_bus_reset+0x46/0x120
[<ffffffffb1306f91>] scsi_eh_ready_devs+0x771/0xc60
[<ffffffffb13087ac>] scsi_error_handler+0x56c/0x8b0
[<ffffffffb1308240>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffffb0ecb4d1>] kthread+0xd1/0xe0
[<ffffffffb0ecb400>] ? insert_kthread_work+0x40/0x40
[<ffffffffb15c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb0ecb400>] ? insert_kthread_work+0x40/0x40
Code: 00 00 48 89 45 a8 4c 8d a0 3c 22 00 00 48 89 7d 98 48 89 75 b0 49 89 f7 8b 50 34 83 ca 0d 89 50 34 48 8d 88 bc 00 00 00 90 8b 11 <85> d2 79 fa ba 01 00 00 00 48 8b 45 a8 89 90 b0 00 00 00 b2 08 
Complete report #524727
Frame # Function Binary Source or offset Line
1
arcmsr_get_firmware_spec
arcmsr 0xffffffffc0181452
Build id: 3.10.0-862.11.6.el7.x86_64
0x82
2
arcmsr_bus_reset
arcmsr 0xffffffffc0181c67
Build id: 3.10.0-862.11.6.el7.x86_64
0x2e7
3
? sched_clock_cpu
vmlinux 0xffffffff9e2d50d5
Build id: 3.10.0-862.11.6.el7.x86_64
0x85
4
scsi_try_bus_reset
vmlinux 0xffffffff9e6a23c6
Build id: 3.10.0-862.11.6.el7.x86_64
0x46
5
scsi_eh_ready_devs
vmlinux 0xffffffff9e6a42b1
Build id: 3.10.0-862.11.6.el7.x86_64
0x771
6
scsi_error_handler
vmlinux 0xffffffff9e6a5a8c
Build id: 3.10.0-862.11.6.el7.x86_64
0x56c
7
? scsi_eh_get_sense
vmlinux 0xffffffff9e6a5520
Build id: 3.10.0-862.11.6.el7.x86_64
0x250
8
kthread
vmlinux 0xffffffff9e2bdf21
Build id: 3.10.0-862.11.6.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffff9e2bde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffff9e9255dd
Build id: 3.10.0-862.11.6.el7.x86_64
0x7
11
? insert_kthread_work
vmlinux 0xffffffff9e2bde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [scsi_eh_0:321]
Modules linked in: rpcsec_gss_krb5 xt_CHECKSUM iptable_mangle tun bridge ebtable_filter ebtables devlink xt_multiport 8021q garp mrp stp llc ipmi_si ipmi_devintf ipmi_msghandler ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c amd64_edac_mod edac_mce_amd kvm_amd kvm irqbypass sp5100_tco joydev pcspkr sg shpchp acpi_cpufreq i2c_piix4 k10temp nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm ahci pata_atiixp e1000e libahci libata i2c_core serio_raw ptp arcmsr pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 321 Comm: scsi_eh_0 Kdump: loaded Tainted: G        W    L ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: Supermicro H8SGL/H8SGL, BIOS 1.2        06/29/11  
task: ffff9c729faccf10 ti: ffff9c729fbbc000 task.ti: ffff9c729fbbc000
RIP: 0010:[<ffffffffc0181452>]  [<ffffffffc0181452>] arcmsr_get_firmware_spec+0x82/0x5b0 [arcmsr]
RSP: 0018:ffff9c729fbbfca0  EFLAGS: 00000246
RAX: ffffb16441c60000 RBX: 000000000065ffbc RCX: ffffb16441c600bc
RDX: 0000000000000000 RSI: ffffb16441c62244 RDI: ffff9c729fbbad90
RBP: ffff9c729fbbfd10 R08: ffff9c729fbbc000 R09: 0000000000000000
R10: 0000000000000004 R11: 0000000000000005 R12: ffffffff9e2a697e
R13: ffff9c729fbbfc38 R14: 0000000000000286 R15: 000000014c0fa038
FS:  000000c420021068(0000) GS:ffff9c73d7cc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f6d8e778000 CR3: 00000002fac0e000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc0181c67>] arcmsr_bus_reset+0x2e7/0x5d0 [arcmsr]
[<ffffffff9e2d50d5>] ? sched_clock_cpu+0x85/0xc0
[<ffffffff9e6a23c6>] scsi_try_bus_reset+0x46/0x100
[<ffffffff9e6a42b1>] scsi_eh_ready_devs+0x771/0xc60
[<ffffffff9e6a5a8c>] scsi_error_handler+0x56c/0x8b0
[<ffffffff9e6a5520>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffff9e2bdf21>] kthread+0xd1/0xe0
[<ffffffff9e2bde50>] ? insert_kthread_work+0x40/0x40
[<ffffffff9e9255dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff9e2bde50>] ? insert_kthread_work+0x40/0x40
Code: 00 00 48 89 45 a8 4c 8d a0 3c 22 00 00 48 89 7d 98 48 89 75 b0 49 89 f7 8b 50 34 83 ca 0d 89 50 34 48 8d 88 bc 00 00 00 90 8b 11 <85> d2 79 fa ba 01 00 00 00 48 8b 45 a8 89 90 b0 00 00 00 b2 08 
Complete report #152575
Frame # Function Binary Source or offset Line
1
arcmsr_get_firmware_spec
arcmsr 0xffffffffc03f6452
Build id: 3.10.0-957.12.1.el7.x86_64
0x82
2
arcmsr_bus_reset
arcmsr 0xffffffffc03f6c67
Build id: 3.10.0-957.12.1.el7.x86_64
0x2e7
3
? sched_clock_cpu
vmlinux 0xffffffff92cd9f15
Build id: 3.10.0-957.12.1.el7.x86_64
0x85
4
scsi_try_bus_reset
vmlinux 0xffffffff930d38d6
Build id: 3.10.0-957.12.1.el7.x86_64
0x46
5
scsi_eh_ready_devs
vmlinux 0xffffffff930d57c1
Build id: 3.10.0-957.12.1.el7.x86_64
0x771
6
scsi_error_handler
vmlinux 0xffffffff930d6f9c
Build id: 3.10.0-957.12.1.el7.x86_64
0x56c
7
? scsi_eh_get_sense
vmlinux 0xffffffff930d6a30
Build id: 3.10.0-957.12.1.el7.x86_64
0x250
8
kthread
vmlinux 0xffffffff92cc1c71
Build id: 3.10.0-957.12.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffff92cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffff93375bf7
Build id: 3.10.0-957.12.1.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffff92cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [scsi_eh_1:1709]
Modules linked in: osst 8021q garp mrp ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm rpcrdma sunrpc rdma_ucm ib_uverbs ib_iser rdma_cm ib_umad iw_cm ib_ipoib libiscsi scsi_transport_iscsi ib_cm mlx4_ib ib_core dm_mirror dm_region_hash dm_log dm_mod iTCO_wdt iTCO_vendor_support raid10 sb_edac intel_powerclamp coretemp
intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif pcspkr joydev st ch mei_me i2c_i801 mei lpc_ich ipmi_si ioatdma ipmi_devintf ipmi_msghandler ip_tables xfs libcrc32c mlx4_en raid1 raid0 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ahci isci arcmsr libahci libsas mlx4_core drm_panel_orientation_quirks igb libata mpt2sas ptp raid_class pps_core scsi_transport_sas devlink dca i2c_algo_bit sg
CPU: 4 PID: 1709 Comm: scsi_eh_1 Kdump: loaded Not tainted 3.10.0-957.12.1.el7.x86_64 #1
Hardware name: Supermicro X9DR3-F/X9DR3-F, BIOS 2.0a 03/27/2013
task: ffff9cf71b120000 ti: ffff9cf71aa7c000 task.ti: ffff9cf71aa7c000
RIP: 0010:[<ffffffffc03f6452>]  [<ffffffffc03f6452>] arcmsr_get_firmware_spec+0x82/0x5b0 [arcmsr]
RSP: 0018:ffff9cf71aa7fca0  EFLAGS: 00000246
RAX: ffffb61e06b20000 RBX: 00000000cafd615e RCX: ffffb61e06b200bc
RDX: 0000000000000000 RSI: ffffb61e06b22244 RDI: ffff9cf71a6a2d90
RBP: ffff9cf71aa7fd10 R08: ffff9cf71aa7c000 R09: 0000000000000001
R10: 00000000002b48c4 R11: 7fffffffffffffff R12: ffffffff92caa11e
R13: ffff9cf71aa7fc38 R14: 0000000000000286 R15: 00000006999319d2
FS:  0000000000000000(0000) GS:ffff9cef1fb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000002759320 CR3: 00000002b4a10000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc03f6c67>] arcmsr_bus_reset+0x2e7/0x5d0 [arcmsr]
[<ffffffff92cd9f15>] ? sched_clock_cpu+0x85/0xc0
[<ffffffff930d38d6>] scsi_try_bus_reset+0x46/0x100
[<ffffffff930d57c1>] scsi_eh_ready_devs+0x771/0xc60
[<ffffffff930d6f9c>] scsi_error_handler+0x56c/0x8b0
[<ffffffff930d6a30>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffff92cc1c71>] kthread+0xd1/0xe0
[<ffffffff92cc1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffff93375bf7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff92cc1ba0>] ? insert_kthread_work+0x40/0x40
Code: 00 00 48 89 45 a8 4c 8d a0 3c 22 00 00 48 89 7d 98 48 89 75 b0 49 89 f7 8b 50 34 83 ca 0d 89 50 34 48 8d 88 bc 00 00 00 90 8b 11 <85> d2 79 fa ba 01 00 00 00 48 8b 45 a8 89 90 b0 00 00 00 b2 08 
Complete report #54106
Frame # Function Binary Source or offset Line
1
arcmsr_get_firmware_spec
arcmsr 0xffffffffc01e8452
Build id: 3.10.0-957.27.2.el7.x86_64
0x82
2
arcmsr_bus_reset
arcmsr 0xffffffffc01e8c67
Build id: 3.10.0-957.27.2.el7.x86_64
0x2e7
3
? dequeue_task_fair
vmlinux 0xffffffffaa6e220e
Build id: 3.10.0-957.27.2.el7.x86_64
0x41e
4
scsi_try_bus_reset
vmlinux 0xffffffffaaad52c6
Build id: 3.10.0-957.27.2.el7.x86_64
0x46
5
scsi_eh_ready_devs
vmlinux 0xffffffffaaad71b1
Build id: 3.10.0-957.27.2.el7.x86_64
0x771
6
scsi_error_handler
vmlinux 0xffffffffaaad898c
Build id: 3.10.0-957.27.2.el7.x86_64
0x56c
7
? scsi_eh_get_sense
vmlinux 0xffffffffaaad8420
Build id: 3.10.0-957.27.2.el7.x86_64
0x250
8
kthread
vmlinux 0xffffffffaa6c2e81
Build id: 3.10.0-957.27.2.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffaa6c2db0
Build id: 3.10.0-957.27.2.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffaad76c1d
Build id: 3.10.0-957.27.2.el7.x86_64
0x7
11
? insert_kthread_work
vmlinux 0xffffffffaa6c2db0
Build id: 3.10.0-957.27.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [scsi_eh_0:477]
Modules linked in: tun devlink bonding ip_set nfnetlink bridge stp llc sunrpc ses enclosure scsi_transport_sas iTCO_wdt iTCO_vendor_support vfat fat skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev esas2raid(POE) mei_me sg i2c_i801 mei lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_pad acpi_cpufreq acpi_power_meter xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm i40e ahci crct10dif_pclmul crct10dif_common drm crc32c_intel libahci libata ptp pps_core arcmsr drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 14 PID: 477 Comm: scsi_eh_0 Kdump: loaded Tainted: P           OE  ------------   3.10.0-957.27.2.el7.x86_64 #1
Hardware name: Supermicro X11DPi-N(T)/X11DPi-NT, BIOS 3.1 04/26/2019
task: ffff9555fafbb0c0 ti: ffff954623918000 task.ti: ffff954623918000
RIP: 0010:[<ffffffffc01e8452>]  [<ffffffffc01e8452>] arcmsr_get_firmware_spec+0x82/0x5b0 [arcmsr]
RSP: 0018:ffff95462391bca0  EFLAGS: 00000246
RAX: ffffac364c5e0000 RBX: 00000000b433f0f6 RCX: ffffac364c5e00bc
RDX: 0000000000000000 RSI: ffffac364c5e2244 RDI: ffff954623916d90
RBP: ffff95462391bd10 R08: ffff954623918000 R09: 0000000000000001
R10: 0000000000000004 R11: 0000000000000005 R12: ffffffffaa6ab32e
R13: ffff95462391bc38 R14: 0000000000000286 R15: 000000011ae08af1
FS:  0000000000000000(0000) GS:ffff9565fdb80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e4ed4e5000 CR3: 0000001a48e10000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffc01e8c67>] arcmsr_bus_reset+0x2e7/0x5d0 [arcmsr]
[<ffffffffaa6e220e>] ? dequeue_task_fair+0x41e/0x660
[<ffffffffaaad52c6>] scsi_try_bus_reset+0x46/0x100
[<ffffffffaaad71b1>] scsi_eh_ready_devs+0x771/0xc60
[<ffffffffaaad898c>] scsi_error_handler+0x56c/0x8b0
[<ffffffffaaad8420>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffffaa6c2e81>] kthread+0xd1/0xe0
[<ffffffffaa6c2db0>] ? insert_kthread_work+0x40/0x40
[<ffffffffaad76c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffaa6c2db0>] ? insert_kthread_work+0x40/0x40
Code: 00 00 48 89 45 a8 4c 8d a0 3c 22 00 00 48 89 7d 98 48 89 75 b0 49 89 f7 8b 50 34 83 ca 0d 89 50 34 48 8d 88 bc 00 00 00 90 8b 11 <85> d2 79 fa ba 01 00 00 00 48 8b 45 a8 89 90 b0 00 00 00 b2 08 
Complete report #17648
Frame # Function Binary Source or offset Line
1
arcmsr_get_firmware_spec
arcmsr 0xffffffffc0108452
Build id: 3.10.0-957.el7.x86_64
0x82
2
arcmsr_bus_reset
arcmsr 0xffffffffc0108c67
Build id: 3.10.0-957.el7.x86_64
0x2e7
3
scsi_try_bus_reset
vmlinux 0xffffffffa56d2d46
Build id: 3.10.0-957.el7.x86_64
0x46
4
scsi_eh_ready_devs
vmlinux 0xffffffffa56d4c31
Build id: 3.10.0-957.el7.x86_64
0x771
5
scsi_error_handler
vmlinux 0xffffffffa56d640c
Build id: 3.10.0-957.el7.x86_64
0x56c
6
? scsi_eh_get_sense
vmlinux 0xffffffffa56d5ea0
Build id: 3.10.0-957.el7.x86_64
0x250
7
kthread
vmlinux 0xffffffffa52c1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
8
? insert_kthread_work
vmlinux 0xffffffffa52c1b60
Build id: 3.10.0-957.el7.x86_64
0x40
9
ret_from_fork_nospec_begin
vmlinux 0xffffffffa5974c37
Build id: 3.10.0-957.el7.x86_64
0x21
10
? insert_kthread_work
vmlinux 0xffffffffa52c1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [scsi_eh_2:910]
Modules linked in: xt_conntrack nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_filter ip_tables 8021q garp mrp binfmt_misc ipt_REJECT nf_reject_ipv4 ip_set nfnetlink bridge stp llc dm_mirror dm_region_hash dm_log dm_mod iTCO_wdt iTCO_vendor_support gpio_ich intel_powerclamp coretemp kvm_intel kvm irqbypass pcspkr joydev lpc_ich i2c_i801 sg tpm_infineon ioatdma i7core_edac acpi_cpufreq bonding xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common mgag200 crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw ahci drm libahci libata drm_panel_orientation_quirks arcmsr igb ptp pps_core dca i2c_algo_bit [last unloaded: ip6_tables]
CPU: 5 PID: 910 Comm: scsi_eh_2 Kdump: loaded Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: Supermicro X8DTU/X8DTU, BIOS 2.1c       08/03/2012
task: ffff979e2f025140 ti: ffff979d6cfd8000 task.ti: ffff979d6cfd8000
RIP: 0010:[<ffffffffc0108452>]  [<ffffffffc0108452>] arcmsr_get_firmware_spec+0x82/0x5b0 [arcmsr]
RSP: 0018:ffff979d6cfdbca0  EFLAGS: 00000246
RAX: ffffad2541c20000 RBX: 0000000078d78ace RCX: ffffad2541c200bc
RDX: 0000000000000000 RSI: ffffad2541c22244 RDI: ffff979e2e60ad90
RBP: ffff979d6cfdbd10 R08: ffff979d6cfd8000 R09: 0000000000000001
R10: 0000000000000090 R11: 7fffffffffffffff R12: ffffffffa52aa0de
R13: ffff979d6cfdbc38 R14: 0000000000000286 R15: 000000020eef8b97
FS:  0000000000000000(0000) GS:ffff979e33140000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9ff2167000 CR3: 000000026d410000 CR4: 00000000000207e0
Call Trace:
[<ffffffffc0108c67>] arcmsr_bus_reset+0x2e7/0x5d0 [arcmsr]
[<ffffffffa56d2d46>] scsi_try_bus_reset+0x46/0x100
[<ffffffffa56d4c31>] scsi_eh_ready_devs+0x771/0xc60
[<ffffffffa56d640c>] scsi_error_handler+0x56c/0x8b0
[<ffffffffa56d5ea0>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffffa52c1c31>] kthread+0xd1/0xe0
[<ffffffffa52c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa5974c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa52c1b60>] ? insert_kthread_work+0x40/0x40
Code: 00 00 48 89 45 a8 4c 8d a0 3c 22 00 00 48 89 7d 98 48 89 75 b0 49 89 f7 8b 50 34 83 ca 0d 89 50 34 48 8d 88 bc 00 00 00 90 8b 11 <85> d2 79 fa ba 01 00 00 00 48 8b 45 a8 89 90 b0 00 00 00 b2 08