Complete report #1096285
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff9a36b355
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x15
|
|
2
|
e1000_update_stats
|
e1000 |
0xffffffffc02b64dc
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x94c
|
|
3
|
e1000_watchdog
|
e1000 |
0xffffffffc02b658d
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x6d
|
|
4
|
|
xfs |
0xffffffffc04c9ff8
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x58
|
|
5
|
process_one_work
|
vmlinux |
0xffffffff99cb9d8f
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffff99cbae26
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffff99cbad00
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffff99cc1c71
Build id: 3.10.0-957.12.1.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffff99cd08c7
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x57
|
|
10
|
|
vmlinux |
0xffffffff99cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
|
11
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff9a375bf7
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x21
|
|
12
|
|
vmlinux |
0xffffffff99cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 50s! [kworker/0:1:110285]
Modules linked in: xt_nat veth tcp_lp nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio pcspkr sg parport_pc parport vmw_vmci i2c_piix4 fuse ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper mptspi
scsi_transport_spi syscopyarea sysfillrect sysimgblt ata_generic pata_acpi fb_sys_fops ttm mptscsih drm ata_piix e1000 mptbase nfit libata drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 110285 Comm: kworker/0:1 Tainted: G L ------------ 3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: events e1000_watchdog [e1000]
task: ffff9141a9039040 ti: ffff9140ce388000 task.ti: ffff9140ce388000
RIP: 0010:[<ffffffff9a36b355>] [<ffffffff9a36b355>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff9140ce38bd90 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff99ca9aed RCX: 0000ba2d8d4ce593
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff9140ce38bd90 R08: ffff9141ace010b8 R09: baf8ddc16a2010b0
R10: baf8ddc16a2010b0 R11: 0000000000000001 R12: ffff913db54264e8
R13: ffffffff9ace3c00 R14: ffffffff9ace3c00 R15: ffff913db54264e8
FS: 0000000000000000(0000) GS:ffff9141ad600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f599eae2000 CR3: 0000000066e10000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc02b64dc>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc02b658d>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffffc04c9ff8>] ? xfs_reclaim_work_queue+0x58/0x60 [xfs]
[<ffffffff99cb9d8f>] process_one_work+0x17f/0x440
[<ffffffff99cbae26>] worker_thread+0x126/0x3c0
[<ffffffff99cbad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff99cc1c71>] kthread+0xd1/0xe0
[<ffffffff99cd08c7>] ? finish_task_switch+0x57/0x1c0
[<ffffffff99cc1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9a375bf7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff99cc1ba0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #565375
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff8a76a0f5
Build id: 3.10.0-957.el7.x86_64
|
0x15
|
|
2
|
e1000_update_stats
|
e1000 |
0xffffffffc03354dc
Build id: 3.10.0-957.el7.x86_64
|
0x94c
|
|
3
|
e1000_watchdog
|
e1000 |
0xffffffffc033558d
Build id: 3.10.0-957.el7.x86_64
|
0x6d
|
|
4
|
|
xfs |
0xffffffffc0505ff8
Build id: 3.10.0-957.el7.x86_64
|
0x58
|
|
5
|
process_one_work
|
vmlinux |
0xffffffff8a0b9d4f
Build id: 3.10.0-957.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffff8a0bade6
Build id: 3.10.0-957.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffff8a0bacc0
Build id: 3.10.0-957.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffff8a0c1c31
Build id: 3.10.0-957.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffff8a0c1b60
Build id: 3.10.0-957.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff8a774c1d
Build id: 3.10.0-957.el7.x86_64
|
0x7
|
|
11
|
|
vmlinux |
0xffffffff8a0c1b60
Build id: 3.10.0-957.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 56s! [kworker/0:2:22551]
INFO: rcu_sched detected stalls on CPUs/tasks: { 0}
Modules linked in:
tcp_lp rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus snd_seq
snd_seq_device btusb snd_pcm btrtl btbcm btintel bluetooth snd_timer snd pcspkr joydev soundcore rfkill sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi vmwgfx serio_raw drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix e1000 mptspi scsi_transport_spi mptscsih libata drm_panel_orientation_quirks nfit mptbase libnvdimm
CPU: 0 PID: 22551 Comm: kworker/0:2 Kdump: loaded Tainted: G L ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: events e1000_watchdog [e1000]
task: ffff8dfacc686180 ti: ffff8dfaa7624000 task.ti: ffff8dfaa7624000
RIP: 0010:[<ffffffff8a76a0f5>] [<ffffffff8a76a0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8dfaa7627d90 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff8a0a9aad RCX: 000014765eecb107
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff8dfaa7627d90 R08: ffff8dfb755730b8 R09: c811b89eb29730b0
R10: c811b89eb29730b0 R11: 0000000000000000 R12: ffff8dfb722e74e8
R13: ffffffff8b0e3c00 R14: ffffffff8b0e3c00 R15: ffff8dfb722e74e8
FS: 0000000000000000(0000) GS:ffff8dfb79600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f28021d6e00 CR3: 000000013407e000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc03354dc>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc033558d>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffffc0505ff8>] ? xfs_reclaim_work_queue+0x58/0x60 [xfs]
[<ffffffff8a0b9d4f>] process_one_work+0x17f/0x440
[<ffffffff8a0bade6>] worker_thread+0x126/0x3c0
[<ffffffff8a0bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff8a0c1c31>] kthread+0xd1/0xe0
[<ffffffff8a0c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff8a774c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff8a0c1b60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #443441
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffb076b175
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x15
|
|
2
|
e1000_update_stats
|
e1000 |
0xffffffffc04f84dc
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x94c
|
|
3
|
e1000_watchdog
|
e1000 |
0xffffffffc04f858d
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x6d
|
|
4
|
|
xfs |
0xffffffffc06c3ff8
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x58
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffb00b9d8f
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffb00bae26
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffb00bad00
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffb00c1c71
Build id: 3.10.0-957.10.1.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffb00c1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb0775c37
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffffb00c1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 44s! [kworker/1:0:70084]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev snd_seq_midi iosf_mbi crc32_pclmul snd_seq_midi_event ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 joydev snd_rawmidi pcspkr snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi e1000 libata scsi_transport_spi mptscsih nfit mptbase drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 70084 Comm: kworker/1:0 Kdump: loaded Tainted: G W L ------------ 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: events e1000_watchdog [e1000]
task: ffff91c4beb5e180 ti: ffff91c2c4bc8000 task.ti: ffff91c2c4bc8000
RIP: 0010:[<ffffffffb076b175>] [<ffffffffb076b175>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff91c2c4bcbd90 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffb00a9aed RCX: 00174eeabefb79bc
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff91c2c4bcbd90 R08: ffff91c43c6af0b8 R09: b8eea01879aaf0b0
R10: b8eea01879aaf0b0 R11: 0000000000000001 R12: ffff91c2f4c974e8
R13: ffff91c4f5653900 R14: ffff91c4f5653900 R15: ffff91c2f4c974e8
FS: 0000000000000000(0000) GS:ffff91c4f5640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe428456000 CR3: 0000000224fda000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc04f84dc>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc04f858d>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffffc06c3ff8>] ? xfs_reclaim_work_queue+0x58/0x60 [xfs]
[<ffffffffb00b9d8f>] process_one_work+0x17f/0x440
[<ffffffffb00bae26>] worker_thread+0x126/0x3c0
[<ffffffffb00bad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffb00c1c71>] kthread+0xd1/0xe0
[<ffffffffb00c1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb0775c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb00c1ba0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #187873
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffbd115375
Build id: 3.10.0-862.el7.x86_64
|
0x15
|
|
2
|
e1000_update_stats
|
e1000 |
0xffffffffc046645c
Build id: 3.10.0-862.el7.x86_64
|
0x94c
|
|
3
|
e1000_watchdog
|
e1000 |
0xffffffffc046650d
Build id: 3.10.0-862.el7.x86_64
|
0x6d
|
|
4
|
|
xfs |
0xffffffffc0698838
Build id: 3.10.0-862.el7.x86_64
|
0x58
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffbcab2dff
Build id: 3.10.0-862.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffbcab3ac6
Build id: 3.10.0-862.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffbcab39a0
Build id: 3.10.0-862.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffbcabae31
Build id: 3.10.0-862.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffbcabad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffbd11f61d
Build id: 3.10.0-862.el7.x86_64
|
0x7
|
|
11
|
|
vmlinux |
0xffffffffbcabad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 32s! [kworker/0:0:54890]
Modules linked in: nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul snd_ens1371 glue_helper ablk_helper cryptd snd_rawmidi snd_ac97_codec ac97_bus vmw_balloon snd_seq snd_seq_device snd_pcm joydev snd_timer pcspkr sg snd soundcore vmw_vmci i2c_piix4 shpchp nfit libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx
crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm ata_piix mptspi scsi_transport_spi mptscsih libata e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 54890 Comm: kworker/0:0 Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: events e1000_watchdog [e1000]
task: ffff9e5d98b49fa0 ti: ffff9e5d861a4000 task.ti: ffff9e5d861a4000
RIP: 0010:[<ffffffffbd115375>] [<ffffffffbd115375>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9e5d861a7d90 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9e5d861a7d30 RCX: 00000a15919d7fd9
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff9e5d861a7d90 R08: ffff9e5dba1f70b8 R09: 8688b5dff75f70b0
R10: 8688b5dff75f70b0 R11: 0000000000000000 R12: ffff9e5d861a7d10
R13: 000000010058d5c0 R14: ffff9e5d3504f4e0 R15: ffffffffbdabfc00
FS: 0000000000000000(0000) GS:ffff9e5dba600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fae85141000 CR3: 0000000096ed4000 CR4: 00000000003607f0
Call Trace:
[<ffffffffc046645c>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc046650d>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffffc0698838>] ? xfs_reclaim_work_queue+0x58/0x60 [xfs]
[<ffffffffbcab2dff>] process_one_work+0x17f/0x440
[<ffffffffbcab3ac6>] worker_thread+0x126/0x3c0
[<ffffffffbcab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffbcabae31>] kthread+0xd1/0xe0
[<ffffffffbcabad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffbd11f61d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffbcabad60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #111586
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffb158a3f5
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x15
|
|
2
|
e1000_update_stats
|
e1000 |
0xffffffffc031260c
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x94c
|
|
3
|
e1000_watchdog
|
e1000 |
0xffffffffc03126bd
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x6d
|
|
4
|
|
xfs |
0xffffffffc0591d68
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x58
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffb0ebde3f
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffb0ebef56
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffb0ebee30
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffb0ec5e11
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffb0ec5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb1594df7
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffffb0ec5d40
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [kworker/0:0:122721]
Modules linked in: talpa_vfshook(OE) talpa_pedconnector(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) tcp_diag inet_diag talpa_pedevice(OE) rfcomm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport bnep vsock sunrpc snd_seq_midi snd_seq_midi_event btusb snd_ens1371 btrtl iosf_mbi btbcm snd_rawmidi btintel bluetooth snd_ac97_codec crc32_pclmul ac97_bus rfkill
snd_seq ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw gf128mul snd_timer glue_helper ablk_helper snd soundcore cryptd ppdev pcspkr joydev vmw_balloon sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect nfit sysimgblt fb_sys_fops ttm libnvdimm ata_piix mptspi drm scsi_transport_spi libata crct10dif_pclmul crct10dif_common crc32c_intel mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: talpa_syscallhook]
CPU: 0 PID: 122721 Comm: kworker/0:0 Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
Workqueue: events e1000_watchdog [e1000]
task: ffff9dad4621a100 ti: ffff9daced798000 task.ti: ffff9daced798000
RIP: 0010:[<ffffffffb158a3f5>] [<ffffffffb158a3f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9daced79bd90 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffb0eada43 RCX: 000e056b863ab980
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff9daced79bd90 R08: ffff9dadb97850b8 R09: 8948b7cc76b850b0
R10: 8948b7cc76b850b0 R11: ffff9dad975ebf00 R12: ffff9dac75e8e4e0
R13: ffffffffb1f04bc0 R14: ffffffffb1f04bc0 R15: ffff9dac75e8e4e0
FS: 0000000000000000(0000) GS:ffff9daed3e00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007eff412df080 CR3: 000000019824e000 CR4: 00000000001607f0
Call Trace:
[<ffffffffc031260c>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc03126bd>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffffc0591d68>] ? xfs_reclaim_work_queue+0x58/0x60 [xfs]
[<ffffffffb0ebde3f>] process_one_work+0x17f/0x440
[<ffffffffb0ebef56>] worker_thread+0x126/0x3c0
[<ffffffffb0ebee30>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb0ec5e11>] kthread+0xd1/0xe0
[<ffffffffb0ec5d40>] ? insert_kthread_work+0x40/0x40
[<ffffffffb1594df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb0ec5d40>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #17642
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff983883c5
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x15
|
|
2
|
e1000_update_stats
|
e1000 |
0xffffffffc04d760c
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x94c
|
|
3
|
e1000_watchdog
|
e1000 |
0xffffffffc04d76bd
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x6d
|
|
4
|
|
xfs |
0xffffffffc06ded98
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x58
|
|
5
|
process_one_work
|
vmlinux |
0xffffffff97cbe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffff97cbf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffff97cbf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffff97cc6691
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffff97cc65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff98392d1d
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x7
|
|
11
|
|
vmlinux |
0xffffffff97cc65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [kworker/0:2:46822]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter fuse vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi ppdev snd_ac97_codec ac97_bus snd_seq btusb snd_seq_device btrtl btbcm btintel vmw_balloon bluetooth aesni_intel lrw gf128mul glue_helper snd_pcm ablk_helper cryptd joydev pcspkr snd_timer rfkill snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm crct10dif_pclmul crct10dif_common libnvdimm crc32c_intel ata_piix mptspi e1000 libata scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 46822 Comm: kworker/0:2 Kdump: loaded Not tainted 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: events e1000_watchdog [e1000]
task: ffff9b8ec550b150 ti: ffff9b8e69da0000 task.ti: ffff9b8e69da0000
RIP: 0010:[<ffffffff983883c5>] [<ffffffff983883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff9b8e69da3d90 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff97cae323 RCX: 000156f3f56b48dd
RDX: 0000000000000000 RSI: 0000000000000246 RDI: 0000000000000246
RBP: ffff9b8e69da3d90 R08: ffff9b8f345a10b8 R09: 91c2ca26f19a10b0
R10: 91c2ca26f19a10b0 R11: 0000000000000005 R12: ffff9b8f2fc274e0
R13: ffffffff98d03bc0 R14: ffffffff98d03bc0 R15: ffff9b8f2fc274e0
FS: 0000000000000000(0000) GS:ffff9b8f35600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1731547eca CR3: 0000000233c34000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc04d760c>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc04d76bd>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffffc06ded98>] ? xfs_reclaim_work_queue+0x58/0x60 [xfs]
[<ffffffff97cbe6bf>] process_one_work+0x17f/0x440
[<ffffffff97cbf7d6>] worker_thread+0x126/0x3c0
[<ffffffff97cbf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff97cc6691>] kthread+0xd1/0xe0
[<ffffffff97cc65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff98392d1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff97cc65c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48