Complete report #1401589
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc02e37f9
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff830c32ef
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff830c4436
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff830c4310
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x2b0
|
|
5
|
kthread
|
vmlinux |
0xffffffff830cb621
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff830cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff837c51f7
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff830cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 30s! [kworker/3:2:16932]
Modules linked in: tcp_lp xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc joydev sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vboxguest(OE) snd_intel8x0 aesni_intel snd_ac97_codec ac97_bus lrw gf128mul snd_seq glue_helper snd_seq_device snd_pcm ablk_helper cryptd snd_timer
snd soundcore parport_pc parport sg pcspkr i2c_piix4 video ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix crct10dif_pclmul libahci crct10dif_common crc32c_intel libata serio_raw e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 16932 Comm: kworker/3:2 Kdump: loaded Tainted: G W OE ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8d9714e29080 ti: ffff8d95b4494000 task.ti: ffff8d95b4494000
RIP: 0010:[<ffffffffc02e37f9>] [<ffffffffc02e37f9>] ata_sff_pio_task+0x99/0x1e0 [libata]
RSP: 0018:ffff8d95b4497df0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff8d9752318158 RCX: 0000000000000000
RDX: 00000000000003f6 RSI: ffff8d97523181a4 RDI: ffff8d9575f13cd8
RBP: ffff8d95b4497e18 R08: 0000000000000001 R09: c9a244d14f5180b8
R10: c9a244d14f5180b8 R11: 0000000000000000 R12: ffff8d9752318158
R13: ffff8d9752318000 R14: 00000000cac18e9f R15: ffff8d95b4497d58
FS: 0000000000000000(0000) GS:ffff8d975fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb8cb6e5000 CR3: 00000001f11da000 CR4: 00000000000606e0
Call Trace:
[<ffffffff830c32ef>] process_one_work+0x17f/0x440
[<ffffffff830c4436>] worker_thread+0x126/0x3c0
[<ffffffff830c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff830cb621>] kthread+0xd1/0xe0
[<ffffffff830cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff837c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff830cb550>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 5a 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff e8 18 ae e3 c2 66 90 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc 0f 1f 84 00 00 00
Complete report #1343732
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0165f19
Build id: 3.10.0-1127.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffb74be6bf
Build id: 3.10.0-1127.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffb74bf7d6
Build id: 3.10.0-1127.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffb74bf6b0
Build id: 3.10.0-1127.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffb74c6691
Build id: 3.10.0-1127.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffb74c65c0
Build id: 3.10.0-1127.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb7b92d37
Build id: 3.10.0-1127.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffffb74c65c0
Build id: 3.10.0-1127.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/2:0:90396]
Modules linked in: nls_utf8 isofs ext4 mbcache jbd2 ip_set ip6table_mangle virtio_blk tcp_diag inet_diag xt_NFQUEUE nfnetlink_log nfnetlink nfring(OE) gsch(OE) redirfs(OE) esmnet(OE) esmfile(OE) binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dsa_filter(POE) ppdev cirrus ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm drm_panel_orientation_quirks parport_pc sg parport pcspkr floppy virtio_net net_failover i2c_piix4 virtio_console failover joydev nfsd auth_rpcgss nfs_acl lockd grace
sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_scsi virtio_balloon ata_piix libata serio_raw virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 90396 Comm: kworker/2:0 Kdump: loaded Tainted: PF OEL ------------ 3.10.0-1127.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8f129d918000 ti: ffff8f1a5b7f8000 task.ti: ffff8f1a5b7f8000
RIP: 0010:[<ffffffffc0165f19>] [<ffffffffc0165f19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff8f1a5b7fbdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8f1a585f4158 RCX: 0000000000000000
RDX: 00000000000003f6 RSI: ffff8f1a585f41a4 RDI: ffff8f18985b0198
RBP: ffff8f1a5b7fbe18 R08: 0000000000000001 R09: c3962d9d557f40b8
R10: c3962d9d557f40b8 R11: 0000000000000246 R12: ffff8f1a585f4158
R13: ffff8f1a585f4000 R14: 00000000b139237e R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff8f1a5dc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2f42a8c000 CR3: 000000081d376000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb74be6bf>] process_one_work+0x17f/0x440
[<ffffffffb74bf7d6>] worker_thread+0x126/0x3c0
[<ffffffffb74bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb74c6691>] kthread+0xd1/0xe0
[<ffffffffb74c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb7b92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb74c65c0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #838830
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc036ef19
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffad6bde5f
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffad6bef76
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffad6bee50
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffad6c5e31
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0xd1
|
|
6
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffadd95ec9
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x96
|
|
7
|
|
vmlinux |
0xffffffffad6c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x40
|
|
8
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffadd95df7
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x21
|
|
9
|
|
vmlinux |
0xffffffffad6c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 29s! [kworker/3:0:4926]
Modules linked in: binfmt_misc tcp_lp nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc nfit libnvdimm snd_intel8x0 ppdev snd_ac97_codec iosf_mbi crc32_pclmul ac97_bus snd_seq ghash_clmulni_intel aesni_intel snd_seq_device snd_pcm lrw snd_timer gf128mul snd glue_helper ablk_helper soundcore cryptd sg parport_pc parport i2c_piix4 pcspkr
video ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix libahci crct10dif_pclmul crct10dif_common crc32c_intel e1000 libata serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 4926 Comm: kworker/3:0 Kdump: loaded Tainted: G W ------------ 3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffffa0b3c4023180 ti: ffffa0b8f2580000 task.ti: ffffa0b8f2580000
RIP: 0010:[<ffffffffc036ef19>] [<ffffffffc036ef19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffffa0b8f2583df0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffffa0b8f0b90158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffffa0b8f0b901a4 RDI: ffffa0b335d36d98
RBP: ffffa0b8f2583e18 R08: 0000000000000001 R09: 7d1bddd4edd900b8
R10: 7d1bddd4edd900b8 R11: 0000000000000293 R12: ffffa0b8f0b90158
R13: ffffa0b8f0b90000 R14: 000000000aa70122 R15: ffffa0b8f2583d58
FS: 0000000000000000(0000) GS:ffffa0b90dd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1488c94000 CR3: 000000034787a000 CR4: 00000000000606e0
Call Trace:
[<ffffffffad6bde5f>] process_one_work+0x17f/0x440
[<ffffffffad6bef76>] worker_thread+0x126/0x3c0
[<ffffffffad6bee50>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffad6c5e31>] kthread+0xd1/0xe0
[<ffffffffadd95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffad6c5d60>] ? insert_kthread_work+0x40/0x40
[<ffffffffadd95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffad6c5d60>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff e8 64 80 3a ed 66 90 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #820942
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc03f5f99
Build id: 3.10.0-957.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffaecb9d4f
Build id: 3.10.0-957.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffaecbade6
Build id: 3.10.0-957.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffaecbacc0
Build id: 3.10.0-957.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffaecc1c31
Build id: 3.10.0-957.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffaecc1b60
Build id: 3.10.0-957.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffaf374c37
Build id: 3.10.0-957.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffffaecc1b60
Build id: 3.10.0-957.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 29s! [kworker/1:0:22224]
Modules linked in: binfmt_misc sysmon_edr(OE) netfilter_edr(OE) ppdev sg i2c_piix4 joydev virtio_balloon parport_pc parport pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net serio_raw cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 22224 Comm: kworker/1:0 Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff9cfd7ba1a080 ti: ffff9cfd4ab78000 task.ti: ffff9cfd4ab78000
RIP: 0010:[<ffffffffc03f5f99>] [<ffffffffc03f5f99>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff9cfd4ab7bdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9d00d9cc4158 RCX: 0000000000000000
RDX: 00000000000003f6 RSI: ffff9d00d9cc41a4 RDI: ffff9d00da3dc9d8
RBP: ffff9cfd4ab7be18 R08: 0000000000000001 R09: 8bfc35cfd6ec40b8
R10: 8bfc35cfd6ec40b8 R11: 0000000000000000 R12: ffff9d00d9cc4158
R13: ffff9d00d9cc4000 R14: 0000000064d7e047 R15: ffffffffaec34919
FS: 0000000000000000(0000) GS:ffff9d00dec40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc3494195b0 CR3: 000000078dec8000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaecb9d4f>] process_one_work+0x17f/0x440
[<ffffffffaecbade6>] worker_thread+0x126/0x3c0
[<ffffffffaecbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffaecc1c31>] kthread+0xd1/0xe0
[<ffffffffaecc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf374c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaecc1b60>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #734593
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0454f96
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x96
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffb4eb9ebf
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffb4ebaf56
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffb4ebae30
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffb4ec1da1
Build id: 3.10.0-957.21.3.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffb4ec1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb5575c37
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffffb4ec1cd0
Build id: 3.10.0-957.21.3.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 37s! [kworker/1:2:13021]
Modules linked in: arc4 md4 xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter rpcsec_gss_krb5 nfsv4 nfs fscache nls_utf8 cifs ccm dns_resolver vmw_vsock_vmci_transport vsock ppdev vmw_balloon coretemp crc32_pclmul ghash_clmulni_intel ext4 aesni_intel lrw joydev gf128mul glue_helper ablk_helper mbcache cryptd jbd2 pcspkr sg parport_pc parport vmw_vmci i2c_piix4 nfsd nfs_acl lockd auth_rpcgss grace binfmt_misc sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic pata_acpi
floppy vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops vmxnet3 ttm ata_piix drm libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks involflt(POE) dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 13021 Comm: kworker/1:2 Kdump: loaded Tainted: P OEL ------------ 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff914abe6ce180 ti: ffff91498fd88000 task.ti: ffff91498fd88000
RIP: 0010:[<ffffffffc0454f96>] [<ffffffffc0454f96>] ata_sff_pio_task+0x96/0x1d0 [libata]
RSP: 0018:ffff91498fd8bdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff915195f84158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff915195f841a4 RDI: ffff9149b57cacd8
RBP: ffff91498fd8be18 R08: 0000000000000001 R09: bab93970931840b8
R10: bab93970931840b8 R11: 0000000000000000 R12: ffff915195f84158
R13: ffff915195f84000 R14: 000000007318b7c5 R15: 0000000000001400
FS: 0000000000000000(0000) GS:ffff9151bfc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f40197787d0 CR3: 00000006b9ae0000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb4eb9ebf>] process_one_work+0x17f/0x440
[<ffffffffb4ebaf56>] worker_thread+0x126/0x3c0
[<ffffffffb4ebae30>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffb4ec1da1>] kthread+0xd1/0xe0
[<ffffffffb4ec1cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb5575c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb4ec1cd0>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 46 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78
Complete report #644622
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0123f96
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x96
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffad4b9d8f
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffad4bae26
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffad4bad00
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffad4c1c71
Build id: 3.10.0-957.12.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffad4c1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffadb75bf7
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffffad4c1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 37s! [vmtoolsd:19860]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 37s! [kworker/1:1:1702]
Modules linked in: falcon_lsm_pinned_14604(E) falcon_lsm_pinned_14504(E) falcon_lsm_pinned_14306(E) falcon_lsm_pinned_14203(E) falcon_lsm_pinned_14108(E) falcon_lsm_pinned_14006(E) falcon_lsm_pinned_14005(E) falcon_lsm_pinned_13804(E) falcon_lsm_pinned_13707(E) falcon_lsm_pinned_13601(E) falcon_lsm_pinned_13501(E) falcon_lsm_pinned_13402(E) falcon_lsm_pinned_13207(E) falcon_lsm_pinned_13109(E) vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd joydev vmw_balloon pcspkr sg nfit libnvdimm parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx vmxnet3 drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi drm_panel_orientation_quirks scsi_transport_spi libata mptscsih mptbase floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 1 PID: 1702 Comm: kworker/1:1 Kdump: loaded Tainted: P EL ------------ T 3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff89d036abc100 ti: ffff89cf81c28000 task.ti: ffff89cf81c28000
RIP: 0010:[<ffffffffc0123f96>] [<ffffffffc0123f96>] ata_sff_pio_task+0x96/0x1d0 [libata]
RSP: 0018:ffff89cf81c2bdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff89d0372c4158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff89d0372c41a4 RDI: ffff89d037a20cd8
RBP: ffff89cf81c2be18 R08: 0000000000000001 R09: d8bead1f344c40b8
R10: d8bead1f344c40b8 R11: 0000000000000000 R12: ffff89d0372c4158
R13: ffff89d0372c4000 R14: 0000000040406872 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff89d03cc40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f413c67a000 CR3: 0000000199c98000 CR4: 00000000000007e0
Call Trace:
[<ffffffffad4b9d8f>] process_one_work+0x17f/0x440
[<ffffffffad4bae26>] worker_thread+0x126/0x3c0
[<ffffffffad4bad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffad4c1c71>] kthread+0xd1/0xe0
[<ffffffffad4c1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffadb75bf7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffad4c1ba0>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 46 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78
Complete report #515898
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0144f16
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x96
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffa2ebde8f
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffa2ebefa6
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffa2ebee80
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffa2ec5e61
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffa2ec5d90
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa3595de4
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xe
|
|
8
|
|
vmlinux |
0xffffffffa2ec5d90
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/2:2:1254865]
Modules linked in: squashfs loop fuse nf_conntrack_netlink xt_addrtype br_netfilter mmfs26(OE) mmfslinux(OE) tracedev(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter nfsv3 nfs fscache overlay(T) sg i2c_piix4 joydev virtio_rng virtio_balloon openafs(POE) pcspkr edac_mce_amd binfmt_misc nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover virtio_console virtio_scsi failover qxl drm_kms_helper crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ghash_clmulni_intel ata_piix drm aesni_intel libata glue_helper lrw floppy gf128mul ablk_helper cryptd serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 1254865 Comm: kworker/2:2 Kdump: loaded Tainted: P OE ------------ T 3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: oVirt RHEL, BIOS 1.14.0-1.el8s 04/01/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff97aabd321080 ti: ffff97a96271c000 task.ti: ffff97a96271c000
RIP: 0010:[<ffffffffc0144f16>] [<ffffffffc0144f16>] ata_sff_pio_task+0x96/0x1d0 [libata]
RSP: 0018:ffff97a96271fdf0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff97ab6e368158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff97ab6e3681a4 RDI: ffff97ab6f1ef158
RBP: ffff97a96271fe18 R08: 0000000000000001 R09: a151ded16b5680b8
R10: a151ded16b5680b8 R11: 0000000000000006 R12: ffff97ab6e368158
R13: ffff97ab6e368000 R14: 000000000d1f1182 R15: ffff97a96271fd70
FS: 00007fee8f443700(0000) GS:ffff97ab73280000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f7004857000 CR3: 00000000ba034000 CR4: 00000000000407e0
Call Trace:
[<ffffffffa2ebde8f>] process_one_work+0x17f/0x440
[<ffffffffa2ebefa6>] worker_thread+0x126/0x3c0
[<ffffffffa2ebee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa2ec5e61>] kthread+0xd1/0xe0
[<ffffffffa2ec5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffa3595de4>] ret_from_fork_nospec_begin+0xe/0x21
[<ffffffffa2ec5d90>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 46 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff e8 64 25 dd e2 66 90 fb 66 66 90 <66> 66 90 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78
Complete report #457934
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0310d16
Build id: 3.10.0-862.el7.x86_64
|
0x96
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffa36b2dff
Build id: 3.10.0-862.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffa36b3ac6
Build id: 3.10.0-862.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffa36b39a0
Build id: 3.10.0-862.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffa36bae31
Build id: 3.10.0-862.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffa36bad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa3d1f637
Build id: 3.10.0-862.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffffa36bad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:0:31882]
Modules linked in: binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg shpchp vmw_vmci i2c_piix4 parport_pc parport xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops mptspi ttm scsi_transport_spi mptscsih drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix mptbase serio_raw e1000 libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 31882 Comm: kworker/0:0 Kdump: loaded Tainted: G W L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/07/2011
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8b7f4cbe1fa0 ti: ffff8b7ddf50c000 task.ti: ffff8b7ddf50c000
RIP: 0010:[<ffffffffc0310d16>] [<ffffffffc0310d16>] ata_sff_pio_task+0x96/0x1d0 [libata]
RSP: 0000:ffff8b7ddf50fdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8b7f745b8158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff8b7f745b81a4 RDI: ffff8b7d76422618
RBP: ffff8b7ddf50fe18 R08: 0000000000000001 R09: d201ba11717b80b8
R10: d201ba11717b80b8 R11: 0000000000000000 R12: ffff8b7f745b8158
R13: ffff8b7f745b8000 R14: 00000000fa858e9b R15: ffffffffa36d8f3e
FS: 0000000000000000(0000) GS:ffff8b7f7fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f507cbb1000 CR3: 0000000108570000 CR4: 00000000000007f0
Call Trace:
[<ffffffffa36b2dff>] process_one_work+0x17f/0x440
[<ffffffffa36b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffa36b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffa36bae31>] kthread+0xd1/0xe0
[<ffffffffa36bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa3d1f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa36bad60>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 46 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78
Complete report #402101
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc027bf19
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff8a8bdfdf
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff8a8bf0f6
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff8a8befd0
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff8a8c5fb1
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xd1
|
|
6
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8af99ec9
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x96
|
|
7
|
|
vmlinux |
0xffffffff8a8c5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x40
|
|
8
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff8af99df7
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x21
|
|
9
|
|
vmlinux |
0xffffffff8a8c5ee0
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/1:2:3200]
Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vboxsf(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc nfit libnvdimm joydev ppdev snd_intel8x0 snd_ac97_codec iosf_mbi crc32_pclmul ghash_clmulni_intel ac97_bus snd_seq aesni_intel lrw gf128mul glue_helper snd_seq_device ablk_helper snd_pcm cryptd
vboxguest(OE) snd_timer snd soundcore sg parport_pc parport video pcspkr i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci ata_piix e1000 libata serio_raw crct10dif_pclmul crct10dif_common crc32c_intel drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 3200 Comm: kworker/1:2 Kdump: loaded Tainted: G OE ------------ 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff9dbcb2423180 ti: ffff9dbfd6dfc000 task.ti: ffff9dbfd6dfc000
RIP: 0010:[<ffffffffc027bf19>] [<ffffffffc027bf19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff9dbfd6dffdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9dbfd6dffdc0 RCX: 0000000000000000
RDX: 00000000000003f6 RSI: ffff9dc0464881a4 RDI: ffff9dc0464e4a98
RBP: ffff9dbfd6dffe18 R08: 0000000000000001 R09: 88fe849e436880b8
R10: 88fe849e436880b8 R11: 0000000000000293 R12: ffffffff8a917cfb
R13: ffff9dbfd6dffd70 R14: ffff9dc0464e4a98 R15: 0000000000000082
FS: 0000000000000000(0000) GS:ffff9dc05fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f53e470d000 CR3: 00000005076cc000 CR4: 00000000000606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8a8bdfdf>] process_one_work+0x17f/0x440
[<ffffffff8a8bf0f6>] worker_thread+0x126/0x3c0
[<ffffffff8a8befd0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8a8c5fb1>] kthread+0xd1/0xe0
[<ffffffff8af99ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff8a8c5ee0>] ? insert_kthread_work+0x40/0x40
[<ffffffff8af99df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8a8c5ee0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff e8 b4 b6 69 ca 66 90 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #375436
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0095a06
Build id: 3.10.0-693.el7.x86_64
|
0x96
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64
|
0x17a
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff810a94e6
Build id: 3.10.0-693.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
|
0xcf
|
|
6
|
|
vmlinux |
0xffffffff8108ddeb
Build id: 3.10.0-693.el7.x86_64
|
0x6bb
|
|
7
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/1:2:14843]
Modules linked in: sunrpc binfmt_misc veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device ppdev sg crc32_pclmul ghash_clmulni_intel aesni_intel snd_pcm snd_timer snd i2c_piix4 soundcore lrw gf128mul glue_helper ablk_helper cryptd pcspkr parport_pc parport video ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel serio_raw drm ahci libahci ata_piix libata e1000 i2c_core dm_mirror dm_region_hash
dm_log dm_mod
CPU: 1 PID: 14843 Comm: kworker/1:2 Tainted: G W L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8800da7fdee0 ti: ffff88020d298000 task.ti: ffff88020d298000
RIP: 0010:[<ffffffffc0095a06>] [<ffffffffc0095a06>] ata_sff_pio_task+0x96/0x1d0 [libata]
RSP: 0018:ffff88020d29bdf0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff880214778158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff8802147781a4 RDI: ffff88003648fcd8
RBP: ffff88020d29be18 R08: 0000000000000001 R09: dff73624119780b8
R10: dff73624119780b8 R11: 0000000000000220 R12: ffff880214778158
R13: ffff880214778000 R14: 000000006aeeca5d R15: 000000006aeeca5d
FS: 0000000000000000(0000) GS:ffff88021fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f684a1be000 CR3: 00000000019f2000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8802147780b8 ffff8800cef4c080 ffff88021fc96540 ffff88021fc9e300
0000000000000040 ffff88020d29be60 ffffffff810a881a 000000001fc96560
0000000000000000 ffff88021fc96560 ffff8800cef4c0b0 ffff8800da7fdee0
Call Trace:
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a94e6>] worker_thread+0x126/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff8108ddeb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 46 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff ff 14 25 50 31 a1 81 fb 66 66 90 <66> 66 90 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78
Complete report #311896
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0356f19
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffb2cbde8f
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffb2cbefa6
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffb2cbee80
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffb2cc5e61
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0xd1
|
|
6
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb3395ec9
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x96
|
|
7
|
|
vmlinux |
0xffffffffb2cc5d90
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x40
|
|
8
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb3395df7
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x21
|
|
9
|
|
vmlinux |
0xffffffffb2cc5d90
Build id: 3.10.0-1160.49.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 31s! [kworker/0:0:4410]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev intel_powerclamp iosf_mbi crc32_pclmul snd_intel8x0 ghash_clmulni_intel snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg snd_timer pcspkr snd i2c_piix4 soundcore parport_pc video parport ip_tables xfs libcrc32c
sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci ata_piix libata e1000 crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 4410 Comm: kworker/0:0 Not tainted 3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff98b75a32a100 ti: ffff98b70c410000 task.ti: ffff98b70c410000
RIP: 0010:[<ffffffffc0356f19>] [<ffffffffc0356f19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff98b70c413df0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff98b73608c158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff98b73608c1a4 RDI: ffff98b735c74b58
RBP: ffff98b70c413e18 R08: 0000000000000001 R09: 9d22c0943328c0b8
R10: 9d22c0943328c0b8 R11: ffff98b735fb8408 R12: ffff98b73608c158
R13: ffff98b73608c000 R14: 00000000644a68b1 R15: 0000000000001400
FS: 0000000000000000(0000) GS:ffff98b75d800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9d42a8e000 CR3: 00000000586ea000 CR4: 00000000000606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb2cbde8f>] process_one_work+0x17f/0x440
[<ffffffffb2cbefa6>] worker_thread+0x126/0x3c0
[<ffffffffb2cbee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb2cc5e61>] kthread+0xd1/0xe0
[<ffffffffb3395ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb2cc5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffb3395df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb2cc5d90>] ? insert_kthread_work+0x40/0x40
Complete report #295347
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc018ecf9
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff810abe2f
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff810acaf6
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff810ac9d0
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff810b4031
Build id: 3.10.0-693.21.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork
|
vmlinux |
0xffffffff816c0577
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x77
|
|
8
|
|
vmlinux |
0xffffffff810b3f60
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [kworker/0:1:109272]
Modules linked in: ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus sb_edac snd_seq edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw gf128mul glue_helper ablk_helper cryptd nfit snd_timer snd libnvdimm ppdev soundcore vmw_balloon pcspkr joydev sg vmw_vmci i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi libata mptscsih crct10dif_pclmul crct10dif_common mptbase crc32c_intel e1000 serio_raw i2c_core dm_mirror dm_region_hash dm_log
dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 109272 Comm: kworker/0:1 Tainted: G W L ------------ 3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8801741d6eb0 ti: ffff8800b2014000 task.ti: ffff8800b2014000
RIP: 0010:[<ffffffffc018ecf9>] [<ffffffffc018ecf9>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff8800b2017df0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff88081637c158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff88081637c1a4 RDI: ffff880817792618
RBP: ffff8800b2017e18 R08: 0000000000000001 R09: dfdf2f291357c0b8
R10: dfdf2f291357c0b8 R11: 0000000000000000 R12: ffff88081637c158
R13: ffff88081637c000 R14: 000000006fcec05b R15: ffffffff810d1f8e
FS: 0000000000000000(0000) GS:ffff88081d600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9db8d66000 CR3: 000000081a59e000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff810abe2f>] process_one_work+0x17f/0x440
[<ffffffff810acaf6>] worker_thread+0x126/0x3c0
[<ffffffff810ac9d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b4031>] kthread+0xd1/0xe0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #261103
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc044cd16
Build id: 3.10.0-862.3.2.el7.x86_64
|
0x96
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff924b312f
Build id: 3.10.0-862.3.2.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff924b3df6
Build id: 3.10.0-862.3.2.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff924b3cd0
Build id: 3.10.0-862.3.2.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff924bb161
Build id: 3.10.0-862.3.2.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff924bb090
Build id: 3.10.0-862.3.2.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff92b20677
Build id: 3.10.0-862.3.2.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff924bb090
Build id: 3.10.0-862.3.2.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [kworker/0:3:19858]
Modules linked in: traps(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon crc32_pclmul ghash_clmulni_intel nfit aesni_intel sg lrw gf128mul glue_helper ablk_helper cryptd libnvdimm pcspkr joydev i2c_piix4 parport_pc parport vmw_vmci shpchp binfmt_misc ip_tables xfs
libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel libata serio_raw vmxnet3 i2c_core vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 19858 Comm: kworker/0:3 Tainted: G OEL ------------ 3.10.0-862.3.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/03/2018
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8ad5e01d8000 ti: ffff8ad56285c000 task.ti: ffff8ad56285c000
RIP: 0010:[<ffffffffc044cd16>] [<ffffffffc044cd16>] ata_sff_pio_task+0x96/0x1d0 [libata]
RSP: 0018:ffff8ad56285fdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8ad771a64158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff8ad771a641a4 RDI: ffff8ad773332b58
RBP: ffff8ad56285fe18 R08: 0000000000000001 R09: d4a1c43e6ec640b8
R10: d4a1c43e6ec640b8 R11: 0000000000000000 R12: ffff8ad771a64158
R13: ffff8ad771a64000 R14: 00000000c3e09543 R15: ffffffff924d926e
FS: 0000000000000000(0000) GS:ffff8ad77fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f163ae56018 CR3: 000000002a70a000 CR4: 00000000000007f0
Call Trace:
[<ffffffff924b312f>] process_one_work+0x17f/0x440
[<ffffffff924b3df6>] worker_thread+0x126/0x3c0
[<ffffffff924b3cd0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff924bb161>] kthread+0xd1/0xe0
[<ffffffff924bb090>] ? insert_kthread_work+0x40/0x40
[<ffffffff92b20677>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff924bb090>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 46 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78
Complete report #229780
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc044bd19
Build id: 3.10.0-862.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff9fab2dff
Build id: 3.10.0-862.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff9fab3ac6
Build id: 3.10.0-862.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff9fab39a0
Build id: 3.10.0-862.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff9fabae31
Build id: 3.10.0-862.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa011f637
Build id: 3.10.0-862.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 25s! [kworker/1:0:2619]
Modules linked in: esn_cfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul
syscopyarea crct10dif_common sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 2619 Comm: kworker/1:0 Kdump: loaded Tainted: G W OEL ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff887093ab8fd0 ti: ffff8870612cc000 task.ti: ffff8870612cc000
RIP: 0010:[<ffffffffc044bd19>] [<ffffffffc044bd19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff8870612cfdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff887074ce0158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff887074ce01a4 RDI: ffff887075820318
RBP: ffff8870612cfe18 R08: 0000000000000001 R09: de3db53871ee00b8
R10: de3db53871ee00b8 R11: 0000000000000000 R12: ffff887074ce0158
R13: ffff887074ce0000 R14: 00000000b4c572a6 R15: ffffffff9fad8f3e
FS: 0000000000000000(0000) GS:ffff8870bc640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f67fd2f4000 CR3: 00000000785fa000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9fab2dff>] process_one_work+0x17f/0x440
[<ffffffff9fab3ac6>] worker_thread+0x126/0x3c0
[<ffffffff9fab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff9fabae31>] kthread+0xd1/0xe0
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa011f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #227628
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0400d16
Build id: 3.10.0-862.el7.x86_64
|
0x96
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff8feb2dff
Build id: 3.10.0-862.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff8feb3ac6
Build id: 3.10.0-862.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff8feb39a0
Build id: 3.10.0-862.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff8febae31
Build id: 3.10.0-862.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff8fec9664
Build id: 3.10.0-862.el7.x86_64
|
0x54
|
|
7
|
|
vmlinux |
0xffffffff8febad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
|
8
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff9051f637
Build id: 3.10.0-862.el7.x86_64
|
0x21
|
|
9
|
|
vmlinux |
0xffffffff8febad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 24s! [kworker/4:2:18936]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink binfmt_misc vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon sg pcspkr joydev parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common mptspi
crc32c_intel scsi_transport_spi libata mptscsih serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 18936 Comm: kworker/4:2 Tainted: G L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff9c790dff0000 ti: ffff9c79bdc18000 task.ti: ffff9c79bdc18000
RIP: 0010:[<ffffffffc0400d16>] [<ffffffffc0400d16>] ata_sff_pio_task+0x96/0x1d0 [libata]
RSP: 0018:ffff9c79bdc1bdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9c78b51f4158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff9c78b51f41a4 RDI: ffff9c78b5e8a858
RBP: ffff9c79bdc1be18 R08: 0000000000000001 R09: 8e1cc7fbb23f40b8
R10: 8e1cc7fbb23f40b8 R11: 0000000000000000 R12: ffff9c78b51f4158
R13: ffff9c78b51f4000 R14: 00000000ed994c9e R15: ffffffff8fed8f3e
FS: 0000000000000000(0000) GS:ffff9c79c6100000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f283156f000 CR3: 000000008de20000 CR4: 00000000000407e0
Call Trace:
[<ffffffff8feb2dff>] process_one_work+0x17f/0x440
[<ffffffff8feb3ac6>] worker_thread+0x126/0x3c0
[<ffffffff8feb39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff8febae31>] kthread+0xd1/0xe0
[<ffffffff8fec9664>] ? finish_task_switch+0x54/0x170
[<ffffffff8febad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff9051f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8febad60>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 46 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78
Complete report #124506
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0412f99
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff884bd1df
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff884be2f6
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff884be1d0
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff884c51b1
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff884d3fe7
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x57
|
|
7
|
|
vmlinux |
0xffffffff884c50e0
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x40
|
|
8
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff88b8bd37
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x21
|
|
9
|
|
vmlinux |
0xffffffff884c50e0
Build id: 3.10.0-1062.4.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 200s! [kworker/3:2:7560]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache tcp_lp nls_utf8 isofs fuse ebtable_filter ebtables devlink ip6table_filter ip6_tables binfmt_misc iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_hda_codec_generic joydev virtio_balloon snd_hda_intel sg snd_hda_codec snd_hda_core snd_hwdep snd_seq parport_pc parport snd_seq_device snd_pcm snd_timer snd soundcore i2c_piix4 pcspkr auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci floppy virtio_ring drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 7560 Comm: kworker/3:2 Kdump: loaded Tainted: G L ------------ 3.10.0-1062.4.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff93071af55230 ti: ffff92fd7bf04000 task.ti: ffff92fd7bf04000
RIP: 0010:[<ffffffffc0412f99>] [<ffffffffc0412f99>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff92fd7bf07df0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9307183d0158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff9307183d01a4 RDI: ffff9307df9c9a58
RBP: ffff92fd7bf07e18 R08: 0000000000000001 R09: b3e33213155d00b8
R10: b3e33213155d00b8 R11: 0000000000000000 R12: ffff9307183d0158
R13: ffff9307183d0000 R14: 0000000093dfdc75 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff93071f0c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000087519003 CR3: 0000002f47a6c000 CR4: 00000000001606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff884bd1df>] process_one_work+0x17f/0x440
[<ffffffff884be2f6>] worker_thread+0x126/0x3c0
[<ffffffff884be1d0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff884c51b1>] kthread+0xd1/0xe0
[<ffffffff884d3fe7>] ? finish_task_switch+0x57/0x1c0
[<ffffffff884c50e0>] ? insert_kthread_work+0x40/0x40
[<ffffffff88b8bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff884c50e0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #62573
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc03fff19
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff980bde9f
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff980befb6
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff980bee90
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff980c5e71
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff980c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff98793df7
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff980c5da0
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [kworker/4:1:4390]
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix serio_raw mptspi scsi_transport_spi libata mptscsih e1000 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 4390 Comm: kworker/4:1 Kdump: loaded Tainted: P W OEL ------------ 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff9cf17b971080 ti: ffff9cf1c0e1c000 task.ti: ffff9cf1c0e1c000
RIP: 0010:[<ffffffffc03fff19>] [<ffffffffc03fff19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff9cf1c0e1fdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9cf2f4b4c158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff9cf2f4b4c1a4 RDI: ffff9cf2f3fe0498
RBP: ffff9cf1c0e1fe18 R08: 0000000000000001 R09: 8c33ca1ff1d4c0b8
R10: 8c33ca1ff1d4c0b8 R11: 0000000000000000 R12: ffff9cf2f4b4c158
R13: ffff9cf2f4b4c000 R14: 000000009cc3485a R15: 000000009cc3485a
FS: 0000000000000000(0000) GS:ffff9cf2ffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000007ea098 CR3: 0000000101040000 CR4: 00000000000407e0
Call Trace:
[<ffffffff980bde9f>] process_one_work+0x17f/0x440
[<ffffffff980befb6>] worker_thread+0x126/0x3c0
[<ffffffff980bee90>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff980c5e71>] kthread+0xd1/0xe0
[<ffffffff980c5da0>] ? insert_kthread_work+0x40/0x40
[<ffffffff98793df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff980c5da0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #60660
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0447f99
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffb4cb9d8f
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffb4cbae26
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffb4cbad00
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffb4cc1c71
Build id: 3.10.0-957.12.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffb4cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb5375bf7
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffffb4cc1ba0
Build id: 3.10.0-957.12.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:1:37711]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink binfmt_misc ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm irqbypass ppdev crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev snd_ens1371
snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix e1000 mptspi scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm
CPU: 0 PID: 37711 Comm: kworker/0:1 Kdump: loaded Tainted: G W L ------------ 3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff9637b5d62080 ti: ffff9638707cc000 task.ti: ffff9638707cc000
RIP: 0010:[<ffffffffc0447f99>] [<ffffffffc0447f99>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff9638707cfdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff963870490158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff9638704901a4 RDI: ffff9638704c6318
RBP: ffff9638707cfe18 R08: 0000000000000001 R09: a71dd5146d6900b8
R10: a71dd5146d6900b8 R11: 0000000000000000 R12: ffff963870490158
R13: ffff963870490000 R14: 0000000080f156c0 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff963879600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efe4d86c000 CR3: 0000000059ff0000 CR4: 00000000001607f0
Call Trace:
[<ffffffffb4cb9d8f>] process_one_work+0x17f/0x440
[<ffffffffb4cbae26>] worker_thread+0x126/0x3c0
[<ffffffffb4cbad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffb4cc1c71>] kthread+0xd1/0xe0
[<ffffffffb4cc1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb5375bf7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb4cc1ba0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #47771
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc02b1d19
Build id: 3.10.0-862.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff81ab2dff
Build id: 3.10.0-862.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff81ab3ac6
Build id: 3.10.0-862.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff81ab39a0
Build id: 3.10.0-862.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff81abae31
Build id: 3.10.0-862.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff81abad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff8211f637
Build id: 3.10.0-862.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff81abad60
Build id: 3.10.0-862.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [kworker/1:2:77291]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul snd_seq_device glue_helper ablk_helper snd_pcm cryptd snd_timer snd ppdev soundcore vmw_balloon nfit sg joydev pcspkr libnvdimm parport_pc parport vmw_vmci i2c_piix4 shpchp ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi e1000 scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common i2c_core crc32c_intel mptbase serio_raw
CPU: 1 PID: 77291 Comm: kworker/1:2 Tainted: G L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff9ad367ab0fd0 ti: ffff9ad378fe8000 task.ti: ffff9ad378fe8000
RIP: 0010:[<ffffffffc02b1d19>] [<ffffffffc02b1d19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff9ad378febdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9ad42f988158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff9ad42f9881a4 RDI: ffff9ad2f57c8d98
RBP: ffff9ad378febe18 R08: 0000000000000001 R09: 94aedc722cb880b8
R10: 94aedc722cb880b8 R11: 0000000000000000 R12: ffff9ad42f988158
R13: ffff9ad42f988000 R14: 0000000087223ce2 R15: 0000000087223ce2
FS: 0000000000000000(0000) GS:ffff9ad438640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000cd65a000 CR3: 00000000a1c7e000 CR4: 00000000003607e0
Call Trace:
[<ffffffff81ab2dff>] process_one_work+0x17f/0x440
[<ffffffff81ab3ac6>] worker_thread+0x126/0x3c0
[<ffffffff81ab39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff81abae31>] kthread+0xd1/0xe0
[<ffffffff81abad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff8211f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff81abad60>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #16598
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc03ddf19
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff9f6bdc4f
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff9f6bed66
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff9f6bec40
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff9f6c5c21
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff9f6c5b50
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff9fd93df7
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff9f6c5b50
Build id: 3.10.0-1160.2.2.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 65s! [kworker/1:0:26962]
Modules linked in: veeamsnap(OE) sunrpc dm_mirror dm_region_hash dm_log dm_mod pcspkr joydev i2c_piix4 sg virtio_balloon ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net net_failover virtio_console failover virtio_scsi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy
CPU: 1 PID: 26962 Comm: kworker/1:0 Tainted: G OEL ------------ 3.10.0-1160.2.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org 04/01/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff95b5191a6300 ti: ffff95b576d64000 task.ti: ffff95b576d64000
RIP: 0010:[<ffffffffc03ddf19>] [<ffffffffc03ddf19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff95b576d67df0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff95b375f88158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff95b375f881a4 RDI: ffff95b574d19318
RBP: ffff95b576d67e18 R08: 0000000000000001 R09: a931bdd1731880b8
R10: a931bdd1731880b8 R11: ffff95b576e63f00 R12: ffff95b375f88158
R13: ffff95b375f88000 R14: 0000000016f00da2 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff95b57fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdb00d04504 CR3: 00000000a7f6a000 CR4: 00000000000006e0
Call Trace:
[<ffffffff9f6bdc4f>] process_one_work+0x17f/0x440
[<ffffffff9f6bed66>] worker_thread+0x126/0x3c0
[<ffffffff9f6bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9f6c5c21>] kthread+0xd1/0xe0
[<ffffffff9f6c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff9fd93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9f6c5b50>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff e8 94 90 33 df 66 90 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #14396
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0203f19
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff912be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff912bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff912bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff912c6691
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff912c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff91992d37
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff912c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/3:2:21335]
Modules linked in: cls_u32 sch_netem sch_htb ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_ipv4 nf_log_common xt_policy xt_nat nf_nat_ftp xt_REDIRECT nf_nat_redirect nf_conntrack_ftp xt_LOG xt_limit xt_multiport xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 drbg ansi_cprng xt_conntrack authenc xfrm6_mode_tunnel xfrm4_mode_tunnel tun bridge stp llc ebtable_filter ebtables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle ip6table_filter ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack devlink ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 iptable_filter xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log nfnetlink bluetooth rfkill ipmi_devintf ipmi_msghandler sunrpc ppdev
sg parport_pc parport pcspkr i2c_piix4 joydev virtio_balloon ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_scsi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci e1000 virtio_ring serio_raw drm_panel_orientation_quirks virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 21335 Comm: kworker/3:2 Kdump: loaded Tainted: G W L ------------ 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff97087d6820e0 ti: ffff97075db88000 task.ti: ffff97075db88000
RIP: 0010:[<ffffffffc0203f19>] [<ffffffffc0203f19>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0000:ffff97075db8bdf0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff9708b4020158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff9708b40201a4 RDI: ffff970735d24798
RBP: ffff97075db8be18 R08: 0000000000000001 R09: a3dcc700b12200b8
R10: a3dcc700b12200b8 R11: 0000000000000000 R12: ffff9708b4020158
R13: ffff9708b4020000 R14: 00000000679f4932 R15: ffff9707b5a2c228
FS: 0000000000000000(0000) GS:ffff9708bfd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559b0f9b7000 CR3: 0000000066810000 CR4: 00000000000006e0
Call Trace:
[<ffffffff912be6bf>] process_one_work+0x17f/0x440
[<ffffffff912bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff912bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff912c6691>] kthread+0xd1/0xe0
[<ffffffff912c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff91992d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff912c65c0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff e8 a4 3a 11 d1 66 90 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #9082
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc00b1a09
Build id: 3.10.0-693.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64
|
0x17a
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff810a94e6
Build id: 3.10.0-693.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64
|
0xcf
|
|
6
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork
|
vmlinux |
0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64
|
0x58
|
|
8
|
|
vmlinux |
0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 25s! [kworker/2:0:5499]
Modules linked in: xt_nat veth xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event edac_core snd_ens1371 iosf_mbi crc32_pclmul snd_rawmidi ghash_clmulni_intel snd_ac97_codec ppdev ac97_bus aesni_intel snd_seq lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm vmw_balloon joydev sg snd_timer pcspkr vmw_vmci snd soundcore i2c_piix4 nfit shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic
sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix crct10dif_pclmul mptscsih crct10dif_common libata crc32c_intel mptbase e1000 serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 5499 Comm: kworker/2:0 Tainted: G L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8802faf14f10 ti: ffff8803a77c0000 task.ti: ffff8803a77c0000
RIP: 0010:[<ffffffffc00b1a09>] [<ffffffffc00b1a09>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff8803a77c3df0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff880427090158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff8804270901a4 RDI: ffff880429683c98
RBP: ffff8803a77c3e18 R08: 0000000000000001 R09: dfeeebe0242900b8
R10: dfeeebe0242900b8 R11: 0000000000000000 R12: ffff880427090158
R13: ffff880427090000 R14: 000000005630794b R15: ffffffff81098ae2
FS: 0000000000000000(0000) GS:ffff88042d680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd7f0bafed8 CR3: 00000003fc66c000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8804270900b8 ffff880004857700 ffff88042d696540 ffff88042d6a0800
0000000000000080 ffff8803a77c3e60 ffffffff810a881a 000000002d696560
0000000000000000 ffff88042d696560 ffff880004857730 ffff8802faf14f10
Call Trace:
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a94e6>] worker_thread+0x126/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 64 49 c7
Complete report #6019
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc015df99
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffffaeeb9d8f
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffffaeebae26
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffffaeebad00
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffffaeec1c71
Build id: 3.10.0-957.10.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffffaeec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffaf575c37
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffffaeec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 41s! [kworker/1:0:27401]
Modules linked in: binfmt_misc ip6table_filter ip6_tables iptable_filter ppdev joydev pcspkr sg virtio_balloon parport_pc parport i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_blk serio_raw cirrus drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix ttm libata e1000 drm virtio_pci virtio_ring drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 27401 Comm: kworker/1:0 Kdump: loaded Tainted: G L ------------ 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff9f74394fa080 ti: ffff9f7438788000 task.ti: ffff9f7438788000
RIP: 0010:[<ffffffffc015df99>] [<ffffffffc015df99>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff9f743878bdf0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9f73f67f8158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff9f73f67f81a4 RDI: ffff9f7438b27558
RBP: ffff9f743878be18 R08: 0000000000000001 R09: 822fc575f39f80b8
R10: 822fc575f39f80b8 R11: 0000000000000000 R12: ffff9f73f67f8158
R13: ffff9f73f67f8000 R14: 00000000e5c05cdb R15: 0000000000000006
FS: 0000000000000000(0000) GS:ffff9f743fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555eac54f679 CR3: 0000000072410000 CR4: 00000000000006e0
Call Trace:
[<ffffffffaeeb9d8f>] process_one_work+0x17f/0x440
[<ffffffffaeebae26>] worker_thread+0x126/0x3c0
[<ffffffffaeebad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffaeec1c71>] kthread+0xd1/0xe0
[<ffffffffaeec1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf575c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaeec1ba0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7
Complete report #4288
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc00e3a09
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff810a882a
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x17a
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff810a94f6
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff810a93d0
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff810b099f
Build id: 3.10.0-693.5.2.el7.x86_64
|
0xcf
|
|
6
|
|
vmlinux |
0xffffffff8108ddfb
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x6bb
|
|
7
|
|
vmlinux |
0xffffffff810b08d0
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x40
|
|
8
|
ret_from_fork
|
vmlinux |
0xffffffff816b4fd8
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x58
|
|
9
|
|
vmlinux |
0xffffffff810b08d0
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:1:12647]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd hv_utils joydev ptp pcspkr sg pps_core i2c_piix4 i2c_core ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi hv_storvsc scsi_transport_fc hv_netvsc ata_piix hyperv_keyboard hid_hyperv scsi_tgt libata crct10dif_pclmul
hyperv_fb crct10dif_common crc32c_intel floppy hv_vmbus serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 12647 Comm: kworker/0:1 Tainted: G L ------------ 3.10.0-693.5.2.el7.x86_64 #1
Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090006 05/23/2012
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8800cfb28fd0 ti: ffff8800cfb20000 task.ti: ffff8800cfb20000
RIP: 0010:[<ffffffffc00e3a09>] [<ffffffffc00e3a09>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff8800cfb23df0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8800366c1d70 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff8800366c1dbc RDI: ffff880036d47ed8
RBP: ffff8800cfb23e18 R08: 0000000000000001 R09: dffeae50338c00b8
R10: dffeae50338c00b8 R11: 0000000000000000 R12: ffff8800366c1d70
R13: ffff8800366c0000 R14: 000000002b4ea8b5 R15: ffff880102616d40
FS: 0000000000000000(0000) GS:ffff880102600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f622faf6000 CR3: 00000000eee6b000 CR4: 00000000001406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800366c00b8 ffff8800d03ecc00 ffff880102616540 ffff88010261eb00
0000000000000000 ffff8800cfb23e60 ffffffff810a882a 0000000002616560
0000000000000000 ffff880102616560 ffff8800d03ecc30 ffff8800cfb28fd0
Call Trace:
[<ffffffff810a882a>] process_one_work+0x17a/0x440
[<ffffffff810a94f6>] worker_thread+0x126/0x3c0
[<ffffffff810a93d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b099f>] kthread+0xcf/0xe0
[<ffffffff8108ddfb>] ? do_exit+0x6bb/0xa40
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4fd8>] ret_from_fork+0x58/0x90
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 64 49 c7
Complete report #2464
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0057a09
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff810a882a
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x17a
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff810a94f6
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff810a93d0
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff810b099f
Build id: 3.10.0-693.5.2.el7.x86_64
|
0xcf
|
|
6
|
|
vmlinux |
0xffffffff810b08d0
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork
|
vmlinux |
0xffffffff816b4fd8
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x58
|
|
8
|
|
vmlinux |
0xffffffff810b08d0
Build id: 3.10.0-693.5.2.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:3:80672]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) bnep dm_mirror dm_region_hash dm_log ext4 snd_seq_midi snd_seq_midi_event mbcache jbd2 coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd dm_mod vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq joydev snd_seq_device pcspkr btusb btrtl btbcm btintel bluetooth snd_pcm snd_timer snd rfkill soundcore nfit vmw_vmci sg i2c_piix4 libnvdimm shpchp parport_pc parport nfsd auth_rpcgss nfs_acl
lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm mptspi scsi_transport_spi mptscsih mptbase ata_piix libata e1000 i2c_core
CPU: 0 PID: 80672 Comm: kworker/0:3 Tainted: G OEL ------------ 3.10.0-693.5.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff880036210fd0 ti: ffff88014b320000 task.ti: ffff88014b320000
RIP: 0010:[<ffffffffc0057a09>] [<ffffffffc0057a09>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0000:ffff88014b323df0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff880035488158 RCX: 0000000000000000
RDX: 00000000000003f6 RSI: ffff8800354881a4 RDI: ffff88003638d998
RBP: ffff88014b323e18 R08: 0000000000000001 R09: dffeb2de326880b8
R10: dffeb2de326880b8 R11: 0000000000000000 R12: ffff880035488158
R13: ffff880035488000 R14: 000000007347ae6a R15: ffff880153616d40
FS: 0000000000000000(0000) GS:ffff880153600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc9352afff0 CR3: 0000000131e5f000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800354880b8 ffff8800b4a74d80 ffff880153616540 ffff880153620600
0000000000000000 ffff88014b323e60 ffffffff810a882a 0000000053616560
0000000000000000 ffff880153616560 ffff8800b4a74db0 ffff880036210fd0
Call Trace:
[<ffffffff810a882a>] process_one_work+0x17a/0x440
[<ffffffff810a94f6>] worker_thread+0x126/0x3c0
[<ffffffff810a93d0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b099f>] kthread+0xcf/0xe0
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4fd8>] ret_from_fork+0x58/0x90
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 64 49 c7
Complete report #2454
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ata_sff_pio_task
|
libata |
0xffffffffc0154f99
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x99
|
|
2
|
process_one_work
|
vmlinux |
0xffffffff95abe21f
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x17f
|
|
3
|
worker_thread
|
vmlinux |
0xffffffff95abf336
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x126
|
|
4
|
|
vmlinux |
0xffffffff95abf210
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x2a0
|
|
5
|
kthread
|
vmlinux |
0xffffffff95ac61f1
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xd1
|
|
6
|
|
vmlinux |
0xffffffff95ac6120
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x40
|
|
7
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff9618dd37
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x21
|
|
8
|
|
vmlinux |
0xffffffff95ac6120
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 24s! [httpd:30636]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [kworker/0:10:29681]
Modules linked in: ip6table_filter ip6_tables tcp_diag inet_diag xt_time nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 xt_owner xt_multiport iptable_filter ext4 mbcache jbd2 loop dm_mirror dm_region_hash dm_log dm_mod ppdev parport_pc parport sg pcspkr joydev i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata e1000 virtio_pci serio_raw virtio_ring virtio drm_panel_orientation_quirks floppy
CPU: 0 PID: 29681 Comm: kworker/0:10 Kdump: loaded Tainted: G L ------------ 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
Workqueue: ata_sff ata_sff_pio_task [libata]
task: ffff8bbc87aeb150 ti: ffff8bbf07708000 task.ti: ffff8bbf07708000
RIP: 0010:[<ffffffffc0154f99>] [<ffffffffc0154f99>] ata_sff_pio_task+0x99/0x1d0 [libata]
RSP: 0018:ffff8bbf0770bdf0 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff8bc013bf4158 RCX: 0000000000000000
RDX: 0000000000000376 RSI: ffff8bc013bf41a4 RDI: ffff8bc014ed6c18
RBP: ffff8bbf0770be18 R08: 0000000000000001 R09: d0ff3cc310df40b8
R10: d0ff3cc310df40b8 R11: ffff8bbff4ffd100 R12: ffff8bc013bf4158
R13: ffff8bc013bf4000 R14: 000000002354395d R15: ffff8bbd7a353b80
FS: 0000000000000000(0000) GS:ffff8bc027c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055d62f4a08c8 CR3: 0000000036830000 CR4: 00000000000006f0
Call Trace:
[<ffffffff95abe21f>] process_one_work+0x17f/0x440
[<ffffffff95abf336>] worker_thread+0x126/0x3c0
[<ffffffff95abf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff95ac61f1>] kthread+0xd1/0xe0
[<ffffffff95ac6120>] ? insert_kthread_work+0x40/0x40
[<ffffffff9618dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff95ac6120>] ? insert_kthread_work+0x40/0x40
Code: 50 25 01 00 01 00 48 83 f8 01 74 52 49 c7 44 24 f8 00 00 00 00 49 8b bc 24 58 ff ff ff e8 74 1e 9c d5 66 90 fb 66 0f 1f 44 00 00 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 84 c0 78 6a 49 c7