Complete report #1390795
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffffbb5b8c6b
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x4b
|
|
2
|
|
virtio_pci |
0xffffffffc01b7c26
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc01a7f90
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc035b0f2
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffbb2c32ef
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffbb2c4436
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffbb2c4310
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x2b0
|
|
8
|
kthread
|
vmlinux |
0xffffffffbb2cb621
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffbb2cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffbb9c51f7
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffffbb2cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 60s! [kworker/0:0:12867]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_18504(E) iptable_filter iosf_mbi crc32_pclmul ghash_clmulni_intel iTCO_wdt iTCO_vendor_support bochs_drm ttm drm_kms_helper aesni_intel syscopyarea lrw sysfillrect gf128mul glue_helper sysimgblt ablk_helper cryptd fb_sys_fops joydev pcspkr lpc_ich i2c_i801 drm virtio_rng virtio_balloon drm_panel_orientation_quirks ip_tables xfs libcrc32c virtio_net net_failover failover virtio_blk virtio_console ahci libahci crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 12867 Comm: kworker/0:0 Kdump: loaded Tainted: P E ------------ 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.16.3-4.el9 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9117faf66300 ti: ffff9118f4fe8000 task.ti: ffff9118f4fe8000
RIP: 0010:[<ffffffffbb5b8c6b>] [<ffffffffbb5b8c6b>] iowrite16+0x4b/0x50
RSP: 0018:ffff9118f4febd90 EFLAGS: 00000296
RAX: ffffffffc01b7c10 RBX: 0000000000000004 RCX: ffff9118f66f6940
RDX: ffffa52e00ea0008 RSI: ffffa52e00ea0008 RDI: 0000000000000002
RBP: ffff9118f4febd98 R08: 00000000000000d0 R09: ffff9118f4febdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff9118ffc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000015f1590 CR3: 00000001837ea000 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
Call Trace:
[<ffffffffc01b7c26>] ? vp_notify+0x16/0x30 [virtio_pci]
[<ffffffffc01a7f90>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc035b0f2>] update_balloon_stats_func+0x82/0xa7 [virtio_balloon]
[<ffffffffbb2c32ef>] process_one_work+0x17f/0x440
[<ffffffffbb2c4436>] worker_thread+0x126/0x3c0
[<ffffffffbb2c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffffbb2cb621>] kthread+0xd1/0xe0
[<ffffffffbb2cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffbb9c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbb2cb550>] ? insert_kthread_work+0x40/0x40
Code: cc 0f 1f 80 00 00 00 00 55 48 c7 c6 06 28 cb bb 48 89 d7 48 89 e5 e8 35 fe ff ff 5d c3 cc cc cc cc 0f 1f 80 00 00 00 00 66 89 3e <c3> cc cc cc cc 48 81 fe ff ff 03 00 48 89 f2 77 34 48 81 fe 00
Complete report #541544
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffffb0768abb
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x3b
|
|
2
|
|
virtio_pci |
0xffffffffc02f3a66
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc02d1710
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc057f022
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffb04b613f
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffb04b71d6
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffb04b70b0
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffb04bdf21
Build id: 3.10.0-862.11.6.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffb04bde50
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb0b255f7
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffffb04bde50
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [kworker/0:2:19759]
Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter bridge stp llc binfmt_misc iptable_nat nf_nat_ipv4 nf_nat ip6table_mangle ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables iptable_mangle ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter overlay(T) dm_mirror dm_region_hash ext4 dm_log dm_mod mbcache jbd2 sg i2c_piix4 ppdev shpchp pcspkr joydev virtio_balloon parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_net virtio_scsi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci i2c_core serio_raw virtio_ring virtio
floppy
CPU: 0 PID: 19759 Comm: kworker/0:2 Kdump: loaded Tainted: G ------------ T 3.10.0-862.11.6.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff96508f90af70 ti: ffff965001148000 task.ti: ffff965001148000
RIP: 0010:[<ffffffffb0768abb>] [<ffffffffb0768abb>] iowrite16+0x3b/0x40
RSP: 0018:ffff96500114bd90 EFLAGS: 00000296
RAX: ffffffffc02f3a50 RBX: 0000000000000000 RCX: ffff9655053c2940
RDX: ffffb8cf83592008 RSI: ffffb8cf83592008 RDI: 0000000000000002
RBP: ffff96500114bd98 R08: 00000000000000d0 R09: 0000000000000000
R10: ffff96500114bdd0 R11: 0000000000000006 R12: 00000000000246d5
R13: 000000000002a0c3 R14: 000000000791e645 R15: 0000001de22cfcbd
FS: 0000000000000000(0000) GS:ffff96550d800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4746ffccb0 CR3: 00000004f8b22000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc02f3a66>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc02d1710>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc057f022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffb04b613f>] process_one_work+0x17f/0x440
[<ffffffffb04b71d6>] worker_thread+0x126/0x3c0
[<ffffffffb04b70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffb04bdf21>] kthread+0xd1/0xe0
[<ffffffffb04bde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb0b255f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb04bde50>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 62 7d e9 b0 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00
Complete report #319166
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffff86b904dc
Build id: 3.10.0-957.el7.x86_64
|
0x1c
|
|
2
|
|
virtio_pci |
0xffffffffc0369b76
Build id: 3.10.0-957.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc0352720
Build id: 3.10.0-957.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc0597022
Build id: 3.10.0-957.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffff868b9d4f
Build id: 3.10.0-957.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffff868bade6
Build id: 3.10.0-957.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffff868bacc0
Build id: 3.10.0-957.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffff868c1c31
Build id: 3.10.0-957.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffff868c1b60
Build id: 3.10.0-957.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff86f74c37
Build id: 3.10.0-957.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffff868c1b60
Build id: 3.10.0-957.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 105s! [kworker/3:2:28071]
Modules linked in: kbase(OE) syshook_linux(OE) binfmt_misc udp_diag tcp_diag inet_diag fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel virtio_balloon aesni_intel parport_pc lrw parport gf128mul glue_helper i2c_piix4 ablk_helper cryptd joydev pcspkr ip_tables ext4 mbcache jbd2 virtio_console
virtio_blk virtio_net ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kbase]
CPU: 3 PID: 28071 Comm: kworker/3:2 Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9b50245de180 ti: ffff9b4fc3b8c000 task.ti: ffff9b4fc3b8c000
RIP: 0010:[<ffffffff86b904dc>] [<ffffffff86b904dc>] iowrite16+0x1c/0x40
RSP: 0018:ffff9b4fc3b8fd90 EFLAGS: 00000206
RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff9b5fb371f000
RDX: 000000000000c0f0 RSI: 000000000001c0f0 RDI: 0000000000000002
RBP: ffff9b4fc3b8fd98 R08: 00000000000000d0 R09: ffff9b4fc3b8fdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000003
R13: 0000000000000032 R14: 000000000004c036 R15: 00000000012668a5
FS: 0000000000000000(0000) GS:ffff9b5fbf2c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f25a40caf89 CR3: 000000103fe5c000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0369b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc0352720>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc0597022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffff868b9d4f>] process_one_work+0x17f/0x440
[<ffffffff868bade6>] worker_thread+0x126/0x3c0
[<ffffffff868bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff868c1c31>] kthread+0xd1/0xe0
[<ffffffff868c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff86f74c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff868c1b60>] ? insert_kthread_work+0x40/0x40
Code: ff ff 5d c3 0f 1f 00 40 88 3e c3 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 00 01 00 76 0b 0f b7 d6 89 f8 66 ef <c3> 0f 1f 00 55 48 c7 c6 51 40 2a 87 48 89 d7 48 89 e5 e8 5d fe
Complete report #255227
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffffbdb9242b
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x3b
|
|
2
|
|
virtio_pci |
0xffffffffc0096b76
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc008a720
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc042f022
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffbd8baf9f
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffbd8bc036
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffbd8bbf10
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffbd8c2e81
Build id: 3.10.0-957.27.2.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffbd8c2db0
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffbdf76c37
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffffbd8c2db0
Build id: 3.10.0-957.27.2.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:2:19140]
Modules linked in: edr(POE) edr_gpl(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul sg glue_helper ablk_helper cryptd joydev pcspkr virtio_rng virtio_balloon parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic virtio_console virtio_net virtio_scsi sr_mod cdrom ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common
crc32c_intel floppy serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: edr_gpl]
CPU: 0 PID: 19140 Comm: kworker/0:2 Kdump: loaded Tainted: P OE ------------ 3.10.0-957.27.2.el7.x86_64 #1
Hardware name: oVirt oVirt Node, BIOS 1.11.0-2.el7 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9d2a8330a080 ti: ffff9d2aa3b58000 task.ti: ffff9d2aa3b58000
RIP: 0010:[<ffffffffbdb9242b>] [<ffffffffbdb9242b>] iowrite16+0x3b/0x40
RSP: 0018:ffff9d2aa3b5bd90 EFLAGS: 00000296
RAX: ffffffffc0096b60 RBX: 0000000000000000 RCX: ffff9d2ab4b59940
RDX: ffffaa1b00440008 RSI: ffffaa1b00440008 RDI: 0000000000000002
RBP: ffff9d2aa3b5bd98 R08: 00000000000000d0 R09: ffff9d2aa3b5bdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 000000000000038a
R13: 0000000000000ed5 R14: 00000000003ebc78 R15: 0000000004d90a38
FS: 0000000000000000(0000) GS:ffff9d2afd200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0f12ade000 CR3: 000000002334a000 CR4: 00000000003606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0096b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc008a720>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc042f022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffbd8baf9f>] process_one_work+0x17f/0x440
[<ffffffffbd8bc036>] worker_thread+0x126/0x3c0
[<ffffffffbd8bbf10>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffbd8c2e81>] kthread+0xd1/0xe0
[<ffffffffbd8c2db0>] ? insert_kthread_work+0x40/0x40
[<ffffffffbdf76c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbd8c2db0>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 2e 4d 2a be 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00
Complete report #221999
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffffb6fa222b
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x3b
|
|
2
|
|
virtio_pci |
0xffffffffc01b4b76
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc01a8eb0
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc043a0d2
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffb6cbde6f
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffb6cbef86
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffb6cbee60
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffb6cc5e41
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0xd1
|
|
9
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb7395ec9
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x96
|
|
10
|
|
vmlinux |
0xffffffffb6cc5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x40
|
|
11
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffb7395df7
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x21
|
|
12
|
|
vmlinux |
0xffffffffb6cc5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:2:10001]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink binfmt_misc sunrpc ppdev pcspkr joydev virtio_balloon sg i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic virtio_net virtio_console net_failover pata_acpi virtio_blk failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 10001 Comm: kworker/0:2 Kdump: loaded Not tainted 3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996)/Standard PC (i440FX + PIIX, 1996), BIOS
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff8ae71e3e8000 ti: ffff8ae838480000 task.ti: ffff8ae838480000
RIP: 0010:[<ffffffffb6fa222b>] [<ffffffffb6fa222b>] iowrite16+0x3b/0x40
RSP: 0018:ffff8ae838483d90 EFLAGS: 00000296
RAX: ffffffffc01b4b60 RBX: 0000000000000000 RCX: ffff8ae7b92fd940
RDX: ffffa1b5807a8008 RSI: ffffa1b5807a8008 RDI: 0000000000000002
RBP: ffff8ae838483d98 R08: 00000000000000d0 R09: ffff8ae838483dd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff8ae83fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efbb40db000 CR3: 000000004792c000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc01b4b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc01a8eb0>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc043a0d2>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffb6cbde6f>] process_one_work+0x17f/0x440
[<ffffffffb6cbef86>] worker_thread+0x126/0x3c0
[<ffffffffb6cbee60>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb6cc5e41>] kthread+0xd1/0xe0
[<ffffffffb7395ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb6cc5d70>] ? insert_kthread_work+0x40/0x40
[<ffffffffb7395df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb6cc5d70>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 9e 12 6b b7 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00
Complete report #17937
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffffaf190bac
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x1c
|
|
2
|
|
virtio_pci |
0xffffffffc007eb76
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc0067720
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc02bd022
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffaeeb9d8f
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffaeebae26
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffaeebad00
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffaeec1c71
Build id: 3.10.0-957.10.1.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffaeec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffaf575c37
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffffaeec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:0:26809]
Modules linked in: binfmt_misc ip6table_filter ip6_tables iptable_filter ppdev joydev pcspkr sg virtio_balloon parport_pc parport i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_blk serio_raw cirrus drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix ttm libata e1000 drm virtio_pci virtio_ring drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 26809 Comm: kworker/0:0 Kdump: loaded Not tainted 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9f743a99b0c0 ti: ffff9f73c870c000 task.ti: ffff9f73c870c000
RIP: 0010:[<ffffffffaf190bac>] [<ffffffffaf190bac>] iowrite16+0x1c/0x40
RSP: 0000:ffff9f73c870fd90 EFLAGS: 00000202
RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff9f73f6045000
RDX: 000000000000c0b0 RSI: 000000000001c0b0 RDI: 0000000000000002
RBP: ffff9f73c870fd98 R08: 00000000000000d0 R09: 0000000000000000
R10: ffff9f73c870fdd0 R11: 0000000000000006 R12: 00000000000001bf
R13: 0000000000000204 R14: 00000000001f3800 R15: 000000000723fb0d
FS: 0000000000000000(0000) GS:ffff9f743fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f752008c000 CR3: 0000000026944000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc007eb76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc0067720>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc02bd022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffaeeb9d8f>] process_one_work+0x17f/0x440
[<ffffffffaeebae26>] worker_thread+0x126/0x3c0
[<ffffffffaeebad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffaeec1c71>] kthread+0xd1/0xe0
[<ffffffffaeec1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf575c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaeec1ba0>] ? insert_kthread_work+0x40/0x40
Code: ff ff 5d c3 0f 1f 00 40 88 3e c3 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 00 01 00 76 0b 0f b7 d6 89 f8 66 ef <c3> 0f 1f 00 55 48 c7 c6 a9 42 8a af 48 89 d7 48 89 e5 e8 5d fe
Complete report #6959
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffff9cba07dc
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x1c
|
|
2
|
|
virtio_pci |
0xffffffffc02b7b76
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc02a0cd0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc0530022
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffff9c8be6bf
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffff9c8bf7d6
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffff9c8bf6b0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffff9c8c6691
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffff9c8c65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff9cf92d37
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffff9c8c65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:2:31037]
Modules linked in: nls_utf8 isofs loop sunrpc ppdev pcspkr joydev i2c_piix4 sg parport_pc parport virtio_balloon ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover failover virtio_console virtio_blk cirrus drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops libata ttm drm crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 31037 Comm: kworker/0:2 Kdump: loaded Not tainted 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996)/Standard PC (i440FX + PIIX, 1996), BIOS
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffffa04af94520e0 ti: ffffa04adee7c000 task.ti: ffffa04adee7c000
RIP: 0010:[<ffffffff9cba07dc>] [<ffffffff9cba07dc>] iowrite16+0x1c/0x40
RSP: 0018:ffffa04adee7fd90 EFLAGS: 00000206
RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffffa049f5b9d000
RDX: 000000000000c090 RSI: 000000000001c090 RDI: 0000000000000002
RBP: ffffa04adee7fd98 R08: 00000000000000d0 R09: ffffa04adee7fdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffffa04affc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6e4a674000 CR3: 000000009af90000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc02b7b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc02a0cd0>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc0530022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffff9c8be6bf>] process_one_work+0x17f/0x440
[<ffffffff9c8bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff9c8bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9c8c6691>] kthread+0xd1/0xe0
[<ffffffff9c8c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9cf92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9c8c65c0>] ? insert_kthread_work+0x40/0x40
Code: ff ff 5d c3 0f 1f 00 40 88 3e c3 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 00 01 00 76 0b 0f b7 d6 89 f8 66 ef <c3> 0f 1f 00 55 48 c7 c6 f6 f9 2a 9d 48 89 d7 48 89 e5 e8 5d fe
Complete report #450
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
iowrite16
|
vmlinux |
0xffffffffa41a07fb
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x3b
|
|
2
|
|
virtio_pci |
0xffffffffc02efb76
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x16
|
|
3
|
virtqueue_kick
|
virtio_ring |
0xffffffffc02e3cd0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x30
|
|
4
|
update_balloon_stats_func
|
virtio_balloon |
0xffffffffc0593022
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x82
|
|
5
|
process_one_work
|
vmlinux |
0xffffffffa3ebe6bf
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x17f
|
|
6
|
worker_thread
|
vmlinux |
0xffffffffa3ebf7d6
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x126
|
|
7
|
|
vmlinux |
0xffffffffa3ebf6b0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x2a0
|
|
8
|
kthread
|
vmlinux |
0xffffffffa3ec6691
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0xd1
|
|
9
|
|
vmlinux |
0xffffffffa3ec65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x40
|
|
10
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa4592d37
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x21
|
|
11
|
|
vmlinux |
0xffffffffa3ec65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [kworker/0:0:2556]
Modules linked in: nf_conntrack_netlink nfnetlink binfmt_misc vboxnetadp(OE) vboxnetflt(OE) xt_conntrack iptable_filter xt_REDIRECT nf_nat_redirect vboxdrv(OE) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sg ppdev joydev pcspkr parport_pc virtio_balloon parport i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net virtio_console net_failover virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 2556 Comm: kworker/0:0 Kdump: loaded Tainted: G OEL ------------ 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff8f9ada76d230 ti: ffff8f9ad7994000 task.ti: ffff8f9ad7994000
RIP: 0010:[<ffffffffa41a07fb>] [<ffffffffa41a07fb>] iowrite16+0x3b/0x40
RSP: 0018:ffff8f9ad7997d90 EFLAGS: 00000296
RAX: ffffffffc02efb60 RBX: 0000000000000000 RCX: ffff8f9d34108940
RDX: ffff9def81b62008 RSI: ffff9def81b62008 RDI: 0000000000000002
RBP: ffff8f9ad7997d98 R08: 00000000000000d0 R09: 0000000000000000
R10: ffff8f9ad7997dd0 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff8f9d3fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1181fdc6a8 CR3: 000000022b50c000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc02efb76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc02e3cd0>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc0593022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffa3ebe6bf>] process_one_work+0x17f/0x440
[<ffffffffa3ebf7d6>] worker_thread+0x126/0x3c0
[<ffffffffa3ebf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa3ec6691>] kthread+0xd1/0xe0
[<ffffffffa3ec65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa4592d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa3ec65c0>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 f6 f9 8a a4 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00