Info

Permalink
Function
iowrite16
First occurrence
2020-09-23
Last occurrence
2026-02-18
State
NEW
Components
kernel
Quality
-26

Statistics

Operating system Count
CentOS 7 339
Architecture Count
x86_64 339
Related packages Count
kernel 339
 0:3.10.0-862.11.6.el7 51
 0:3.10.0-957.27.2.el7 41
 0:3.10.0-1062.1.1.el7 38
 0:3.10.0-1160.81.1.el7 38
 0:3.10.0-957.el7 38
 0:3.10.0-1160.88.1.el7 22
 0:3.10.0-1160.25.1.el7 17
 0:3.10.0-1127.13.1.el7 14
 0:3.10.0-1160.95.1.el7 13
 0:3.10.0-1160.119.1.el7 10
 0:3.10.0-1160.49.1.el7 7
 0:3.10.0-1160.53.1.el7 7
 0:3.10.0-1160.62.1.el7 6
 0:3.10.0-957.10.1.el7 6
 0:3.10.0-1062.4.3.el7 4
 0:3.10.0-1160.45.1.el7 3
 0:3.10.0-1160.66.1.el7 3
 0:3.10.0-1160.92.1.el7 2
 0:3.10.0-1062.el7 2
 0:3.10.0-1127.el7 2
 0:3.10.0-1160.114.2.el7 2
 0:3.10.0-1160.15.2.el7 2
 0:3.10.0-1160.el7 2
 0:3.10.0-957.1.3.el7 2
 0:3.10.0-1062.12.1.el7 1
 0:3.10.0-1062.18.1.el7 1
 0:3.10.0-1160.108.1.el7 1
 0:3.10.0-1160.11.1.el7 1
 0:3.10.0-1160.83.1.el7 1
 0:3.10.0-1160.99.1.el7 1
 0:3.10.0-1160.36.2.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1390795
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffffbb5b8c6b
Build id: 3.10.0-1160.119.1.el7.x86_64
0x4b
2
? vp_notify
virtio_pci 0xffffffffc01b7c26
Build id: 3.10.0-1160.119.1.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc01a7f90
Build id: 3.10.0-1160.119.1.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc035b0f2
Build id: 3.10.0-1160.119.1.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffffbb2c32ef
Build id: 3.10.0-1160.119.1.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffffbb2c4436
Build id: 3.10.0-1160.119.1.el7.x86_64
0x126
7
? manage_workers.isra.26
vmlinux 0xffffffffbb2c4310
Build id: 3.10.0-1160.119.1.el7.x86_64
0x2b0
8
kthread
vmlinux 0xffffffffbb2cb621
Build id: 3.10.0-1160.119.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffbb2cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffbb9c51f7
Build id: 3.10.0-1160.119.1.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffbb2cb550
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 60s! [kworker/0:0:12867]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_18504(E) iptable_filter iosf_mbi crc32_pclmul ghash_clmulni_intel iTCO_wdt iTCO_vendor_support bochs_drm ttm drm_kms_helper aesni_intel syscopyarea lrw sysfillrect gf128mul glue_helper sysimgblt ablk_helper cryptd fb_sys_fops joydev pcspkr lpc_ich i2c_i801 drm virtio_rng virtio_balloon drm_panel_orientation_quirks ip_tables xfs libcrc32c virtio_net net_failover failover virtio_blk virtio_console ahci libahci crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 12867 Comm: kworker/0:0 Kdump: loaded Tainted: P            E  ------------   3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.16.3-4.el9 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9117faf66300 ti: ffff9118f4fe8000 task.ti: ffff9118f4fe8000
RIP: 0010:[<ffffffffbb5b8c6b>]  [<ffffffffbb5b8c6b>] iowrite16+0x4b/0x50
RSP: 0018:ffff9118f4febd90  EFLAGS: 00000296
RAX: ffffffffc01b7c10 RBX: 0000000000000004 RCX: ffff9118f66f6940
RDX: ffffa52e00ea0008 RSI: ffffa52e00ea0008 RDI: 0000000000000002
RBP: ffff9118f4febd98 R08: 00000000000000d0 R09: ffff9118f4febdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff9118ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000015f1590 CR3: 00000001837ea000 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
Call Trace:
[<ffffffffc01b7c26>] ? vp_notify+0x16/0x30 [virtio_pci]
[<ffffffffc01a7f90>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc035b0f2>] update_balloon_stats_func+0x82/0xa7 [virtio_balloon]
[<ffffffffbb2c32ef>] process_one_work+0x17f/0x440
[<ffffffffbb2c4436>] worker_thread+0x126/0x3c0
[<ffffffffbb2c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffffbb2cb621>] kthread+0xd1/0xe0
[<ffffffffbb2cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffbb9c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbb2cb550>] ? insert_kthread_work+0x40/0x40
Code: cc 0f 1f 80 00 00 00 00 55 48 c7 c6 06 28 cb bb 48 89 d7 48 89 e5 e8 35 fe ff ff 5d c3 cc cc cc cc 0f 1f 80 00 00 00 00 66 89 3e <c3> cc cc cc cc 48 81 fe ff ff 03 00 48 89 f2 77 34 48 81 fe 00 
Complete report #541544
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffffb0768abb
Build id: 3.10.0-862.11.6.el7.x86_64
0x3b
2
? vp_notify
virtio_pci 0xffffffffc02f3a66
Build id: 3.10.0-862.11.6.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc02d1710
Build id: 3.10.0-862.11.6.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc057f022
Build id: 3.10.0-862.11.6.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffffb04b613f
Build id: 3.10.0-862.11.6.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffffb04b71d6
Build id: 3.10.0-862.11.6.el7.x86_64
0x126
7
? manage_workers.isra.24
vmlinux 0xffffffffb04b70b0
Build id: 3.10.0-862.11.6.el7.x86_64
0x2a0
8
kthread
vmlinux 0xffffffffb04bdf21
Build id: 3.10.0-862.11.6.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffb04bde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffb0b255f7
Build id: 3.10.0-862.11.6.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffb04bde50
Build id: 3.10.0-862.11.6.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [kworker/0:2:19759]
Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter bridge stp llc binfmt_misc iptable_nat nf_nat_ipv4 nf_nat ip6table_mangle ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables iptable_mangle ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter overlay(T) dm_mirror dm_region_hash ext4 dm_log dm_mod mbcache jbd2 sg i2c_piix4 ppdev shpchp pcspkr joydev virtio_balloon parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_net virtio_scsi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci i2c_core serio_raw virtio_ring virtio
floppy
CPU: 0 PID: 19759 Comm: kworker/0:2 Kdump: loaded Tainted: G               ------------ T 3.10.0-862.11.6.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff96508f90af70 ti: ffff965001148000 task.ti: ffff965001148000
RIP: 0010:[<ffffffffb0768abb>]  [<ffffffffb0768abb>] iowrite16+0x3b/0x40
RSP: 0018:ffff96500114bd90  EFLAGS: 00000296
RAX: ffffffffc02f3a50 RBX: 0000000000000000 RCX: ffff9655053c2940
RDX: ffffb8cf83592008 RSI: ffffb8cf83592008 RDI: 0000000000000002
RBP: ffff96500114bd98 R08: 00000000000000d0 R09: 0000000000000000
R10: ffff96500114bdd0 R11: 0000000000000006 R12: 00000000000246d5
R13: 000000000002a0c3 R14: 000000000791e645 R15: 0000001de22cfcbd
FS:  0000000000000000(0000) GS:ffff96550d800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4746ffccb0 CR3: 00000004f8b22000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc02f3a66>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc02d1710>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc057f022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffb04b613f>] process_one_work+0x17f/0x440
[<ffffffffb04b71d6>] worker_thread+0x126/0x3c0
[<ffffffffb04b70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffb04bdf21>] kthread+0xd1/0xe0
[<ffffffffb04bde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffb0b255f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb04bde50>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 62 7d e9 b0 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 
Complete report #319166
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffff86b904dc
Build id: 3.10.0-957.el7.x86_64
0x1c
2
? vp_notify
virtio_pci 0xffffffffc0369b76
Build id: 3.10.0-957.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc0352720
Build id: 3.10.0-957.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc0597022
Build id: 3.10.0-957.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffff868b9d4f
Build id: 3.10.0-957.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffff868bade6
Build id: 3.10.0-957.el7.x86_64
0x126
7
? manage_workers.isra.25
vmlinux 0xffffffff868bacc0
Build id: 3.10.0-957.el7.x86_64
0x2a0
8
kthread
vmlinux 0xffffffff868c1c31
Build id: 3.10.0-957.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffff868c1b60
Build id: 3.10.0-957.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffff86f74c37
Build id: 3.10.0-957.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffff868c1b60
Build id: 3.10.0-957.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 105s! [kworker/3:2:28071]
Modules linked in: kbase(OE) syshook_linux(OE) binfmt_misc udp_diag tcp_diag inet_diag fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel virtio_balloon aesni_intel parport_pc lrw parport gf128mul glue_helper i2c_piix4 ablk_helper cryptd joydev pcspkr ip_tables ext4 mbcache jbd2 virtio_console
virtio_blk virtio_net ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kbase]
CPU: 3 PID: 28071 Comm: kworker/3:2 Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9b50245de180 ti: ffff9b4fc3b8c000 task.ti: ffff9b4fc3b8c000
RIP: 0010:[<ffffffff86b904dc>]  [<ffffffff86b904dc>] iowrite16+0x1c/0x40
RSP: 0018:ffff9b4fc3b8fd90  EFLAGS: 00000206
RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff9b5fb371f000
RDX: 000000000000c0f0 RSI: 000000000001c0f0 RDI: 0000000000000002
RBP: ffff9b4fc3b8fd98 R08: 00000000000000d0 R09: ffff9b4fc3b8fdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000003
R13: 0000000000000032 R14: 000000000004c036 R15: 00000000012668a5
FS:  0000000000000000(0000) GS:ffff9b5fbf2c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f25a40caf89 CR3: 000000103fe5c000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0369b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc0352720>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc0597022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffff868b9d4f>] process_one_work+0x17f/0x440
[<ffffffff868bade6>] worker_thread+0x126/0x3c0
[<ffffffff868bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff868c1c31>] kthread+0xd1/0xe0
[<ffffffff868c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff86f74c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff868c1b60>] ? insert_kthread_work+0x40/0x40
Code: ff ff 5d c3 0f 1f 00 40 88 3e c3 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 00 01 00 76 0b 0f b7 d6 89 f8 66 ef <c3> 0f 1f 00 55 48 c7 c6 51 40 2a 87 48 89 d7 48 89 e5 e8 5d fe 
Complete report #255227
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffffbdb9242b
Build id: 3.10.0-957.27.2.el7.x86_64
0x3b
2
? vp_notify
virtio_pci 0xffffffffc0096b76
Build id: 3.10.0-957.27.2.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc008a720
Build id: 3.10.0-957.27.2.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc042f022
Build id: 3.10.0-957.27.2.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffffbd8baf9f
Build id: 3.10.0-957.27.2.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffffbd8bc036
Build id: 3.10.0-957.27.2.el7.x86_64
0x126
7
? manage_workers.isra.25
vmlinux 0xffffffffbd8bbf10
Build id: 3.10.0-957.27.2.el7.x86_64
0x2a0
8
kthread
vmlinux 0xffffffffbd8c2e81
Build id: 3.10.0-957.27.2.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffbd8c2db0
Build id: 3.10.0-957.27.2.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffbdf76c37
Build id: 3.10.0-957.27.2.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffbd8c2db0
Build id: 3.10.0-957.27.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:2:19140]
Modules linked in: edr(POE) edr_gpl(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul sg glue_helper ablk_helper cryptd joydev pcspkr virtio_rng virtio_balloon parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic virtio_console virtio_net virtio_scsi sr_mod cdrom ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common
crc32c_intel floppy serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: edr_gpl]
CPU: 0 PID: 19140 Comm: kworker/0:2 Kdump: loaded Tainted: P           OE  ------------   3.10.0-957.27.2.el7.x86_64 #1
Hardware name: oVirt oVirt Node, BIOS 1.11.0-2.el7 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9d2a8330a080 ti: ffff9d2aa3b58000 task.ti: ffff9d2aa3b58000
RIP: 0010:[<ffffffffbdb9242b>]  [<ffffffffbdb9242b>] iowrite16+0x3b/0x40
RSP: 0018:ffff9d2aa3b5bd90  EFLAGS: 00000296
RAX: ffffffffc0096b60 RBX: 0000000000000000 RCX: ffff9d2ab4b59940
RDX: ffffaa1b00440008 RSI: ffffaa1b00440008 RDI: 0000000000000002
RBP: ffff9d2aa3b5bd98 R08: 00000000000000d0 R09: ffff9d2aa3b5bdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 000000000000038a
R13: 0000000000000ed5 R14: 00000000003ebc78 R15: 0000000004d90a38
FS:  0000000000000000(0000) GS:ffff9d2afd200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0f12ade000 CR3: 000000002334a000 CR4: 00000000003606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0096b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc008a720>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc042f022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffbd8baf9f>] process_one_work+0x17f/0x440
[<ffffffffbd8bc036>] worker_thread+0x126/0x3c0
[<ffffffffbd8bbf10>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffbd8c2e81>] kthread+0xd1/0xe0
[<ffffffffbd8c2db0>] ? insert_kthread_work+0x40/0x40
[<ffffffffbdf76c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffbd8c2db0>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 2e 4d 2a be 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 
Complete report #221999
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffffb6fa222b
Build id: 3.10.0-1160.36.2.el7.x86_64
0x3b
2
? vp_notify
virtio_pci 0xffffffffc01b4b76
Build id: 3.10.0-1160.36.2.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc01a8eb0
Build id: 3.10.0-1160.36.2.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc043a0d2
Build id: 3.10.0-1160.36.2.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffffb6cbde6f
Build id: 3.10.0-1160.36.2.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffffb6cbef86
Build id: 3.10.0-1160.36.2.el7.x86_64
0x126
7
? manage_workers.isra.26
vmlinux 0xffffffffb6cbee60
Build id: 3.10.0-1160.36.2.el7.x86_64
0x2a0
8
kthread
vmlinux 0xffffffffb6cc5e41
Build id: 3.10.0-1160.36.2.el7.x86_64
0xd1
9
? system_call_after_swapgs
vmlinux 0xffffffffb7395ec9
Build id: 3.10.0-1160.36.2.el7.x86_64
0x96
10
? insert_kthread_work
vmlinux 0xffffffffb6cc5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
11
ret_from_fork_nospec_begin
vmlinux 0xffffffffb7395df7
Build id: 3.10.0-1160.36.2.el7.x86_64
0x21
12
? insert_kthread_work
vmlinux 0xffffffffb6cc5d70
Build id: 3.10.0-1160.36.2.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:2:10001]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink binfmt_misc sunrpc ppdev pcspkr joydev virtio_balloon sg i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic virtio_net virtio_console net_failover pata_acpi virtio_blk failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 10001 Comm: kworker/0:2 Kdump: loaded Not tainted 3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996)/Standard PC (i440FX + PIIX, 1996), BIOS  
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff8ae71e3e8000 ti: ffff8ae838480000 task.ti: ffff8ae838480000
RIP: 0010:[<ffffffffb6fa222b>]  [<ffffffffb6fa222b>] iowrite16+0x3b/0x40
RSP: 0018:ffff8ae838483d90  EFLAGS: 00000296
RAX: ffffffffc01b4b60 RBX: 0000000000000000 RCX: ffff8ae7b92fd940
RDX: ffffa1b5807a8008 RSI: ffffa1b5807a8008 RDI: 0000000000000002
RBP: ffff8ae838483d98 R08: 00000000000000d0 R09: ffff8ae838483dd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8ae83fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efbb40db000 CR3: 000000004792c000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc01b4b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc01a8eb0>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc043a0d2>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffb6cbde6f>] process_one_work+0x17f/0x440
[<ffffffffb6cbef86>] worker_thread+0x126/0x3c0
[<ffffffffb6cbee60>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb6cc5e41>] kthread+0xd1/0xe0
[<ffffffffb7395ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb6cc5d70>] ? insert_kthread_work+0x40/0x40
[<ffffffffb7395df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb6cc5d70>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 9e 12 6b b7 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 
Complete report #17937
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffffaf190bac
Build id: 3.10.0-957.10.1.el7.x86_64
0x1c
2
? vp_notify
virtio_pci 0xffffffffc007eb76
Build id: 3.10.0-957.10.1.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc0067720
Build id: 3.10.0-957.10.1.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc02bd022
Build id: 3.10.0-957.10.1.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffffaeeb9d8f
Build id: 3.10.0-957.10.1.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffffaeebae26
Build id: 3.10.0-957.10.1.el7.x86_64
0x126
7
? manage_workers.isra.25
vmlinux 0xffffffffaeebad00
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a0
8
kthread
vmlinux 0xffffffffaeec1c71
Build id: 3.10.0-957.10.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffaeec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffaf575c37
Build id: 3.10.0-957.10.1.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffaeec1ba0
Build id: 3.10.0-957.10.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:0:26809]
Modules linked in: binfmt_misc ip6table_filter ip6_tables iptable_filter ppdev joydev pcspkr sg virtio_balloon parport_pc parport i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_blk serio_raw cirrus drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix ttm libata e1000 drm virtio_pci virtio_ring drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 26809 Comm: kworker/0:0 Kdump: loaded Not tainted 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff9f743a99b0c0 ti: ffff9f73c870c000 task.ti: ffff9f73c870c000
RIP: 0010:[<ffffffffaf190bac>]  [<ffffffffaf190bac>] iowrite16+0x1c/0x40
RSP: 0000:ffff9f73c870fd90  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff9f73f6045000
RDX: 000000000000c0b0 RSI: 000000000001c0b0 RDI: 0000000000000002
RBP: ffff9f73c870fd98 R08: 00000000000000d0 R09: 0000000000000000
R10: ffff9f73c870fdd0 R11: 0000000000000006 R12: 00000000000001bf
R13: 0000000000000204 R14: 00000000001f3800 R15: 000000000723fb0d
FS:  0000000000000000(0000) GS:ffff9f743fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f752008c000 CR3: 0000000026944000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc007eb76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc0067720>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc02bd022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffaeeb9d8f>] process_one_work+0x17f/0x440
[<ffffffffaeebae26>] worker_thread+0x126/0x3c0
[<ffffffffaeebad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffaeec1c71>] kthread+0xd1/0xe0
[<ffffffffaeec1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf575c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaeec1ba0>] ? insert_kthread_work+0x40/0x40
Code: ff ff 5d c3 0f 1f 00 40 88 3e c3 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 00 01 00 76 0b 0f b7 d6 89 f8 66 ef <c3> 0f 1f 00 55 48 c7 c6 a9 42 8a af 48 89 d7 48 89 e5 e8 5d fe 
Complete report #6959
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffff9cba07dc
Build id: 3.10.0-1127.13.1.el7.x86_64
0x1c
2
? vp_notify
virtio_pci 0xffffffffc02b7b76
Build id: 3.10.0-1127.13.1.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc02a0cd0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc0530022
Build id: 3.10.0-1127.13.1.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffff9c8be6bf
Build id: 3.10.0-1127.13.1.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffff9c8bf7d6
Build id: 3.10.0-1127.13.1.el7.x86_64
0x126
7
? manage_workers.isra.26
vmlinux 0xffffffff9c8bf6b0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x2a0
8
kthread
vmlinux 0xffffffff9c8c6691
Build id: 3.10.0-1127.13.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffff9c8c65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffff9cf92d37
Build id: 3.10.0-1127.13.1.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffff9c8c65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:2:31037]
Modules linked in: nls_utf8 isofs loop sunrpc ppdev pcspkr joydev i2c_piix4 sg parport_pc parport virtio_balloon ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover failover virtio_console virtio_blk cirrus drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops libata ttm drm crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 31037 Comm: kworker/0:2 Kdump: loaded Not tainted 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996)/Standard PC (i440FX + PIIX, 1996), BIOS  
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffffa04af94520e0 ti: ffffa04adee7c000 task.ti: ffffa04adee7c000
RIP: 0010:[<ffffffff9cba07dc>]  [<ffffffff9cba07dc>] iowrite16+0x1c/0x40
RSP: 0018:ffffa04adee7fd90  EFLAGS: 00000206
RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffffa049f5b9d000
RDX: 000000000000c090 RSI: 000000000001c090 RDI: 0000000000000002
RBP: ffffa04adee7fd98 R08: 00000000000000d0 R09: ffffa04adee7fdd0
R10: 0000000000000005 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffffa04affc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6e4a674000 CR3: 000000009af90000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc02b7b76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc02a0cd0>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc0530022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffff9c8be6bf>] process_one_work+0x17f/0x440
[<ffffffff9c8bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff9c8bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9c8c6691>] kthread+0xd1/0xe0
[<ffffffff9c8c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9cf92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9c8c65c0>] ? insert_kthread_work+0x40/0x40
Code: ff ff 5d c3 0f 1f 00 40 88 3e c3 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00 00 01 00 76 0b 0f b7 d6 89 f8 66 ef <c3> 0f 1f 00 55 48 c7 c6 f6 f9 2a 9d 48 89 d7 48 89 e5 e8 5d fe 
Complete report #450
Frame # Function Binary Source or offset Line
1
iowrite16
vmlinux 0xffffffffa41a07fb
Build id: 3.10.0-1127.13.1.el7.x86_64
0x3b
2
? vp_notify
virtio_pci 0xffffffffc02efb76
Build id: 3.10.0-1127.13.1.el7.x86_64
0x16
3
virtqueue_kick
virtio_ring 0xffffffffc02e3cd0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x30
4
update_balloon_stats_func
virtio_balloon 0xffffffffc0593022
Build id: 3.10.0-1127.13.1.el7.x86_64
0x82
5
process_one_work
vmlinux 0xffffffffa3ebe6bf
Build id: 3.10.0-1127.13.1.el7.x86_64
0x17f
6
worker_thread
vmlinux 0xffffffffa3ebf7d6
Build id: 3.10.0-1127.13.1.el7.x86_64
0x126
7
? manage_workers.isra.26
vmlinux 0xffffffffa3ebf6b0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x2a0
8
kthread
vmlinux 0xffffffffa3ec6691
Build id: 3.10.0-1127.13.1.el7.x86_64
0xd1
9
? insert_kthread_work
vmlinux 0xffffffffa3ec65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x40
10
ret_from_fork_nospec_begin
vmlinux 0xffffffffa4592d37
Build id: 3.10.0-1127.13.1.el7.x86_64
0x21
11
? insert_kthread_work
vmlinux 0xffffffffa3ec65c0
Build id: 3.10.0-1127.13.1.el7.x86_64
0x40
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [kworker/0:0:2556]
Modules linked in: nf_conntrack_netlink nfnetlink binfmt_misc vboxnetadp(OE) vboxnetflt(OE) xt_conntrack iptable_filter xt_REDIRECT nf_nat_redirect vboxdrv(OE) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sg ppdev joydev pcspkr parport_pc virtio_balloon parport i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net virtio_console net_failover virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 2556 Comm: kworker/0:0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org 04/01/2014
Workqueue: events_freezable update_balloon_stats_func [virtio_balloon]
task: ffff8f9ada76d230 ti: ffff8f9ad7994000 task.ti: ffff8f9ad7994000
RIP: 0010:[<ffffffffa41a07fb>]  [<ffffffffa41a07fb>] iowrite16+0x3b/0x40
RSP: 0018:ffff8f9ad7997d90  EFLAGS: 00000296
RAX: ffffffffc02efb60 RBX: 0000000000000000 RCX: ffff8f9d34108940
RDX: ffff9def81b62008 RSI: ffff9def81b62008 RDI: 0000000000000002
RBP: ffff8f9ad7997d98 R08: 00000000000000d0 R09: 0000000000000000
R10: ffff8f9ad7997dd0 R11: 0000000000000006 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8f9d3fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1181fdc6a8 CR3: 000000022b50c000 CR4: 00000000000006f0
Call Trace:
[<ffffffffc02efb76>] ? vp_notify+0x16/0x20 [virtio_pci]
[<ffffffffc02e3cd0>] virtqueue_kick+0x30/0x50 [virtio_ring]
[<ffffffffc0593022>] update_balloon_stats_func+0x82/0xa3 [virtio_balloon]
[<ffffffffa3ebe6bf>] process_one_work+0x17f/0x440
[<ffffffffa3ebf7d6>] worker_thread+0x126/0x3c0
[<ffffffffa3ebf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa3ec6691>] kthread+0xd1/0xe0
[<ffffffffa3ec65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa4592d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa3ec65c0>] ? insert_kthread_work+0x40/0x40
Code: 00 01 00 76 0b 0f b7 d6 89 f8 66 ef c3 0f 1f 00 55 48 c7 c6 f6 f9 8a a4 48 89 d7 48 89 e5 e8 5d fe ff ff 5d c3 0f 1f 00 66 89 3e <c3> 0f 1f 40 00 48 81 fe ff ff 03 00 48 89 f2 77 2c 48 81 fe 00