NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [3_scheduler:9212]
Modules linked in: iptable_filter ip_tables tcp_lp fuse tun devlink binfmt_misc ip_set nfnetlink bridge stp llc sunrpc ext4 mbcache jbd2 ppdev sg joydev parport_pc virtio_balloon parport i2c_piix4 pcspkr xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk virtio_scsi cirrus floppy drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 8 PID: 9212 Comm: 3_scheduler Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org 04/01/2014
task: ffff97a13a6a5140 ti: ffff97a31725c000 task.ti: ffff97a31725c000
RIP: 0010:[<ffffffff9d311ffd>]  [<ffffffff9d311ffd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff97a31725fd18  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffd4000c27ab70
RBP: ffff97a31725fd18 R08: 0000000000000000 R09: ffffd40006d98040
R10: 0000000000004282 R11: 00003ffffffff000 R12: 0000000000000001
R13: ffff97a13fd9e000 R14: ffffd40006d98000 R15: ffff97a39e631900
FS:  00007f8609639700(0000) GS:ffff97a3a3a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8570433000 CR3: 00000003576a4000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9d95bf4b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9d96a400>] _raw_spin_lock+0x20/0x30
[<ffffffff9d3e9817>] handle_pte_fault+0x9d7/0xd10
[<ffffffff9d2c57e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff9d3ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9d96f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9d96f9c6>] trace_do_page_fault+0x56/0x150
[<ffffffff9d96ef42>] do_async_page_fault+0x22/0xf0
[<ffffffff9d96b788>] async_page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kube-scheduler:2659]
Modules linked in: ip6t_MASQUERADE nf_nat_masquerade_ipv6 xt_comment xt_mark nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack isofs ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sb_edac iosf_mbi kvm_intel i2c_piix4 pcspkr virtio_balloon kvm joydev irqbypass binfmt_misc ip_tables xfs libcrc32c ata_generic pata_acpi virtio_net virtio_scsi net_failover virtio_blk failover qxl
drm_kms_helper syscopyarea crct10dif_pclmul crct10dif_common sysfillrect crc32_pclmul crc32c_intel sysimgblt fb_sys_fops ttm ghash_clmulni_intel drm ata_piix serio_raw aesni_intel libata lrw gf128mul glue_helper ablk_helper cryptd virtio_pci virtio_ring drm_panel_orientation_quirks virtio sunrpc dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 2659 Comm: kube-scheduler Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: DigitalOcean Droplet/Droplet, BIOS 20171212 12/12/2017
task: ffff9f84a7951080 ti: ffff9f84a09d0000 task.ti: ffff9f84a09d0000
RIP: 0010:[<ffffffff8b317acd>]  [<ffffffff8b317acd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff9f84a09d3d90  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff9f84a09d3d68 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffecc890a7f470
RBP: ffff9f84a09d3d90 R08: 0000000000000000 R09: ffffecc890070e40
R10: ffff9f84a09d3df0 R11: ffff9f84a39a0e58 R12: ffff9f81f8238700
R13: 0000000000000001 R14: ffff9f84bfd71000 R15: ffffecc890070e00
FS:  000000c000345490(0000) GS:ffff9f84bfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff8b3285268 CR3: 000000003255c000 CR4: 00000000001606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8b980d63>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff8b98ed00>] _raw_spin_lock+0x20/0x30
[<ffffffff8b3f6903>] handle_mm_fault+0xd83/0xfb0
[<ffffffff8b994653>] __do_page_fault+0x213/0x500
[<ffffffff8b994a26>] trace_do_page_fault+0x56/0x150
[<ffffffff8b993fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8b9907a8>] async_page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [$_NIOREACTOR-7-:125589]
Modules linked in: binfmt_misc ext4 mbcache jbd2 ppdev sg joydev virtio_balloon parport_pc parport i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm floppy drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 15 PID: 125589 Comm: $_NIOREACTOR-7- Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9da9d67ad140 ti: ffff9da9d0b8c000 task.ti: ffff9da9d0b8c000
RIP: 0010:[<ffffffffa5b11ffd>]  [<ffffffffa5b11ffd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff9da9d0b8fd18  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff9da6f8a82200 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffe999204eedf0
RBP: ffff9da9d0b8fd18 R08: ffff9da9d3bb7c18 R09: 0000000000000001
R10: 0000000000000001 R11: 00003ffffffff000 R12: ffff9da9d67ad140
R13: ffff9da9d67ad7b8 R14: ffff9da9d67ad140 R15: ffff9da9dd19ab80
FS:  00007f6a3b3d7700(0000) GS:ffff9da9dd1c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b3fd830c8 CR3: 00000002abe62000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa615bf4b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa616a400>] _raw_spin_lock+0x20/0x30
[<ffffffffa5be8fa0>] handle_pte_fault+0x160/0xd10
[<ffffffffa5bebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa616f5e3>] __do_page_fault+0x203/0x500
[<ffffffffa616f9c6>] trace_do_page_fault+0x56/0x150
[<ffffffffa616ef42>] do_async_page_fault+0x22/0xf0
[<ffffffffa616b788>] async_page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f