NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [java:19737]
INFO: rcu_sched self-detected stall on CPU { 8}  (t=204079 jiffies g=401407422 c=401407421 q=4011)
Task dump for CPU 8:
scsi_eh_0       R  running task        0   339      2 0x00000000
Call Trace:
<IRQ>  [<ffffffffad2e0168>] sched_show_task+0xa8/0x110
[<ffffffffad2e3ef9>] dump_cpu_task+0x39/0x70
[<ffffffffad360c10>] rcu_dump_cpu_stacks+0x90/0xd0
[<ffffffffad364422>] rcu_check_callbacks+0x442/0x730
[<ffffffffad317470>] ? tick_sched_do_timer+0x60/0x60
[<ffffffffad2b3e66>] update_process_times+0x46/0x80
[<ffffffffad3171a0>] tick_sched_handle+0x30/0x80
[<ffffffffad3174a9>] tick_sched_timer+0x39/0x80
[<ffffffffad2cf9ce>] __hrtimer_run_queues+0x10e/0x270
[<ffffffffad2cff5f>] hrtimer_interrupt+0xaf/0x1e0
[<ffffffffad25fa2b>] local_apic_timer_interrupt+0x3b/0x70
[<ffffffffad9ca053>] smp_apic_timer_interrupt+0x43/0x60
[<ffffffffad9c63f2>] apic_timer_interrupt+0x172/0x180
<EOI>  [<ffffffffad9ba385>] ? _raw_spin_unlock_irqrestore+0x15/0x20
[<ffffffffc045c5d0>] ata_bmdma_error_handler+0x50/0x130 [libata]
[<ffffffffc0458e82>] ata_scsi_port_error_handler+0x532/0x900 [libata]
[<ffffffffc04592f0>] ata_scsi_error+0xa0/0xe0 [libata]
[<ffffffffad708336>] scsi_error_handler+0xf6/0x8b0
[<ffffffffad708240>] ? scsi_eh_get_sense+0x250/0x250
[<ffffffffad2cb4d1>] kthread+0xd1/0xe0
[<ffffffffad2cb400>] ? insert_kthread_work+0x40/0x40
[<ffffffffad9c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffad2cb400>] ? insert_kthread_work+0x40/0x40

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 71s! [kworker/0:1:19173]
INFO: rcu_sched self-detected stall on CPU { 1}  (t=76177 jiffies g=368857909 c=368857908 q=433)
Task dump for CPU 1:
kworker/1:1     R  running task        0 18766      2 0x00000080
Workqueue: events e1000_watchdog [e1000]
Call Trace:
<IRQ>  [<ffffffff8b6da0c8>] sched_show_task+0xa8/0x110
[<ffffffff8b6ddeb9>] dump_cpu_task+0x39/0x70
[<ffffffff8b758220>] rcu_dump_cpu_stacks+0x90/0xd0
[<ffffffff8b75b8e2>] rcu_check_callbacks+0x442/0x730
[<ffffffff8b7100d0>] ? tick_sched_do_timer+0x50/0x50
[<ffffffff8b6aef66>] update_process_times+0x46/0x80
[<ffffffff8b70fe40>] tick_sched_handle+0x30/0x70
[<ffffffff8b710109>] tick_sched_timer+0x39/0x80
[<ffffffff8b6ca01e>] __hrtimer_run_queues+0x10e/0x270
[<ffffffff8b6ca57f>] hrtimer_interrupt+0xaf/0x1d0
[<ffffffff8b65cdcb>] local_apic_timer_interrupt+0x3b/0x60
[<ffffffff8bd98a83>] smp_apic_timer_interrupt+0x43/0x60
[<ffffffff8bd94fba>] apic_timer_interrupt+0x16a/0x170
<EOI>  [<ffffffff8b6ad65a>] ? __internal_add_timer+0xaa/0x130
[<ffffffff8bd89785>] ? _raw_spin_unlock_irqrestore+0x15/0x20
[<ffffffffc01b360c>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc01b36bd>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffff8b6bdc4f>] process_one_work+0x17f/0x440
[<ffffffff8b6bed66>] worker_thread+0x126/0x3c0
[<ffffffff8b6bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff8b6c5c21>] kthread+0xd1/0xe0
[<ffffffff8b6c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8bd93df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8b6c5b50>] ? insert_kthread_work+0x40/0x40

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [python:67311]
INFO: rcu_sched self-detected stall on CPU { 4} 
(t=265835 jiffies g=16836313 c=16836312 q=1373)
INFO: rcu_sched detected stalls on CPUs/tasks: { 4} (detected by 0, t=265835 jiffies, g=16836313, c=16836312, q=1373)
Task dump for CPU 4:
khugepaged      R  running task        0    68      2 0x00000008
Call Trace:
<IRQ>  [<ffffffffb50da468>] sched_show_task+0xa8/0x110
[<ffffffffb50de0d9>] dump_cpu_task+0x39/0x70
[<ffffffffb51587a0>] rcu_dump_cpu_stacks+0x90/0xd0
[<ffffffffb515be62>] rcu_check_callbacks+0x442/0x730
[<ffffffffb51107b0>] ? tick_sched_do_timer+0x50/0x50
[<ffffffffb50af2e6>] update_process_times+0x46/0x80
[<ffffffffb5110520>] tick_sched_handle+0x30/0x70
[<ffffffffb51107e9>] tick_sched_timer+0x39/0x80
[<ffffffffb50ca38e>] __hrtimer_run_queues+0x10e/0x270
[<ffffffffb50ca8ef>] hrtimer_interrupt+0xaf/0x1d0
[<ffffffffb505cf0b>] local_apic_timer_interrupt+0x3b/0x60
[<ffffffffb579ea23>] smp_apic_timer_interrupt+0x43/0x60
[<ffffffffb579afba>] apic_timer_interrupt+0x16a/0x170
<EOI>  [<ffffffffb5234d15>] ? khugepaged_scan_mm_slot+0x375/0xfd0
[<ffffffffb5234d09>] ? khugepaged_scan_mm_slot+0x369/0xfd0
[<ffffffffb5235aab>] khugepaged+0x13b/0x490
[<ffffffffb50c7080>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffb5235970>] ? khugepaged_scan_mm_slot+0xfd0/0xfd0
[<ffffffffb50c5f91>] kthread+0xd1/0xe0
[<ffffffffb50c5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb5799ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffb50c5ec0>] ? insert_kthread_work+0x40/0x40

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 153s! [sleep:90571]
<EOI> 
INFO: rcu_sched self-detected stall on CPU
{ 3} 
(t=178608 jiffies g=59906208 c=59906207 q=283)
Task dump for CPU 3:
INFO: rcu_sched detected stalls on CPUs/tasks: { 3} (detected by 2, t=178608 jiffies, g=59906208, c=59906207, q=283)
kworker/3:0     R  running task        0 71652      2 0x00000088
Workqueue: events_freezable vmballoon_work [vmw_balloon]
Call Trace:
<IRQ>  [<ffffffff90eda4f8>] sched_show_task+0xa8/0x110
[<ffffffff90ede169>] dump_cpu_task+0x39/0x70
[<ffffffff90f58820>] rcu_dump_cpu_stacks+0x90/0xd0
[<ffffffff90f5bee2>] rcu_check_callbacks+0x442/0x730
[<ffffffff90f10830>] ? tick_sched_do_timer+0x50/0x50
[<ffffffff90eaf2e6>] update_process_times+0x46/0x80
[<ffffffff90f105a0>] tick_sched_handle+0x30/0x70
[<ffffffff90f10869>] tick_sched_timer+0x39/0x80
[<ffffffff90eca38e>] __hrtimer_run_queues+0x10e/0x270
[<ffffffff90eca8ef>] hrtimer_interrupt+0xaf/0x1d0
[<ffffffff90e5ce2b>] local_apic_timer_interrupt+0x3b/0x60
[<ffffffff9159ea23>] smp_apic_timer_interrupt+0x43/0x60
[<ffffffff9159afba>] apic_timer_interrupt+0x16a/0x170
<EOI>  [<ffffffffc044dbed>] ? vmballoon_work+0xbd/0x6ff [vmw_balloon]
[<ffffffff90ebdfbf>] process_one_work+0x17f/0x440
[<ffffffff90ebf0d6>] worker_thread+0x126/0x3c0
[<ffffffff90ebefb0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff90ec5f91>] kthread+0xd1/0xe0
[<ffffffff90ec5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffff91599df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff90ec5ec0>] ? insert_kthread_work+0x40/0x40

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 110s! [ksoftirqd/4:29]
Modules linked in: binfmt_misc veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus ppdev iosf_mbi crc32_pclmul snd_seq ghash_clmulni_intel snd_seq_device aesni_intel vmw_balloon snd_pcm lrw
gf128mul glue_helper ablk_helper cryptd snd_timer snd sg joydev pcspkr soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ttm crct10dif_common crc32c_intel ata_piix drm serio_raw libata mptspi nfit scsi_transport_spi e1000 mptscsih libnvdimm mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 29 Comm: ksoftirqd/4 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8abc798f1070 ti: ffff8abc798fc000 task.ti: ffff8abc798fc000
RIP: 0010:[<ffffffffc0467c2e>]  [<ffffffffc0467c2e>] e1000_alloc_rx_buffers+0x3be/0x610 [e1000]
RSP: 0018:ffff8abc798ffbd8  EFLAGS: 00000286
RAX: ffff99cb81942818 RBX: ffffffff9d653c19 RCX: 00000000000005f2
INFO: rcu_sched self-detected stall on CPU
INFO: rcu_sched detected stalls on CPUs/tasks: {
{
0
0
4
} 
(t=117357 jiffies g=155094655 c=155094654 q=20131)
} (detected by 2, t=117357 jiffies, g=155094655, c=155094654, q=20131)
Task dump for CPU 0:
kworker/0:4     R  running task        0 127141      2 0x00000080
Workqueue: events e1000_watchdog [e1000]
Call Trace:
<IRQ>  [<ffffffff9d0dab18>] sched_show_task+0xa8/0x110
[<ffffffff9d0de919>] dump_cpu_task+0x39/0x70
[<ffffffff9d158b30>] rcu_dump_cpu_stacks+0x90/0xd0
[<ffffffff9d15c1f2>] rcu_check_callbacks+0x442/0x730
[<ffffffff9d110b00>] ? tick_sched_do_timer+0x50/0x50
[<ffffffff9d0af9d6>] update_process_times+0x46/0x80
[<ffffffff9d110870>] tick_sched_handle+0x30/0x70
[<ffffffff9d110b39>] tick_sched_timer+0x39/0x80
[<ffffffff9d0caa8e>] __hrtimer_run_queues+0x10e/0x270
[<ffffffff9d0cafef>] hrtimer_interrupt+0xaf/0x1d0
[<ffffffff9d05ccfb>] local_apic_timer_interrupt+0x3b/0x60
[<ffffffff9d7979c3>] smp_apic_timer_interrupt+0x43/0x60
[<ffffffff9d793efa>] apic_timer_interrupt+0x16a/0x170
<EOI>  [<ffffffff9d7883c5>] ? _raw_spin_unlock_irqrestore+0x15/0x20
[<ffffffffc046b60c>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc046b6bd>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffff9d0be6bf>] process_one_work+0x17f/0x440
[<ffffffff9d0bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff9d0bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9d0c6691>] kthread+0xd1/0xe0
[<ffffffff9d0c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff9d792d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9d0c65c0>] ? insert_kthread_work+0x40/0x40

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [cpaneld - servi:20803]
INFO: rcu_sched self-detected stall on CPU { 0}  (t=61114 jiffies g=124202003 c=124202002 q=756)
Task dump for CPU 0:
kworker/0:22    R  running task        0 16125      2 0x00000080
Workqueue: events e1000_watchdog [e1000]
Call Trace:
<IRQ>  [<ffffffff9b2da648>] sched_show_task+0xa8/0x110
[<ffffffff9b2de3e9>] dump_cpu_task+0x39/0x70
[<ffffffff9b357fa0>] rcu_dump_cpu_stacks+0x90/0xd0
[<ffffffff9b35b662>] rcu_check_callbacks+0x442/0x730
[<ffffffff9b30ff40>] ? tick_sched_do_timer+0x50/0x50
[<ffffffff9b2af536>] update_process_times+0x46/0x80
[<ffffffff9b30fcb0>] tick_sched_handle+0x30/0x70
[<ffffffff9b30ff79>] tick_sched_timer+0x39/0x80
[<ffffffff9b2ca5ee>] __hrtimer_run_queues+0x10e/0x270
[<ffffffff9b2cab4f>] hrtimer_interrupt+0xaf/0x1d0
[<ffffffff9b25c60b>] local_apic_timer_interrupt+0x3b/0x60
[<ffffffff9b9929d3>] smp_apic_timer_interrupt+0x43/0x60
[<ffffffff9b98eefa>] apic_timer_interrupt+0x16a/0x170
<EOI>  [<ffffffff9b983025>] ? _raw_spin_unlock_irqrestore+0x15/0x20
[<ffffffffc03de60c>] e1000_update_stats+0x94c/0x990 [e1000]
[<ffffffffc03de6bd>] e1000_watchdog+0x6d/0x550 [e1000]
[<ffffffff9b2be21f>] process_one_work+0x17f/0x440
[<ffffffff9b2bf336>] worker_thread+0x126/0x3c0
[<ffffffff9b2bf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9b2c61f1>] kthread+0xd1/0xe0
[<ffffffff9b2c6120>] ? insert_kthread_work+0x40/0x40
[<ffffffff9b98dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9b2c6120>] ? insert_kthread_work+0x40/0x40