NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [smtpd:15182]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nls_utf8 isofs ext4 mbcache jbd2 ppdev pcspkr sg joydev parport_pc i2c_piix4 shpchp parport virtio_balloon nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_scsi bochs_drm drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata e1000 floppy serio_raw virtio_pci virtio_ring virtio i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 15182 Comm: smtpd Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff88007a0e2f70 ti: ffff880077fc4000 task.ti: ffff880077fc4000
RIP: 0010:[<ffffffffc0089ee7>]  [<ffffffffc0089ee7>] e1000_xmit_frame+0x947/0x1040 [e1000]
RSP: 0018:ffff880077fc78a8  EFLAGS: 00000286
RAX: ffffc90000403818 RBX: ffff880077fc7830 RCX: 0000000000000094
RDX: 0000000000000094 RSI: ffff8800360e6740 RDI: ffff88007ac52000
RBP: ffff880077fc7928 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffff880079601310
R13: ffffffff81e87180 R14: ffffffffc052e280 R15: ffffffffc050274d
FS:  00007f72a5c74840(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000012fc510 CR3: 000000007a769000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffffc0189d06 ffff88007c253098 00000092c0189d06 0000000900000000
000000007b9e1300 ffff880000000000 ffffffff00000000 0000000100001000
ffff88007afd2af8 ffff88007ac52000 ffff88007c253000 ffff88007ac52000
Call Trace:
[<ffffffffc0189d06>] ? ipt_do_table+0x336/0x710 [ip_tables]
[<ffffffff81586771>] dev_hard_start_xmit+0x171/0x3b0
[<ffffffff815af50a>] sch_direct_xmit+0x11a/0x240
[<ffffffff81589566>] __dev_queue_xmit+0x226/0x550
[<ffffffff815898a0>] dev_queue_xmit+0x10/0x20
[<ffffffff815cfd3e>] ip_finish_output+0x52e/0x780
[<ffffffff815d0293>] ip_output+0x73/0xe0
[<ffffffff815cf810>] ? __ip_append_data.isra.48+0xa00/0xa00
[<ffffffff815cdf31>] ip_local_out_sk+0x31/0x40
[<ffffffff815ce2a3>] ip_queue_xmit+0x143/0x3a0
[<ffffffff815e7c0f>] tcp_transmit_skb+0x4cf/0x9b0
[<ffffffff815e824a>] tcp_write_xmit+0x15a/0xce0
[<ffffffff815e904e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff815d77dc>] tcp_push+0xec/0x120
[<ffffffff815db0f8>] tcp_sendmsg+0xc8/0xc30
[<ffffffff81606893>] inet_sendmsg+0x63/0xb0
[<ffffffff815697c7>] sock_aio_write+0x157/0x180
[<ffffffff8120026d>] do_sync_write+0x8d/0xd0
[<ffffffff81200e25>] vfs_write+0x1b5/0x1e0
[<ffffffff81201b3f>] SyS_write+0x7f/0xe0
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 79s! [vmtoolsd:796]
[<ffffffff98fee051>] __dev_queue_xmit+0x4a1/0x660
[<ffffffff98fee220>] dev_queue_xmit+0x10/0x20
[<ffffffff99037416>] ip_finish_output+0x546/0x7a0
[<ffffffff99037973>] ip_output+0x73/0xe0
[<ffffffff99036ed0>] ? __ip_append_data.isra.50+0xa50/0xa50
[<ffffffff99035567>] ip_local_out_sk+0x37/0x40
[<ffffffff990358d3>] ip_queue_xmit+0x143/0x3a0
[<ffffffff9904f7b4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff9904fe22>] tcp_write_xmit+0x172/0xd00
[<ffffffff99050c2e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff9903f12c>] tcp_push+0xec/0x120
[<ffffffff99042ae2>] tcp_sendmsg+0xd2/0xc80
[<ffffffff9906e5a9>] inet_sendmsg+0x69/0xb0
[<ffffffff98cd36b3>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff98fcb84d>] sock_aio_write+0x15d/0x180
[<ffffffff98c1a1f3>] do_sync_write+0x93/0xe0
[<ffffffff98c1add5>] vfs_write+0x1c5/0x1f0
[<ffffffff98c1baff>] SyS_write+0x7f/0xf0
[<ffffffff9911f7d5>] system_call_fastpath+0x1c/0x21

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 41s! [SISServiceHctjH:2087]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq btusb btrtl btbcm btintel iosf_mbi snd_seq_device snd_pcm bluetooth crc32_pclmul snd_timer ghash_clmulni_intel
snd aesni_intel lrw rfkill gf128mul glue_helper soundcore ablk_helper cryptd ppdev pcspkr joydev sg vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops ttm ata_piix nfit libata e1000 libnvdimm drm serio_raw mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2087 Comm: SISServiceHctjH Kdump: loaded Not tainted 3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff90b86580e2a0 ti: ffff90b86b308000 task.ti: ffff90b86b308000
RIP: 0010:[<ffffffffc01a1877>]  [<ffffffffc01a1877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff90b86b30b830  EFLAGS: 00000286
RAX: ffffab61406c3818 RBX: 0000000000000001 RCX: 000000000000004d
RDX: 000000000000004d RSI: ffff90b7f5d99280 RDI: ffff90b8f243c000
RBP: ffff90b86b30b8b0 R08: 0000000000000100 R09: 0000000022100000
R10: 4500082ca4e7290c R11: 0000000000000000 R12: 0000000000000202
R13: 0000000133c1d0ce R14: 00000001f5d99280 R15: ffffffff813b0770
FS:  00007fd3cb611700(0000) GS:ffff90b8f96c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f42dc683000 CR3: 000000013398e000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8163b6f7>] ? kfree_skbmem+0x37/0x90
[<ffffffff81653386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8168131a>] sch_direct_xmit+0x11a/0x250
[<ffffffff81656301>] __dev_queue_xmit+0x491/0x650
[<ffffffff816564d0>] dev_queue_xmit+0x10/0x20
[<ffffffff816a429b>] ip_finish_output+0x67b/0x8d0
[<ffffffff816a481b>] ip_output+0x7b/0xf0
[<ffffffff816a3c20>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff816a2247>] ip_local_out_sk+0x37/0x40
[<ffffffff816a25e4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff816bccc4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff816bd34a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff816be18e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff816ac23c>] tcp_push+0xec/0x120
[<ffffffff816afc82>] tcp_sendmsg+0xd2/0xc60
[<ffffffff81638a18>] ? release_sock+0x128/0x180
[<ffffffff816dc379>] inet_sendmsg+0x69/0xb0
[<ffffffff8130b283>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff8163257d>] sock_aio_write+0x15d/0x180
[<ffffffff8124c663>] do_sync_write+0x93/0xe0
[<ffffffff8124d255>] vfs_write+0x1c5/0x1f0
[<ffffffff8124df1f>] SyS_write+0x7f/0xf0
[<ffffffff81792ed2>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [java:95344]
Modules linked in: vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi ppdev snd_ac97_codec aesni_intel lrw vmw_balloon ac97_bus gf128mul btusb snd_seq glue_helper ablk_helper cryptd btrtl btbcm snd_seq_device snd_pcm btintel bluetooth joydev pcspkr snd_timer snd rfkill soundcore sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul drm mptspi crct10dif_common scsi_transport_spi libata crc32c_intel nfit mptscsih mptbase e1000 serio_raw drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 95344 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff968871e6e2a0 ti: ffff96879a050000 task.ti: ffff96879a050000
RIP: 0010:[<ffffffffc01c0877>]  [<ffffffffc01c0877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff96879a053830  EFLAGS: 00000286
RAX: ffffa3b800983818 RBX: 0000000000000001 RCX: 0000000000000027
RDX: 0000000000000027 RSI: ffff968871b3e880 RDI: ffff9688757dc000
RBP: ffff96879a0538b0 R08: 0000000000000100 R09: 0000000022100000
R10: 5d0d0100450008d1 R11: 0000000000000000 R12: 0000000000000202
R13: 000000013220b8ce R14: 0000000100000000 R15: ffffffff9c3aabc0
FS:  00007fb7a01c6700(0000) GS:ffff968879600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f42997e08b0 CR3: 000000009a1ba000 CR4: 00000000007607f0
PKRU: 55555554
Call Trace:
[<ffffffff9c64d346>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff9c67b0ba>] sch_direct_xmit+0x11a/0x250
[<ffffffff9c6502b1>] __dev_queue_xmit+0x491/0x650
[<ffffffff9c650480>] dev_queue_xmit+0x10/0x20
[<ffffffff9c69e97b>] ip_finish_output+0x67b/0x8d0
[<ffffffff9c69eefb>] ip_output+0x7b/0xf0
[<ffffffff9c69c927>] ip_local_out_sk+0x37/0x40
[<ffffffff9c69ccc4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff9c6b7304>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff9c6b798a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff9c6b87be>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff9c6a68ec>] tcp_push+0xec/0x120
[<ffffffff9c6aa322>] tcp_sendmsg+0xd2/0xc60
[<ffffffff9c77ec28>] ? __schedule+0x448/0x9c0
[<ffffffff9c6d6939>] inet_sendmsg+0x69/0xb0
[<ffffffff9c1114e5>] ? futex_wait+0x1a5/0x280
[<ffffffff9c62c57d>] sock_aio_write+0x15d/0x180
[<ffffffff9c0c8d10>] ? hrtimer_get_res+0x50/0x50
[<ffffffff9c247ba3>] do_sync_write+0x93/0xe0
[<ffffffff9c248795>] vfs_write+0x1c5/0x1f0
[<ffffffff9c2494af>] SyS_write+0x7f/0xf0
[<ffffffff9c78bede>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [smtp:92537]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq vmw_balloon aesni_intel snd_seq_device snd_pcm lrw gf128mul glue_helper ablk_helper
cryptd snd_timer joydev pcspkr sg snd vmw_vmci i2c_piix4 soundcore parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit libnvdimm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel e1000 libata serio_raw mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 92537 Comm: smtp Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.105.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff983c27831080 ti: ffff983ab2550000 task.ti: ffff983ab2550000
RIP: 0010:[<ffffffffc026a8e7>]  [<ffffffffc026a8e7>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff983ab2553830  EFLAGS: 00000286
RAX: ffffa651c0fc3818 RBX: 0000000000000001 RCX: 0000000000000012
RDX: 0000000000000012 RSI: ffff983a75ca8700 RDI: ffff983c747b0000
RBP: ffff983ab25538b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000202
R13: 00000001eb659b17 R14: 0000000186735bc0 R15: ffffffff859c9110
FS:  00007f89973e8880(0000) GS:ffff983c75600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9e2821a000 CR3: 000000008ba00000 CR4: 00000000007607f0
PKRU: 55555554
Call Trace:
[<ffffffff85c7edc6>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff85cadd72>] sch_direct_xmit+0x122/0x250
[<ffffffff85c81dd1>] __dev_queue_xmit+0x491/0x650
[<ffffffff85c81fa0>] dev_queue_xmit+0x10/0x20
[<ffffffff85cd260b>] ip_finish_output+0x67b/0x8d0
[<ffffffff85cd2b9b>] ip_output+0x7b/0x100
[<ffffffff85cd1f90>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff85cd0587>] ip_local_out_sk+0x37/0x40
[<ffffffff85cd0924>] ip_queue_xmit+0x144/0x3d0
[<ffffffff85ceb5f4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff85cebc7a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff85cecace>] __tcp_push_pending_frames+0x2e/0xd0
[<ffffffff85cda7cc>] tcp_push+0xec/0x120
[<ffffffff85cde272>] tcp_sendmsg+0xd2/0xc60
[<ffffffff85c63ea8>] ? release_sock+0x128/0x180
[<ffffffff85d0b4e9>] inet_sendmsg+0x69/0xb0
[<ffffffff85d0b690>] ? inet_recvmsg+0x80/0xb0
[<ffffffff85c5d675>] sock_aio_write+0x165/0x180
[<ffffffff8585b263>] do_sync_write+0x93/0xe0
[<ffffffff8585be35>] vfs_write+0x1c5/0x1f0
[<ffffffff8585cae5>] SyS_write+0x55/0xd0
[<ffffffff85dc539a>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 43s! [java:3190]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmhgfs(OE) vmw_vsock_vmci_transport vsock bnep snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 snd_ens1371 snd_rawmidi snd_ac97_codec crc32_pclmul ghash_clmulni_intel ac97_bus snd_seq ppdev snd_seq_device aesni_intel snd_pcm btusb lrw gf128mul vmw_balloon glue_helper ablk_helper cryptd btrtl btbcm btintel bluetooth snd_timer nfit joydev snd sg pcspkr rfkill libnvdimm soundcore vmw_vmci shpchp i2c_piix4 parport_pc parport nfsd
auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ata_piix libata e1000 mptspi scsi_transport_spi mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 3190 Comm: java Tainted: G           OE  ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8800a9f89fa0 ti: ffff8800a9fd4000 task.ti: ffff8800a9fd4000
RIP: 0010:[<ffffffffc0087ee7>]  [<ffffffffc0087ee7>] e1000_xmit_frame+0x947/0x1040 [e1000]
RSP: 0018:ffff8800a9fd78a8  EFLAGS: 00000286
RAX: ffffc90000d43818 RBX: 000000020bf0d0ce RCX: 000000000000005c
RDX: 000000000000005c RSI: ffff88023ff19640 RDI: ffff880235542000
RBP: ffff8800a9fd7928 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffffffff813492f0
R13: ffff8800a9fd7870 R14: 0000000000000202 R15: 0000000000000202
FS:  00007ff31c9ba700(0000) GS:ffff880235680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbf200ee000 CR3: 0000000226338000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffffc00a9d06 ffff880235ded098 0000005ac00a9d06 0000000900000000
000000002226d000 ffff880000000000 ffffffff00000000 0000000100001000
ffff8800ac2316f8 ffff880235542000 ffff880235ded000 ffff880235542000
Call Trace:
[<ffffffffc00a9d06>] ? ipt_do_table+0x336/0x710 [ip_tables]
[<ffffffff81586771>] dev_hard_start_xmit+0x171/0x3b0
[<ffffffff815af50a>] sch_direct_xmit+0x11a/0x240
[<ffffffff81589566>] __dev_queue_xmit+0x226/0x550
[<ffffffff815898a0>] dev_queue_xmit+0x10/0x20
[<ffffffff815cfd3e>] ip_finish_output+0x52e/0x780
[<ffffffff815d0293>] ip_output+0x73/0xe0
[<ffffffff815cf810>] ? __ip_append_data.isra.48+0xa00/0xa00
[<ffffffff815cdf31>] ip_local_out_sk+0x31/0x40
[<ffffffff815ce2a3>] ip_queue_xmit+0x143/0x3a0
[<ffffffff815e7c0f>] tcp_transmit_skb+0x4cf/0x9b0
[<ffffffff815e824a>] tcp_write_xmit+0x15a/0xce0
[<ffffffff815e904e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff815d77dc>] tcp_push+0xec/0x120
[<ffffffff815db0f8>] tcp_sendmsg+0xc8/0xc30
[<ffffffff81606893>] inet_sendmsg+0x63/0xb0
[<ffffffff810f5e15>] ? futex_wait+0x1a5/0x280
[<ffffffff815697c7>] sock_aio_write+0x157/0x180
[<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8120026d>] do_sync_write+0x8d/0xd0
[<ffffffff81200e25>] vfs_write+0x1b5/0x1e0
[<ffffffff81201b3f>] SyS_write+0x7f/0xe0
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#11 stuck for 21s! [sshd:3000]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables vboxsf(OE) ip6table_filter ip6_tables iptable_filter sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device iTCO_wdt snd_pcm iTCO_vendor_support snd_timer crc32_pclmul ppdev joydev ghash_clmulni_intel lpc_ich snd aesni_intel parport_pc lrw i2c_piix4 gf128mul glue_helper ablk_helper
vboxguest(OE) soundcore video pcspkr parport cryptd sg ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect ahci sysimgblt fb_sys_fops libahci ata_piix crct10dif_pclmul crct10dif_common ttm serio_raw drm crc32c_intel libata e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 11 PID: 3000 Comm: sshd Kdump: loaded Tainted: G           OE  ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8ac84d3e6300 ti: ffff8aceba448000 task.ti: ffff8aceba448000
RIP: 0010:[<ffffffffc0360877>]  [<ffffffffc0360877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff8aceba44b7d8  EFLAGS: 00010286
RAX: ffff98bc03703818 RBX: 0000000000000004 RCX: 00000000000000ba
RDX: 00000000000000ba RSI: ffff8ac85f7a0680 RDI: ffff8ac844782000
RBP: ffff8aceba44b858 R08: 0000000000000100 R09: 0000000026100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000735df0c8c
R13: 0000000197ee46dc R14: ffffffff981b1c00 R15: ffff8aceba44b790
FS:  00007f2947da58c0(0000) GS:ffff8acf032c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3e7ff25000 CR3: 0000000150fae000 CR4: 00000000000206e0
Call Trace:
[<ffffffff98455f96>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff984840aa>] sch_direct_xmit+0x11a/0x250
[<ffffffff9848426e>] __qdisc_run+0x8e/0x360
[<ffffffff98458f29>] __dev_queue_xmit+0x4a9/0x650
[<ffffffff984590e0>] dev_queue_xmit+0x10/0x20
[<ffffffff984a70eb>] ip_finish_output+0x67b/0x8d0
[<ffffffff984a766b>] ip_output+0x7b/0xf0
[<ffffffff984a6a70>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff984a5097>] ip_local_out_sk+0x37/0x40
[<ffffffff984a5434>] ip_queue_xmit+0x144/0x3c0
[<ffffffff984bfb74>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff984c01fa>] tcp_write_xmit+0x18a/0xd40
[<ffffffff984c103e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff984af08c>] tcp_push+0xec/0x120
[<ffffffff984b2ad2>] tcp_sendmsg+0xd2/0xc60
[<ffffffff9843ba18>] ? release_sock+0x128/0x180
[<ffffffff984df289>] inet_sendmsg+0x69/0xb0
[<ffffffff9843557d>] sock_aio_write+0x15d/0x180
[<ffffffff9804d313>] do_sync_write+0x93/0xe0
[<ffffffff9804df05>] vfs_write+0x1c5/0x1f0
[<ffffffff98595ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff9804ebdf>] SyS_write+0x7f/0xf0
[<ffffffff98595ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff98595f92>] system_call_fastpath+0x25/0x2a
[<ffffffff98595ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 380s! [frpc:813]
Modules linked in: vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper pcspkr ablk_helper cryptd joydev vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix drm libata serio_raw mptspi scsi_transport_spi mptscsih mptbase e1000 nfit drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 813 Comm: frpc Kdump: loaded Not tainted 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9aecf4cc0000 ti: ffff9aecf4cc8000 task.ti: ffff9aecf4cc8000
RIP: 0010:[<ffffffffc01f0877>]  [<ffffffffc01f0877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff9aecf4ccb830  EFLAGS: 00000286
RAX: ffffbbe200403818 RBX: ffff9aecf4ccb7c0 RCX: 0000000000000064
RDX: 0000000000000064 RSI: ffff9aecb5e88200 RDI: ffff9aecf5b92000
RBP: ffff9aecf4ccb8b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000010
R13: ffffffffc01f0877 R14: ffffffffffffff6b R15: ffff9aecf5b92000
FS:  000000c000040490(0000) GS:ffff9aecfb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0c9aff5000 CR3: 00000000764e0000 CR4: 00000000003607f0
Call Trace:
[<ffffffff84854386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8488242a>] sch_direct_xmit+0x11a/0x250
[<ffffffff84857301>] __dev_queue_xmit+0x491/0x650
[<ffffffff848574d0>] dev_queue_xmit+0x10/0x20
[<ffffffff848a545b>] ip_finish_output+0x67b/0x8d0
[<ffffffff848a59db>] ip_output+0x7b/0xf0
[<ffffffff848a4de0>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff848a3407>] ip_local_out_sk+0x37/0x40
[<ffffffff848a37a4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff848bde94>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff848be51a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff848bf35e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff848ad3fc>] tcp_push+0xec/0x120
[<ffffffff848b0e42>] tcp_sendmsg+0xd2/0xc60
[<ffffffff842de185>] ? sched_clock_cpu+0x85/0xc0
[<ffffffff848dd589>] inet_sendmsg+0x69/0xb0
[<ffffffff8450bf93>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff8483357d>] sock_aio_write+0x15d/0x180
[<ffffffff8444d063>] do_sync_write+0x93/0xe0
[<ffffffff8444dc55>] vfs_write+0x1c5/0x1f0
[<ffffffff8444e92f>] SyS_write+0x7f/0xf0
[<ffffffff84993f92>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 29s! [sshd:8374]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device pcspkr joydev snd_pcm snd_timer snd soundcore sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel
serio_raw ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix e1000 mptspi libata scsi_transport_spi mptscsih nfit mptbase drm_panel_orientation_quirks libnvdimm
CPU: 0 PID: 8374 Comm: sshd Kdump: loaded Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff921436224100 ti: ffff921435e5c000 task.ti: ffff921435e5c000
RIP: 0010:[<ffffffffc019d857>]  [<ffffffffc019d857>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff921435e5f850  EFLAGS: 00000286
RAX: ffffb45500443818 RBX: ffff9214000001c1 RCX: 00000000000000b7
RDX: 00000000000000b7 RSI: ffff9213f632c300 RDI: ffff92143897c000
RBP: ffff921435e5f8d0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffffffffc068278d
R13: ffff921435e5f890 R14: 7fffffffffffffff R15: ffffffffa8b656a1
FS:  00007f6c45a768c0(0000) GS:ffff92143b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c410d8458 CR3: 0000000075e4a000 CR4: 00000000003607f0
Call Trace:
[<ffffffffa8a38c66>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffa8a65cea>] sch_direct_xmit+0x11a/0x250
[<ffffffffa8a3bbe1>] __dev_queue_xmit+0x491/0x650
[<ffffffffa8a3bdb0>] dev_queue_xmit+0x10/0x20
[<ffffffffa8a88736>] ip_finish_output+0x556/0x7b0
[<ffffffffa8a88c93>] ip_output+0x73/0xe0
[<ffffffffa8a881e0>] ? __ip_append_data.isra.51+0xa50/0xa50
[<ffffffffa8a86817>] ip_local_out_sk+0x37/0x40
[<ffffffffa8a86bb4>] ip_queue_xmit+0x144/0x3c0
[<ffffffffa8aa0f64>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffa8aa15d2>] tcp_write_xmit+0x172/0xd00
[<ffffffffa8aa23de>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffa8a9063c>] tcp_push+0xec/0x120
[<ffffffffa8a94092>] tcp_sendmsg+0xd2/0xc60
[<ffffffffa85f6e08>] ? page_add_new_anon_rmap+0xb8/0x170
[<ffffffffa8abfeb9>] inet_sendmsg+0x69/0xb0
[<ffffffffa8a1857d>] sock_aio_write+0x15d/0x180
[<ffffffffa86405b3>] do_sync_write+0x93/0xe0
[<ffffffffa86411a5>] vfs_write+0x1c5/0x1f0
[<ffffffffa8641ebf>] SyS_write+0x7f/0xf0
[<ffffffffa8b74ddb>] system_call_fastpath+0x22/0x27
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 55s! [gitlab-runner:5986]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vboxsf(OE) nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dm_mirror dm_region_hash dm_log dm_mod joydev ppdev nfit libnvdimm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_intel8x0 lrw gf128mul snd_ac97_codec glue_helper ac97_bus ablk_helper cryptd snd_seq snd_seq_device snd_pcm sg pcspkr i2c_piix4 parport_pc vboxguest(OE) snd_timer video parport snd soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix drm ahci libahci e1000 libata drm_panel_orientation_quirks
CPU: 3 PID: 5986 Comm: gitlab-runner Kdump: loaded Tainted: G           OE  ------------   3.10.0-957.12.2.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9d961da2c100 ti: ffff9d914ec8c000 task.ti: ffff9d914ec8c000
RIP: 0010:[<ffffffffc0160857>]  [<ffffffffc0160857>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff9d914ec8f850  EFLAGS: 00010286
RAX: ffffae38c36c3818 RBX: 0000000000000001 RCX: 00000000000000a2
RDX: 00000000000000a2 RSI: ffff9d969fe82b00 RDI: ffff9d9686972000
RBP: ffff9d914ec8f8d0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000202
R13: 00000005fd2798ce R14: 0000000100000100 R15: ffffffffb2da0c60
FS:  000000c000090790(0000) GS:ffff9d969fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f147d400000 CR3: 00000005f4bbe000 CR4: 00000000000206e0
Call Trace:
[<ffffffffb3039ce6>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffb3066d6a>] sch_direct_xmit+0x11a/0x250
[<ffffffffb303cc61>] __dev_queue_xmit+0x491/0x650
[<ffffffffb303ce30>] dev_queue_xmit+0x10/0x20
[<ffffffffb30897b6>] ip_finish_output+0x556/0x7b0
[<ffffffffb3089d13>] ip_output+0x73/0xe0
[<ffffffffb3089260>] ? __ip_append_data.isra.51+0xa50/0xa50
[<ffffffffb3087897>] ip_local_out_sk+0x37/0x40
[<ffffffffb3087c34>] ip_queue_xmit+0x144/0x3c0
[<ffffffffb30a1fe4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffb30a2652>] tcp_write_xmit+0x172/0xd00
[<ffffffffb30a345e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffb30916bc>] tcp_push+0xec/0x120
[<ffffffffb3095112>] tcp_sendmsg+0xd2/0xc60
[<ffffffffb301fa00>] ? release_sock+0x120/0x170
[<ffffffffb30c1079>] inet_sendmsg+0x69/0xb0
[<ffffffffb2cfcc63>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffffb301957d>] sock_aio_write+0x15d/0x180
[<ffffffffb2c40d23>] do_sync_write+0x93/0xe0
[<ffffffffb2c41915>] vfs_write+0x1c5/0x1f0
[<ffffffffb3175d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb2c4262f>] SyS_write+0x7f/0xf0
[<ffffffffb3175d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb3175ddb>] system_call_fastpath+0x22/0x27
[<ffffffffb3175d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 27s! [kube-apiserver:3012]
Modules linked in: rbd libceph dns_resolver xt_multiport ipt_rpfilter ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth xt_set ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set nf_tables(T) ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs dummy iptable_raw xt_CT ip6table_mangle ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel aesni_intel lrw
gf128mul glue_helper ablk_helper cryptd snd_ens1371 joydev pcspkr snd_rawmidi snd_ac97_codec ac97_bus snd_seq nfit snd_seq_device snd_pcm ppdev snd_timer snd sg soundcore libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 overlay(T) br_netfilter bridge stp llc nfsd nf_conntrack_ipv4 auth_rpcgss nf_defrag_ipv4 nfs_acl lockd grace sunrpc nf_conntrack libcrc32c ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix mptspi scsi_transport_spi libata mptscsih serio_raw e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 3012 Comm: kube-apiserver Tainted: G               ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9848ade4eeb0 ti: ffff9849f7e48000 task.ti: ffff9849f7e48000
RIP: 0010:[<ffffffffa7feb1cb>]  [<ffffffffa7feb1cb>] dev_hard_start_xmit+0x2ab/0x3b0
RSP: 0018:ffff9849f7e4b8f0  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff984a37535098 RCX: 000000000000002b
RDX: 00000000b352e66b RSI: ffff9849c3d730c0 RDI: ffff98497c510000
RBP: ffff9849f7e4b958 R08: 0000000000000100 R09: 0000000026100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000080a586e0
R13: ffffffffc040c837 R14: ffff9849f7e4b8e0 R15: 0000000000000618
FS:  000000c000100c90(0000) GS:ffff984a35680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcd7c51c000 CR3: 00000001eae2a000 CR4: 00000000003407e0
Call Trace:
[<ffffffffa8016a2a>] sch_direct_xmit+0x11a/0x250
[<ffffffffa7fee051>] __dev_queue_xmit+0x4a1/0x660
[<ffffffffa7fee220>] dev_queue_xmit+0x10/0x20
[<ffffffffa8037416>] ip_finish_output+0x546/0x7a0
[<ffffffffa8037973>] ip_output+0x73/0xe0
[<ffffffffa8036ed0>] ? __ip_append_data.isra.50+0xa50/0xa50
[<ffffffffa8035567>] ip_local_out_sk+0x37/0x40
[<ffffffffa80358d3>] ip_queue_xmit+0x143/0x3a0
[<ffffffffa804f7b4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffa804fe22>] tcp_write_xmit+0x172/0xd00
[<ffffffffa8050c2e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffa803f12c>] tcp_push+0xec/0x120
[<ffffffffa8042ae2>] tcp_sendmsg+0xd2/0xc80
[<ffffffffa806e5a9>] inet_sendmsg+0x69/0xb0
[<ffffffffa806e750>] ? inet_recvmsg+0x80/0xb0
[<ffffffffa7fcb84d>] sock_aio_write+0x15d/0x180
[<ffffffffa7c1a1f3>] do_sync_write+0x93/0xe0
[<ffffffffa7c1add5>] vfs_write+0x1c5/0x1f0
[<ffffffffa7c1baff>] SyS_write+0x7f/0xf0
[<ffffffffa811fa51>] tracesys+0x9d/0xc3
Code: 00 03 0f 84 98 00 00 00 0f b6 55 b7 31 db b8 10 00 00 00 84 d2 48 0f 45 5d a8 41 0f 44 c5 48 8b 55 98 89 02 48 89 d8 48 8b 55 d0 <65> 48 33 14 25 28 00 00 00 0f 85 c9 00 00 00 48 83 c4 40 5b 41 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [mysqld:100528]
Modules linked in: xt_CHECKSUM tun lp devlink ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq coretemp snd_seq_device snd_pcm snd_timer snd iosf_mbi crc32_pclmul soundcore ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd ppdev pcspkr sg vmw_balloon joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul crct10dif_common ttm crc32c_intel drm ata_piix mptspi serio_raw scsi_transport_spi mptscsih mptbase libata e1000 drm_panel_orientation_quirks floppy fuse
CPU: 0 PID: 100528 Comm: mysqld Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.83.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
task: ffff8840f57d8000 ti: ffff8841efc64000 task.ti: ffff8841efc64000
RIP: 0010:[<ffffffffc03758e7>]  [<ffffffffc03758e7>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff8841efc677d8  EFLAGS: 00000286
RAX: ffffa7ee40703818 RBX: 0000000000000041 RCX: 0000000000000008
RDX: 0000000000000008 RSI: ffff8840f5d18140 RDI: ffff8841f45ba000
RBP: ffff8841efc67858 R08: 0000000000000000 R09: 0000000022100000
R10: 0000450008abd6ed R11: 0000000000000000 R12: 0000000134e430da
R13: 00000001f16d10e8 R14: ffffffffb39c8ad0 R15: ffff8841efc67790
FS:  00002ae1c818b700(0000) GS:ffff8841f9600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f789dc61500 CR3: 00000000aa066000 CR4: 00000000001607f0
Call Trace:
[<ffffffffb3c7edc6>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffb3cadd72>] sch_direct_xmit+0x122/0x250
[<ffffffffb3cadf2e>] __qdisc_run+0x8e/0x360
[<ffffffffb3c81de9>] __dev_queue_xmit+0x4a9/0x650
[<ffffffffb3c81fa0>] dev_queue_xmit+0x10/0x20
[<ffffffffb3cd260b>] ip_finish_output+0x67b/0x8d0
[<ffffffffb3cd2b9b>] ip_output+0x7b/0x100
[<ffffffffb3cd1f90>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffffb3cd0587>] ip_local_out_sk+0x37/0x40
[<ffffffffb3cd0924>] ip_queue_xmit+0x144/0x3d0
[<ffffffffb3ceb5f4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffb3cebc7a>] tcp_write_xmit+0x18a/0xd40
[<ffffffffb3cecace>] __tcp_push_pending_frames+0x2e/0xd0
[<ffffffffb3cda7cc>] tcp_push+0xec/0x120
[<ffffffffb3cde272>] tcp_sendmsg+0xd2/0xc60
[<ffffffffb3c63ea8>] ? release_sock+0x128/0x180
[<ffffffffb3d0b4e9>] inet_sendmsg+0x69/0xb0
[<ffffffffb3d0b690>] ? inet_recvmsg+0x80/0xb0
[<ffffffffb3c5d675>] sock_aio_write+0x165/0x180
[<ffffffffb385ae73>] do_sync_write+0x93/0xe0
[<ffffffffb385ba45>] vfs_write+0x1c5/0x1f0
[<ffffffffb385c6f5>] SyS_write+0x55/0xd0
[<ffffffffb3dc539a>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [sshd:19030]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device ppdev joydev snd_pcm snd_timer snd sg parport_pc iosf_mbi soundcore vboxguest(OE) parport i2c_piix4 video pcspkr binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix libahci drm crc32c_intel drm_panel_orientation_quirks libata e1000 dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 19030 Comm: sshd Kdump: loaded Tainted: G        W  OE  ------------   3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8e4c4bec6300 ti: ffff8e496274c000 task.ti: ffff8e496274c000
RIP: 0010:[<ffffffffc0446877>]  [<ffffffffc0446877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff8e496274f7f0  EFLAGS: 00010286
RAX: ffffa2b081c03818 RBX: 0000000000000001 RCX: 00000000000000ea
RDX: 00000000000000ea RSI: ffff8e4c5ad33b80 RDI: ffff8e4a2bfcc000
RBP: ffff8e496274f870 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000202
R13: 00000003975890da R14: 0000000191ee116e R15: ffffffff921b1690
FS:  00007fd939fec8c0(0000) GS:ffff8e4c6aa00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4d77b4e160 CR3: 00000000d8f36000 CR4: 00000000000206f0
Call Trace:
[<ffffffff9243c6f7>] ? kfree_skbmem+0x37/0x90
[<ffffffff924543e6>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff9248248a>] sch_direct_xmit+0x11a/0x250
[<ffffffff92457361>] __dev_queue_xmit+0x491/0x650
[<ffffffff91ea4017>] ? local_bh_enable+0x17/0x20
[<ffffffff92457530>] dev_queue_xmit+0x10/0x20
[<ffffffff9246425d>] neigh_resolve_output+0x11d/0x220
[<ffffffff924a50cc>] ip_finish_output+0x27c/0x8d0
[<ffffffff924a5a4b>] ip_output+0x7b/0xf0
[<ffffffff924a4e50>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff924a3477>] ip_local_out_sk+0x37/0x40
[<ffffffff924a3814>] ip_queue_xmit+0x144/0x3c0
[<ffffffff924bdf24>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff924be5aa>] tcp_write_xmit+0x18a/0xd40
[<ffffffff924bf3ee>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff924ad46c>] tcp_push+0xec/0x120
[<ffffffff924b0eb2>] tcp_sendmsg+0xd2/0xc60
[<ffffffff92439a18>] ? release_sock+0x128/0x180
[<ffffffff924dd619>] inet_sendmsg+0x69/0xb0
[<ffffffff9243357d>] sock_aio_write+0x15d/0x180
[<ffffffff9204d0a3>] do_sync_write+0x93/0xe0
[<ffffffff9204dc95>] vfs_write+0x1c5/0x1f0
[<ffffffff92594ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff9204e96f>] SyS_write+0x7f/0xf0
[<ffffffff92594ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff92594f92>] system_call_fastpath+0x25/0x2a
[<ffffffff92594ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 33s! [java:18876]
pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm e1000 mptspi ata_piix scsi_transport_spi mptscsih mptbase libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 18876 Comm: java Kdump: loaded Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff959f48ec0000 ti: ffff959f48f70000 task.ti: ffff959f48f70000
RIP: 0010:[<ffffffffc0195857>]  [<ffffffffc0195857>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff959f48f73850  EFLAGS: 00000286
RAX: ffffa47340403818 RBX: ffff959f621c6250 RCX: 0000000000000081
RDX: 0000000000000081 RSI: ffff959f350a5240 RDI: ffff959f784ba000
RBP: ffff959f48f738d0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffffffffc067478d
R13: ffff959f48f73890 R14: ffff959f784ba000 R15: 0000000000002328
FS:  00007f672699a700(0000) GS:ffff959f7b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000f040e000 CR3: 000000000135e000 CR4: 0000000000760ff0
PKRU: 55555554
Call Trace:
[<ffffffffb2038c66>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffb2065cea>] sch_direct_xmit+0x11a/0x250
[<ffffffffb203bbe1>] __dev_queue_xmit+0x491/0x650
[<ffffffffb203bdb0>] dev_queue_xmit+0x10/0x20
[<ffffffffb2088736>] ip_finish_output+0x556/0x7b0
[<ffffffffb2088c93>] ip_output+0x73/0xe0
[<ffffffffb20881e0>] ? __ip_append_data.isra.51+0xa50/0xa50
[<ffffffffb2086817>] ip_local_out_sk+0x37/0x40
[<ffffffffb2086bb4>] ip_queue_xmit+0x144/0x3c0
[<ffffffffb20a0f64>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffb20a15d2>] tcp_write_xmit+0x172/0xd00
[<ffffffffb20a23de>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffb209063c>] tcp_push+0xec/0x120
[<ffffffffb2094092>] tcp_sendmsg+0xd2/0xc60
[<ffffffffb216770f>] ? __schedule+0x3ff/0x890
[<ffffffffb20bfeb9>] inet_sendmsg+0x69/0xb0
[<ffffffffb1cfc4d3>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffffb201857d>] sock_aio_write+0x15d/0x180
[<ffffffffb1c405b3>] do_sync_write+0x93/0xe0
[<ffffffffb1c411a5>] vfs_write+0x1c5/0x1f0
[<ffffffffb1c41ebf>] SyS_write+0x7f/0xf0
[<ffffffffb2174ddb>] system_call_fastpath+0x22/0x27
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 119s! [sshd:27454]
Modules linked in: vboxsf(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc squashfs loop snd_intel8x0 snd_ac97_codec ac97_bus snd_seq ppdev snd_seq_device joydev snd_pcm iosf_mbi pcspkr snd_timer sg snd parport_pc vboxguest(OE) soundcore parport video i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_generic cdrom crct10dif_common
ata_generic pata_acpi crc32c_intel ahci libahci ata_piix vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw libata drm_panel_orientation_quirks e1000 dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 27454 Comm: sshd Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff89a910799080 ti: ffff89a781c6c000 task.ti: ffff89a781c6c000
RIP: 0010:[<ffffffffc042b877>]  [<ffffffffc042b877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff89a781c6f7f0  EFLAGS: 00010286
RAX: ffffa45741bc3818 RBX: 0000000000000001 RCX: 0000000000000029
RDX: 0000000000000029 RSI: ffff89a6b5cdf000 RDI: ffff89a915072000
RBP: ffff89a781c6f870 R08: 0000000000000100 R09: 0000000026100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000202
R13: 00000002942858da R14: 000000018b4d3d4a R15: ffffffff8b7b20d0
FS:  00007f308232d8c0(0000) GS:ffff89a91fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00003980b2faa010 CR3: 000000011b770000 CR4: 00000000000206e0
Call Trace:
[<ffffffff8ba58f86>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8ba8709a>] sch_direct_xmit+0x11a/0x250
[<ffffffff8ba5bf01>] __dev_queue_xmit+0x491/0x650
[<ffffffff8ba5c0d0>] dev_queue_xmit+0x10/0x20
[<ffffffff8ba68dfd>] neigh_resolve_output+0x11d/0x220
[<ffffffff8baaa85c>] ip_finish_output+0x27c/0x8d0
[<ffffffff8baab1db>] ip_output+0x7b/0xf0
[<ffffffff8baaa5e0>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff8baa8c07>] ip_local_out_sk+0x37/0x40
[<ffffffff8baa8fa4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff8bac3734>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff8bac3dba>] tcp_write_xmit+0x18a/0xd40
[<ffffffff8bac4bfe>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff8bab2c0c>] tcp_push+0xec/0x120
[<ffffffff8bab6652>] tcp_sendmsg+0xd2/0xc60
[<ffffffff8bae2e49>] inet_sendmsg+0x69/0xb0
[<ffffffff8b70c7d3>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff8ba3856d>] sock_aio_write+0x15d/0x180
[<ffffffff8b64da43>] do_sync_write+0x93/0xe0
[<ffffffff8b64e5d5>] vfs_write+0x1c5/0x1f0
[<ffffffff8b64f255>] SyS_write+0x55/0xd0
[<ffffffff8bb99ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff8bb99f92>] system_call_fastpath+0x25/0x2a
[<ffffffff8bb99ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 52s! [bird:4484]
Modules linked in: veth xt_multiport xt_set ipt_rpfilter iptable_raw ip_set_hash_ip ip_set_hash_net ip_set ipip tunnel4 ip_tunnel xt_nat xt_statistic ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev iosf_mbi snd_ens1371 crc32_pclmul ghash_clmulni_intel snd_rawmidi vmw_balloon snd_ac97_codec aesni_intel ac97_bus snd_seq
lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm joydev pcspkr snd_timer snd sg vmw_vmci soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c vmwgfx sr_mod sd_mod drm_kms_helper cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix nfit serio_raw mptspi libnvdimm libata scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 4484 Comm: bird Kdump: loaded Tainted: G               ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff972ff5fc5230 ti: ffff972ee483c000 task.ti: ffff972ee483c000
RIP: 0010:[<ffffffffc03dc877>]  [<ffffffffc03dc877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff972ee483f830  EFLAGS: 00000286
RAX: ffffc042809c3818 RBX: ffffffff8fb01c00 RCX: 0000000000000037
RDX: 0000000000000037 RSI: ffff972ef5eaeb00 RDI: ffff972ff571a000
RBP: ffff972ee483f8b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffff972ee483f878
R13: ffffffff8ed9f12d R14: ffff972ee483f7a8 R15: ffff972efca5fb78
FS:  00000000021c0340(0063) GS:ffff972ff9600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0fde9a3000 CR3: 00000000457e8000 CR4: 00000000003607f0
Call Trace:
[<ffffffff8f04d346>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8f07b0ba>] sch_direct_xmit+0x11a/0x250
[<ffffffff8f0502b1>] __dev_queue_xmit+0x491/0x650
[<ffffffff8f050480>] dev_queue_xmit+0x10/0x20
[<ffffffff8f09e97b>] ip_finish_output+0x67b/0x8d0
[<ffffffff8f09eefb>] ip_output+0x7b/0xf0
[<ffffffff8f09e300>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff8f09c927>] ip_local_out_sk+0x37/0x40
[<ffffffff8f09ccc4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff8f0b7304>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff8f0b798a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff8f0b87be>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff8f0a68ec>] tcp_push+0xec/0x120
[<ffffffff8f0aa322>] tcp_sendmsg+0xd2/0xc60
[<ffffffff8f08b5e8>] ? netlink_sendmsg+0x148/0x420
[<ffffffff8f0d6939>] inet_sendmsg+0x69/0xb0
[<ffffffff8f02c57d>] sock_aio_write+0x15d/0x180
[<ffffffff8ec47ba3>] do_sync_write+0x93/0xe0
[<ffffffff8ec48795>] vfs_write+0x1c5/0x1f0
[<ffffffff8ec494af>] SyS_write+0x7f/0xf0
[<ffffffff8f18bede>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [khugepaged:37]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [mysqld:121474]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi iosf_mbi crc32_pclmul snd_seq_midi_event ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg parport_pc parport vmw_vmci i2c_piix4
ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi nfit e1000 mptscsih mptbase libnvdimm ata_piix libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 121474 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff98027767a080 ti: ffff980137a08000 task.ti: ffff980137a08000
RIP: 0010:[<ffffffffc0197857>]  [<ffffffffc0197857>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff980137a0b850  EFLAGS: 00000286
RAX: ffffb75700d43818 RBX: 0000000000000001 RCX: 00000000000000e2
RDX: 00000000000000e2 RSI: ffff9800b5dc4600 RDI: ffff9802bfa4c000
RBP: ffff980137a0b8d0 R08: 0000000000000100 R09: 0000000022100000
R10: ac8fae06400040ea R11: 0000000000000000 R12: 0000000000000202
R13: 0000000229a200da R14: 0000000100000000 R15: ffffffffb8da04b0
FS:  00007f9affa6f700(0000) GS:ffff9802b5600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9b0c0b7388 CR3: 0000000227358000 CR4: 00000000007607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb9038c66>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffb9065cea>] sch_direct_xmit+0x11a/0x250
[<ffffffffb903bbe1>] __dev_queue_xmit+0x491/0x650
[<ffffffffb903bdb0>] dev_queue_xmit+0x10/0x20
[<ffffffffb9088736>] ip_finish_output+0x556/0x7b0
[<ffffffffb9088c93>] ip_output+0x73/0xe0
[<ffffffffb90881e0>] ? __ip_append_data.isra.51+0xa50/0xa50
[<ffffffffb9086817>] ip_local_out_sk+0x37/0x40
[<ffffffffb9086bb4>] ip_queue_xmit+0x144/0x3c0
[<ffffffffb90a0f64>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffb90a15d2>] tcp_write_xmit+0x172/0xd00
[<ffffffffb90a23de>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffb909063c>] tcp_push+0xec/0x120
[<ffffffffb9094092>] tcp_sendmsg+0xd2/0xc60
[<ffffffffb901e980>] ? release_sock+0x120/0x170
[<ffffffffb90bfeb9>] inet_sendmsg+0x69/0xb0
[<ffffffffb90c0060>] ? inet_recvmsg+0x80/0xb0
[<ffffffffb901857d>] sock_aio_write+0x15d/0x180
[<ffffffffb8bf6e08>] ? page_add_new_anon_rmap+0xb8/0x170
[<ffffffffb8c405b3>] do_sync_write+0x93/0xe0
[<ffffffffb8c411a5>] vfs_write+0x1c5/0x1f0
[<ffffffffb8c41ebf>] SyS_write+0x7f/0xf0
[<ffffffffb9174ddb>] system_call_fastpath+0x22/0x27
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [oracle_39617_wi:39617]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel lrw gf128mul ac97_bus snd_seq glue_helper ablk_helper vmw_balloon snd_seq_device cryptd snd_pcm joydev pcspkr snd_timer sg snd soundcore vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables ext4 mbcache jbd2 vmwgfx ata_generic pata_acpi drm_kms_helper sd_mod crc_t10dif
crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit libnvdimm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix mptspi libata scsi_transport_spi mptscsih serio_raw e1000 mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 39617 Comm: oracle_39617_wi Kdump: loaded Tainted: G             L ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff9d0750ce2100 ti: ffff9d071acb0000 task.ti: ffff9d071acb0000
RIP: 0010:[<ffffffff90256be2>]  [<ffffffff90256be2>] __dev_queue_xmit+0x162/0x650
RSP: 0018:ffff9d071acb3988  EFLAGS: 00000246
RAX: ffff9d08f73aea00 RBX: ffff9d071acb39d0 RCX: ffff9d06f5cc76c0
RDX: 0000000000000000 RSI: ffff9d07513a28f8 RDI: ffff9d08f6e79000
RBP: ffff9d071acb39f0 R08: 0000000000000010 R09: ffffffffc0776280
R10: ffff9d083df943e0 R11: ffff9d08f6e79000 R12: ffff9d0775e23a10
R13: ffff9d083df94010 R14: 0000000000000004 R15: ffff9d072502e5c8
FS:  00007f8a8915b3c0(0000) GS:ffff9d08f5600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000e7ec7a14 CR3: 00000000140ce000 CR4: 00000000003607f0
Call Trace:
[<ffffffff902570e0>] dev_queue_xmit+0x10/0x20
[<ffffffff902a507b>] ip_finish_output+0x67b/0x8d0
[<ffffffff902a55fb>] ip_output+0x7b/0xf0
[<ffffffff902a4a00>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff902a3027>] ip_local_out_sk+0x37/0x40
[<ffffffff902a33c4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff902bdad4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff902be15a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff902bef9e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff902ad01c>] tcp_push+0xec/0x120
[<ffffffff902b0a62>] tcp_sendmsg+0xd2/0xc60
[<ffffffff90239a18>] ? release_sock+0x128/0x180
[<ffffffff902dd1c9>] inet_sendmsg+0x69/0xb0
[<ffffffff8ff0c173>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff9023357d>] sock_aio_write+0x15d/0x180
[<ffffffff8fe4d243>] do_sync_write+0x93/0xe0
[<ffffffff8fe4de35>] vfs_write+0x1c5/0x1f0
[<ffffffff8fe4eb0f>] SyS_write+0x7f/0xf0
[<ffffffff90393f92>] system_call_fastpath+0x25/0x2a
Code: 0b 49 89 46 58 3e ff 80 80 00 00 00 48 89 da 4c 89 f6 4c 89 e7 e8 9f fd ff ff 48 89 04 24 48 8b 58 08 0f 1f 44 00 00 48 83 3b 00 <0f> 84 0b 02 00 00 48 c7 44 24 18 00 00 00 00 48 8b 73 20 4c 8d 
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:33:plane-0] flip_done timed out

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [sshd:109200]
Modules linked in: bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel ac97_bus lrw gf128mul glue_helper ablk_helper snd_seq snd_seq_device cryptd vmw_balloon snd_pcm pcspkr joydev snd_timer snd sg soundcore parport_pc parport vmw_vmci i2c_piix4 xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod vmwgfx crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common nfit crc32c_intel ata_piix libata serio_raw mptspi scsi_transport_spi e1000 mptscsih libnvdimm mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ip_tables]
CPU: 0 PID: 109200 Comm: sshd Tainted: G        W    L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff8ebc29478000 ti: ffff8ebc0e78c000 task.ti: ffff8ebc0e78c000
RIP: 0010:[<ffffffffc01bd877>]  [<ffffffffc01bd877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff8ebc0e78f830  EFLAGS: 00000286
RAX: ffffa39200283818 RBX: ffff8ebc0e78f83c RCX: 0000000000000058
RDX: 0000000000000058 RSI: ffff8ebc35cc4140 RDI: ffff8ebc38474000
RBP: ffff8ebc0e78f8b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffff8ebc0e78f818
R13: 000000000000000e R14: ffff8ebc39f6b000 R15: ffff8ebc2bf2def8
FS:  00007fe18ec6d8c0(0000) GS:ffff8ebc3c600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffc6087be88 CR3: 0000000012610000 CR4: 00000000003607f0
Call Trace:
[<ffffffff8e454386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8e48242a>] sch_direct_xmit+0x11a/0x250
[<ffffffff8e457301>] __dev_queue_xmit+0x491/0x650
[<ffffffff8e4574d0>] dev_queue_xmit+0x10/0x20
[<ffffffff8e4a545b>] ip_finish_output+0x67b/0x8d0
[<ffffffff8e4a59db>] ip_output+0x7b/0xf0
[<ffffffff8e4a4de0>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff8e4a3407>] ip_local_out_sk+0x37/0x40
[<ffffffff8e4a37a4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff8e4bde94>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff8e4be51a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff8e4bf35e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff8e4ad3fc>] tcp_push+0xec/0x120
[<ffffffff8e4b0e42>] tcp_sendmsg+0xd2/0xc60
[<ffffffff8e03e322>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff8e4dd589>] inet_sendmsg+0x69/0xb0
[<ffffffff8e10bf93>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff8e43357d>] sock_aio_write+0x15d/0x180
[<ffffffff8dff16bb>] ? do_wp_page+0xfb/0x720
[<ffffffff8e04d063>] do_sync_write+0x93/0xe0
[<ffffffff8e04dc55>] vfs_write+0x1c5/0x1f0
[<ffffffff8e04e92f>] SyS_write+0x7f/0xf0
[<ffffffff8e593f92>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [kubelet:5571]
Modules linked in: tcp_lp xt_nat nf_tables ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs ip6t_MASQUERADE nf_nat_masquerade_ipv6 xt_comment xt_mark nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod joydev ppdev vboxguest(OE) sg parport_pc parport pcspkr video i2c_piix4
binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi crc32c_intel serio_raw vmwgfx ahci drm_kms_helper libahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata e1000 drm_panel_orientation_quirks fuse
CPU: 2 PID: 5571 Comm: kubelet Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8af759ad9080 ti: ffff8af76bdd8000 task.ti: ffff8af76bdd8000
RIP: 0010:[<ffffffffc02c6877>]  [<ffffffffc02c6877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff8af76bddb830  EFLAGS: 00010286
RAX: ffffb69481bc3818 RBX: 0000000000000001 RCX: 0000000000000010
RDX: 0000000000000010 RSI: ffff8af8d06dc480 RDI: ffff8af7b923e000
RBP: ffff8af76bddb8b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000202
R13: 000000012aee58ce R14: 00000001bbee48b9 R15: ffffffffbc1b1d60
FS:  00007f301effd700(0000) GS:ffff8af8dfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e08e84a4c0 CR3: 00000000d7e54000 CR4: 00000000000006e0
Call Trace:
[<ffffffffbc455f96>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffbc4840aa>] sch_direct_xmit+0x11a/0x250
[<ffffffffbc458f11>] __dev_queue_xmit+0x491/0x650
[<ffffffffbc4590e0>] dev_queue_xmit+0x10/0x20
[<ffffffffbc4a7cdb>] ip_finish_output+0x67b/0x8d0
[<ffffffffbc4a825b>] ip_output+0x7b/0xf0
[<ffffffffbc4a7660>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffffbc4a5c87>] ip_local_out_sk+0x37/0x40
[<ffffffffbc4a6024>] ip_queue_xmit+0x144/0x3c0
[<ffffffffbc4c0764>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffbc4c0dea>] tcp_write_xmit+0x18a/0xd40
[<ffffffffbc4c1c2e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffbc4afc7c>] tcp_push+0xec/0x120
[<ffffffffbc4b36c2>] tcp_sendmsg+0xd2/0xc60
[<ffffffffbc43ba18>] ? release_sock+0x128/0x180
[<ffffffffbc4dfe79>] inet_sendmsg+0x69/0xb0
[<ffffffffbc10c403>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffffbc43557d>] sock_aio_write+0x15d/0x180
[<ffffffffbc04d593>] do_sync_write+0x93/0xe0
[<ffffffffbc04e185>] vfs_write+0x1c5/0x1f0
[<ffffffffbc595ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffbc04ee5f>] SyS_write+0x7f/0xf0
[<ffffffffbc595ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffbc595f92>] system_call_fastpath+0x25/0x2a
[<ffffffffbc595ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [vmtoolsd:778]
[<ffffffffaec55f96>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffaec840aa>] sch_direct_xmit+0x11a/0x250
[<ffffffffaec58f11>] __dev_queue_xmit+0x491/0x650
[<ffffffffaec590e0>] dev_queue_xmit+0x10/0x20
[<ffffffffaeca70eb>] ip_finish_output+0x67b/0x8d0
[<ffffffffaeca766b>] ip_output+0x7b/0xf0
[<ffffffffaeca6a70>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffffaeca5097>] ip_local_out_sk+0x37/0x40
[<ffffffffaeca5434>] ip_queue_xmit+0x144/0x3c0
[<ffffffffaecbfb44>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffaecc01ca>] tcp_write_xmit+0x18a/0xd40
[<ffffffffaecc100e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffaecaf08c>] tcp_push+0xec/0x120
[<ffffffffaecb2ad2>] tcp_sendmsg+0xd2/0xc60
[<ffffffffaec3ba18>] ? release_sock+0x128/0x180
[<ffffffffaecdf259>] inet_sendmsg+0x69/0xb0
[<ffffffffae90c2a3>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffffaec3557d>] sock_aio_write+0x15d/0x180
[<ffffffffae84d1e3>] do_sync_write+0x93/0xe0

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 33s! [httpd:3298]
Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter unix_diag vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 ppdev snd_rawmidi iosf_mbi snd_ac97_codec ac97_bus crc32_pclmul vmw_balloon snd_seq ghash_clmulni_intel aesni_intel snd_seq_device lrw gf128mul glue_helper snd_pcm ablk_helper cryptd snd_timer snd pcspkr joydev soundcore sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common
crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm ata_piix mptspi scsi_transport_spi drm mptscsih libata mptbase e1000 nfit libnvdimm drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 3298 Comm: httpd Kdump: loaded Tainted: G             L ------------   3.10.0-957.27.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff88d0b16b4100 ti: ffff88d0203fc000 task.ti: ffff88d0203fc000
RIP: 0010:[<ffffffffc02a4857>]  [<ffffffffc02a4857>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff88d0203ff7f8  EFLAGS: 00000286
RAX: ffffaf2e80883818 RBX: 0000000000000000 RCX: 0000000000000006
RDX: 0000000000000006 RSI: ffff88d0b1a43e00 RDI: ffff88d0b50a4000
RBP: ffff88d0203ff878 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000000
R13: ffff88d0b6c35800 R14: 0000000000000000 R15: 0000000000000000
FS:  00007f257f0a8700(0000) GS:ffff88d0be640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e55571e148 CR3: 00000001327ce000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8d03ace6>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8d067d6a>] sch_direct_xmit+0x11a/0x250
[<ffffffff8d067f2e>] __qdisc_run+0x8e/0x360
[<ffffffff8d03dc79>] __dev_queue_xmit+0x4a9/0x650
[<ffffffff8d03de30>] dev_queue_xmit+0x10/0x20
[<ffffffff8d08a7b6>] ip_finish_output+0x556/0x7b0
[<ffffffff8d08ad13>] ip_output+0x73/0xe0
[<ffffffff8d08a260>] ? __ip_append_data.isra.51+0xa50/0xa50
[<ffffffff8d088897>] ip_local_out_sk+0x37/0x40
[<ffffffff8d088c34>] ip_queue_xmit+0x144/0x3c0
[<ffffffff8d0a30b4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff8d0a3722>] tcp_write_xmit+0x172/0xd00
[<ffffffff8d0a452e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff8d0926bc>] tcp_push+0xec/0x120
[<ffffffff8d096112>] tcp_sendmsg+0xd2/0xc60
[<ffffffff8d169aba>] ? __schedule+0x42a/0x860
[<ffffffff8d0c2199>] inet_sendmsg+0x69/0xb0
[<ffffffff8cb0ef95>] ? futex_wait+0x1a5/0x280
[<ffffffff8d01a57d>] sock_aio_write+0x15d/0x180
[<ffffffff8cac6a30>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8cc41c13>] do_sync_write+0x93/0xe0
[<ffffffff8cc42805>] vfs_write+0x1c5/0x1f0
[<ffffffff8cc4351f>] SyS_write+0x7f/0xf0
[<ffffffff8d176ddb>] system_call_fastpath+0x22/0x27
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 46s! [sshd:16169]
Modules linked in: tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter bnep iptable_raw iptable_security xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat vmw_vsock_vmci_transport vmw_vmci nf_conntrack vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi ppdev crc32_pclmul ac97_bus snd_seq btusb btrtl btbcm btintel ghash_clmulni_intel snd_seq_device bluetooth aesni_intel snd_pcm lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_timer snd rfkill soundcore pcspkr joydev sg parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif
crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul crct10dif_common ttm crc32c_intel nfit drm libnvdimm ata_piix mptspi serio_raw libata scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 16169 Comm: sshd Kdump: loaded Not tainted 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff92ab7e10d230 ti: ffff92abbfc08000 task.ti: ffff92abbfc08000
RIP: 0010:[<ffffffffc02f7877>]  [<ffffffffc02f7877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff92abbfc0b7f0  EFLAGS: 00000286
RAX: ffffa5cfc0443818 RBX: ffff92abbfc0b7c0 RCX: 0000000000000012
RDX: 0000000000000012 RSI: ffff92ab763eb740 RDI: ffff92abb847c000
RBP: ffff92abbfc0b870 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffff92abbb61ad40
R13: ffffffffa34e28db R14: ffff92abbfc0b7c8 R15: 0000000000000046
FS:  00007f4a865b18c0(0000) GS:ffff92abbb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f587d3037d0 CR3: 0000000075f62000 CR4: 00000000003607f0
Call Trace:
[<ffffffffa3a53386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffa3a8131a>] sch_direct_xmit+0x11a/0x250
[<ffffffffa3a56301>] __dev_queue_xmit+0x491/0x650
[<ffffffffa34a4b17>] ? local_bh_enable+0x17/0x20
[<ffffffffa3a564d0>] dev_queue_xmit+0x10/0x20
[<ffffffffa3a6312d>] neigh_resolve_output+0x11d/0x220
[<ffffffffa3aa3e9c>] ip_finish_output+0x27c/0x8d0
[<ffffffffa3aa481b>] ip_output+0x7b/0xf0
[<ffffffffa3aa3c20>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffffa3aa2247>] ip_local_out_sk+0x37/0x40
[<ffffffffa3aa25e4>] ip_queue_xmit+0x144/0x3c0
[<ffffffffa3abccd4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffa3abd35a>] tcp_write_xmit+0x18a/0xd40
[<ffffffffa3abe19e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffa3aac23c>] tcp_push+0xec/0x120
[<ffffffffa3aafc82>] tcp_sendmsg+0xd2/0xc60
[<ffffffffa3a38a18>] ? release_sock+0x128/0x180
[<ffffffffa3adc389>] inet_sendmsg+0x69/0xb0
[<ffffffffa3adc530>] ? inet_recvmsg+0x80/0xb0
[<ffffffffa3a3257d>] sock_aio_write+0x15d/0x180
[<ffffffffa364c713>] do_sync_write+0x93/0xe0
[<ffffffffa364d305>] vfs_write+0x1c5/0x1f0
[<ffffffffa364dfcf>] SyS_write+0x7f/0xf0
[<ffffffffa3b92ed2>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 26s! [sshd:13522]
Modules linked in: xt_time nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ip6table_filter ip6_tables tcp_diag inet_diag ipt_REJECT nf_reject_ipv4 xt_owner xt_multiport iptable_filter ext4 mbcache jbd2 loop dm_mirror dm_region_hash dm_log dm_mod ppdev sg parport_pc parport pcspkr i2c_piix4 joydev ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_blk drm ata_piix e1000 libata virtio_pci serio_raw virtio_ring virtio drm_panel_orientation_quirks floppy
CPU: 3 PID: 13522 Comm: sshd Kdump: loaded Tainted: G             L ------------   3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff9f4914f98000 ti: ffff9f4569608000 task.ti: ffff9f4569608000
RIP: 0010:[<ffffffffc03d9877>]  [<ffffffffc03d9877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff9f456960b798  EFLAGS: 00000286
RAX: ffffb6d3c1c03818 RBX: 000000000000003c RCX: 000000000000000d
RDX: 000000000000000d RSI: ffff9f45369abdc0 RDI: ffff9f49151fc000
RBP: ffff9f456960b818 R08: 0000000000000000 R09: 0000000002000000
R10: 0000000000000000 R11: 0000000000000000 R12: 00000004156ad802
R13: 0000000100000000 R14: ffffffff9b5acd80 R15: ffff9f456960b750
FS:  00007f2c15eee8c0(0000) GS:ffff9f4927cc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2c1302ecfb CR3: 00000003e07f0000 CR4: 00000000000006e0
Call Trace:
[<ffffffff9b84f346>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff9b87d0ba>] sch_direct_xmit+0x11a/0x250
[<ffffffff9b87d27e>] __qdisc_run+0x8e/0x360
[<ffffffff9b8522c9>] __dev_queue_xmit+0x4a9/0x650
[<ffffffff9b852480>] dev_queue_xmit+0x10/0x20
[<ffffffff9b85efed>] neigh_resolve_output+0x11d/0x220
[<ffffffff9b8a057c>] ip_finish_output+0x27c/0x8d0
[<ffffffff9b8a0efb>] ip_output+0x7b/0xf0
[<ffffffff9b8a0300>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff9b89e927>] ip_local_out_sk+0x37/0x40
[<ffffffff9b89ecc4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff9b8b9374>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff9b8b99fa>] tcp_write_xmit+0x18a/0xd40
[<ffffffff9b8ba82e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff9b8a88ec>] tcp_push+0xec/0x120
[<ffffffff9b8ac332>] tcp_sendmsg+0xd2/0xc60
[<ffffffffc058675c>] ? xfs_filemap_fault+0x2c/0x30 [xfs]
[<ffffffff9b8d89b9>] inet_sendmsg+0x69/0xb0
[<ffffffff9b82e57d>] sock_aio_write+0x15d/0x180
[<ffffffff9b3eda74>] ? tlb_flush_mmu_free+0x34/0x50
[<ffffffff9b449d03>] do_sync_write+0x93/0xe0
[<ffffffff9b44a8f5>] vfs_write+0x1c5/0x1f0
[<ffffffff9b98de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9b44b60f>] SyS_write+0x7f/0xf0
[<ffffffff9b98de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9b98dede>] system_call_fastpath+0x25/0x2a
[<ffffffff9b98de21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 26s! [sshd:2295]
Modules linked in: binfmt_misc nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore ppdev coretemp pcspkr joydev vmw_balloon sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi mptscsih e1000 crc32c_intel mptbase serio_raw drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2295 Comm: sshd Kdump: loaded Tainted: G               ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff8c1cf9fb62a0 ti: ffff8c1ccdc34000 task.ti: ffff8c1ccdc34000
RIP: 0010:[<ffffffffc030a877>]  [<ffffffffc030a877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff8c1ccdc37830  EFLAGS: 00000286
RAX: ffffb6eb40703818 RBX: ffff8c1cfb6d3940 RCX: 00000000000000b7
RDX: 00000000000000b7 RSI: ffff8c1c759d8bc0 RDI: ffff8c1cfa910000
RBP: ffff8c1ccdc378b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffff8c1ccdc37878
R13: ffffffffab6e2e89 R14: ffff8c1ccdc377c8 R15: ffffffffab6e69a1
FS:  00007f9cb15778c0(0000) GS:ffff8c1cfb6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558642f15210 CR3: 0000000097dbc000 CR4: 00000000000007e0
Call Trace:
[<ffffffffabc53386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffabc8131a>] sch_direct_xmit+0x11a/0x250
[<ffffffffabc56301>] __dev_queue_xmit+0x491/0x650
[<ffffffffabc564d0>] dev_queue_xmit+0x10/0x20
[<ffffffffabca429b>] ip_finish_output+0x67b/0x8d0
[<ffffffffabca481b>] ip_output+0x7b/0xf0
[<ffffffffabca3c20>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffffabca2247>] ip_local_out_sk+0x37/0x40
[<ffffffffabca25e4>] ip_queue_xmit+0x144/0x3c0
[<ffffffffabcbccd4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffabcbd35a>] tcp_write_xmit+0x18a/0xd40
[<ffffffffabcbe19e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffabcac23c>] tcp_push+0xec/0x120
[<ffffffffabcafc82>] tcp_sendmsg+0xd2/0xc60
[<ffffffffab6d3c52>] ? __wake_up_common+0x82/0x120
[<ffffffffabcdc389>] inet_sendmsg+0x69/0xb0
[<ffffffffabc3257d>] sock_aio_write+0x15d/0x180
[<ffffffffab84c713>] do_sync_write+0x93/0xe0
[<ffffffffab84d305>] vfs_write+0x1c5/0x1f0
[<ffffffffabd92e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffab84dfcf>] SyS_write+0x7f/0xf0
[<ffffffffabd92e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffabd92ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffabd92e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 31s! [java:23869]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper joydev vmw_balloon sg cryptd pcspkr parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel nfit libnvdimm libata mptspi serio_raw
scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 23869 Comm: java Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff89c1fb84c1c0 ti: ffff89c1c6d08000 task.ti: ffff89c1c6d08000
RIP: 0010:[<ffffffffc01c1877>]  [<ffffffffc01c1877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff89c1c6d0b830  EFLAGS: 00000286
RAX: ffffb24b00283818 RBX: ffffffff88103bc0 RCX: 0000000000000068
RDX: 0000000000000068 RSI: ffff89c1f56c9140 RDI: ffff89c1f9e66000
RBP: ffff89c1c6d0b8b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffff89c1c6d0b878
R13: ffff89c1f9e66000 R14: ffff89c1f56c9140 R15: 00000000000000c3
FS:  00007f9d448f0700(0000) GS:ffff89c1fc600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb517d61de0 CR3: 000000000851c000 CR4: 00000000003607f0
Call Trace:
[<ffffffff87653386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8768131a>] sch_direct_xmit+0x11a/0x250
[<ffffffff87656301>] __dev_queue_xmit+0x491/0x650
[<ffffffff876564d0>] dev_queue_xmit+0x10/0x20
[<ffffffff876a429b>] ip_finish_output+0x67b/0x8d0
[<ffffffff876a481b>] ip_output+0x7b/0xf0
[<ffffffff876a3c20>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff876a2247>] ip_local_out_sk+0x37/0x40
[<ffffffff876a25e4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff876bccc4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff876bd34a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff876be18e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff876ac23c>] tcp_push+0xec/0x120
[<ffffffff876afc82>] tcp_sendmsg+0xd2/0xc60
[<ffffffff87785922>] ? __schedule+0x402/0x840
[<ffffffff876dc379>] inet_sendmsg+0x69/0xb0
[<ffffffff8730b283>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff8763257d>] sock_aio_write+0x15d/0x180
[<ffffffff8724c663>] do_sync_write+0x93/0xe0
[<ffffffff8724d255>] vfs_write+0x1c5/0x1f0
[<ffffffff8724df1f>] SyS_write+0x7f/0xf0
[<ffffffff87792ed2>] system_call_fastpath+0x25/0x2a
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [auditbeat:25137]
Modules linked in: xt_REDIRECT nf_nat_redirect ip_vs_rr xt_ipvs ip_vs xt_nat veth vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) snd_hda_codec_generic ext4 mbcache jbd2 snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq iosf_mbi crc32_pclmul snd_seq_device ghash_clmulni_intel ppdev joydev snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr snd_timer virtio_balloon snd virtio_rng parport_pc parport i2c_piix4 soundcore ip_tables xfs libcrc32c ata_generic pata_acpi virtio_console virtio_blk qxl drm_kms_helper ata_piix syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm libata crct10dif_pclmul crct10dif_common drm crc32c_intel serio_raw floppy e1000 virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 25137 Comm: auditbeat Tainted: G               ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.11.0-2.el7 04/01/2014
task: ffff91dc967d9070 ti: ffff91dcc3ebc000 task.ti: ffff91dcc3ebc000
RIP: 0010:[<ffffffffc0228877>]  [<ffffffffc0228877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff91dcc3ebf830  EFLAGS: 00000286
RAX: ffffabc280f03818 RBX: 0000000000000001 RCX: 00000000000000e7
RDX: 00000000000000e7 RSI: ffff91dd334ab000 RDI: ffff91dd35e64000
RBP: ffff91dcc3ebf8b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000202
R13: 00000002021a60ce R14: 00000001c3ebf7a0 R15: ffffffffaf3b0870
FS:  00007f70e67fc700(0000) GS:ffff91dd3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7f5e6ed4a0 CR3: 000000016bd20000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaf653386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffffaf68131a>] sch_direct_xmit+0x11a/0x250
[<ffffffffaf656301>] __dev_queue_xmit+0x491/0x650
[<ffffffffaf6564d0>] dev_queue_xmit+0x10/0x20
[<ffffffffaf6a429b>] ip_finish_output+0x67b/0x8d0
[<ffffffffaf6a481b>] ip_output+0x7b/0xf0
[<ffffffffaf6a3c20>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffffaf6a2247>] ip_local_out_sk+0x37/0x40
[<ffffffffaf6a25e4>] ip_queue_xmit+0x144/0x3c0
[<ffffffffaf6bccd4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffffaf6bd35a>] tcp_write_xmit+0x18a/0xd40
[<ffffffffaf6be19e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffffaf6ac23c>] tcp_push+0xec/0x120
[<ffffffffaf6afc82>] tcp_sendmsg+0xd2/0xc60
[<ffffffffaf796b14>] ? ftrace_regs_call+0x5/0x81
[<ffffffffaf6dc389>] inet_sendmsg+0x69/0xb0
[<ffffffffaf30b303>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffffaf63257d>] sock_aio_write+0x15d/0x180
[<ffffffffaf24c713>] do_sync_write+0x93/0xe0
[<ffffffffaf24d305>] vfs_write+0x1c5/0x1f0
[<ffffffffaf24dfcf>] SyS_write+0x7f/0xf0
[<ffffffffaf793166>] tracesys+0xa6/0xcc
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sendmail:14728]
Modules linked in: dm_mirror dm_region_hash dm_log dm_mod ppdev parport_pc sg pcspkr joydev parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm e1000 serio_raw virtio_pci virtio_ring ata_piix virtio libata drm_panel_orientation_quirks floppy
CPU: 1 PID: 14728 Comm: sendmail Kdump: loaded Not tainted 3.10.0-957.27.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org 04/01/2014
task: ffff9d492f4930c0 ti: ffff9d48bf4dc000 task.ti: ffff9d48bf4dc000
RIP: 0010:[<ffffffffc03a4857>]  [<ffffffffc03a4857>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff9d48bf4df850  EFLAGS: 00000286
RAX: ffffbb4c40e83818 RBX: 0000000000000001 RCX: 00000000000000f7
RDX: 00000000000000f7 RSI: ffff9d47b6964dc0 RDI: ffff9d49aa854000
RBP: ffff9d48bf4df8d0 R08: 0000000000000100 R09: 0000000022100000
R10: 000045000888f7b7 R11: 0000000000000000 R12: 0000000000000202
R13: 00000001d16140da R14: 0000000100000001 R15: ffffffff8fba23e0
FS:  00007f3b5e48a840(0000) GS:ffff9d49b3d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3b5dbd001c CR3: 000000011dac4000 CR4: 00000000000006e0
Call Trace:
[<ffffffff8fe3ace6>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8fe67d6a>] sch_direct_xmit+0x11a/0x250
[<ffffffff8fe3dc61>] __dev_queue_xmit+0x491/0x650
[<ffffffff8f9d9778>] ? zone_statistics+0x88/0xa0
[<ffffffff8fe3de30>] dev_queue_xmit+0x10/0x20
[<ffffffff8fe8a7b6>] ip_finish_output+0x556/0x7b0
[<ffffffff8fe8ad13>] ip_output+0x73/0xe0
[<ffffffff8fe88897>] ip_local_out_sk+0x37/0x40
[<ffffffff8fe88c34>] ip_queue_xmit+0x144/0x3c0
[<ffffffff8fea30b4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff8fea3722>] tcp_write_xmit+0x172/0xd00
[<ffffffff8fea452e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff8fe926bc>] tcp_push+0xec/0x120
[<ffffffff8fe96112>] tcp_sendmsg+0xd2/0xc60
[<ffffffff8fec2199>] inet_sendmsg+0x69/0xb0
[<ffffffff8fe1a57d>] sock_aio_write+0x15d/0x180
[<ffffffff8f9ea874>] ? handle_pte_fault+0x2f4/0xd10
[<ffffffff8fa41c13>] do_sync_write+0x93/0xe0
[<ffffffff8fa42805>] vfs_write+0x1c5/0x1f0
[<ffffffff8ff76d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8fa4351f>] SyS_write+0x7f/0xf0
[<ffffffff8ff76d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8ff76ddb>] system_call_fastpath+0x22/0x27
[<ffffffff8ff76d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [java:3751]
Modules linked in: nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr sg joydev nfit vmw_balloon libnvdimm parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_piix vmwgfx drm_kms_helper libata serio_raw syscopyarea mptspi sysfillrect sysimgblt fb_sys_fops ttm e1000
scsi_transport_spi drm mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3751 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff93d7de441fa0 ti: ffff93d8d664c000 task.ti: ffff93d8d664c000
RIP: 0010:[<ffffffffc0449867>]  [<ffffffffc0449867>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffff93d8d664f860  EFLAGS: 00000286
RAX: ffffb438c0d03818 RBX: ffffffff8ecbfc00 RCX: 00000000000000fd
RDX: 00000000000000fd RSI: ffff93d8f3335980 RDI: ffff93d8bd622000
RBP: ffff93d8d664f8e0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffffffffc086b74d
R13: ffff93d8d664f8a0 R14: 00000001f76ac098 R15: ffffffff8df73e00
FS:  00007fb21adba700(0000) GS:ffff93d8f7600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb2317f9438 CR3: 00000001a0bba000 CR4: 00000000003607f0
Call Trace:
[<ffffffff8e1eb166>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8e216a2a>] sch_direct_xmit+0x11a/0x250
[<ffffffff8e1ee051>] __dev_queue_xmit+0x4a1/0x660
[<ffffffff8e1ee220>] dev_queue_xmit+0x10/0x20
[<ffffffff8e237416>] ip_finish_output+0x546/0x7a0
[<ffffffff8e237973>] ip_output+0x73/0xe0
[<ffffffff8e236ed0>] ? __ip_append_data.isra.50+0xa50/0xa50
[<ffffffff8e235567>] ip_local_out_sk+0x37/0x40
[<ffffffff8e2358d3>] ip_queue_xmit+0x143/0x3a0
[<ffffffff8e24f7b4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff8e24fe22>] tcp_write_xmit+0x172/0xd00
[<ffffffff8e250c2e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff8e23f12c>] tcp_push+0xec/0x120
[<ffffffff8e242ae2>] tcp_sendmsg+0xd2/0xc80
[<ffffffff8e26e5a9>] inet_sendmsg+0x69/0xb0
[<ffffffff8dd04265>] ? futex_wait+0x1a5/0x280
[<ffffffff8e1cb84d>] sock_aio_write+0x15d/0x180
[<ffffffff8dcbe870>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8de1a1f3>] do_sync_write+0x93/0xe0
[<ffffffff8de1add5>] vfs_write+0x1c5/0x1f0
[<ffffffff8de1baff>] SyS_write+0x7f/0xf0
[<ffffffff8e31f7d5>] system_call_fastpath+0x1c/0x21
Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 95s! [etcd:34402]
INFO: rcu_sched detected stalls on CPUs/tasks:
Modules linked in: xt_nat veth ip_vs_nq ip_set_hash_ipportnet ip_set_hash_ipport ip_set_hash_ipportip ip_set_bitmap_port dummy ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink br_netfilter bridge stp llc ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs overlay(T) xt_set ip_set nfnetlink xt_addrtype iptable_nat nf_nat_ipv4 nf_nat xt_comment nf_conntrack_ipv4 nf_defrag_ipv4 xt_mark xt_conntrack nf_conntrack vmw_vsock_vmci_transport vsock iptable_filter iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul
crct10dif_common crc32c_intel ata_piix serio_raw nfit libata e1000 mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 34402 Comm: etcd Kdump: loaded Tainted: G               ------------ T 3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffffa060e5b8b150 ti: ffffa060d8bf4000 task.ti: ffffa060d8bf4000
RIP: 0010:[<ffffffffc03d4877>]  [<ffffffffc03d4877>] e1000_xmit_frame+0x7f7/0x1120 [e1000]
RSP: 0018:ffffa060d8bf7830  EFLAGS: 00000286
RAX: ffffc05d40783818 RBX: ffffffff89103bc0 RCX: 00000000000000da
RDX: 00000000000000da RSI: ffffa060b677cbc0 RDI: ffffa060f848e000
RBP: ffffa060d8bf78b0 R08: 0000000000000100 R09: 0000000022100000
R10: 0000000080000000 R11: 0000000000000000 R12: ffffa060d8bf7878
R13: ffffffff88785830 R14: ffffffff88785824 R15: ffffffff88785830
FS:  000000c420027068(0000) GS:ffffa060fb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f086722f420 CR3: 000000003ce44000 CR4: 00000000003607f0
Call Trace:
[<ffffffff88653386>] dev_hard_start_xmit+0x246/0x3b0
[<ffffffff8868131a>] sch_direct_xmit+0x11a/0x250
[<ffffffff88656301>] __dev_queue_xmit+0x491/0x650
[<ffffffff886564d0>] dev_queue_xmit+0x10/0x20
[<ffffffff886a429b>] ip_finish_output+0x67b/0x8d0
[<ffffffff886a481b>] ip_output+0x7b/0xf0
[<ffffffff886a3c20>] ? __ip_append_data.isra.50+0xa60/0xa60
[<ffffffff886a2247>] ip_local_out_sk+0x37/0x40
[<ffffffff886a25e4>] ip_queue_xmit+0x144/0x3c0
[<ffffffff886bccc4>] tcp_transmit_skb+0x4e4/0x9e0
[<ffffffff886bd34a>] tcp_write_xmit+0x18a/0xd40
[<ffffffff886be18e>] __tcp_push_pending_frames+0x2e/0xc0
[<ffffffff886ac23c>] tcp_push+0xec/0x120
[<ffffffff886afc82>] tcp_sendmsg+0xd2/0xc60
[<ffffffff88638a18>] ? release_sock+0x128/0x180
[<ffffffff886dc379>] inet_sendmsg+0x69/0xb0
[<ffffffff8830b283>] ? selinux_socket_sendmsg+0x23/0x30
[<ffffffff8863257d>] sock_aio_write+0x15d/0x180
[<ffffffff8824c663>] do_sync_write+0x93/0xe0
[<ffffffff8824d255>] vfs_write+0x1c5/0x1f0
[<ffffffff8824df1f>] SyS_write+0x7f/0xf0
[<ffffffff88792ed2>] system_call_fastpath+0x25/0x2a