| Operating system | Count |
|---|---|
| CentOS 7 | 47 |
| Architecture | Count |
|---|---|
| x86_64 | 47 |
| Related packages | Count |
|---|---|
| kernel | 47 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [ksoftirqd/0:6] Modules linked in: tcp_diag udp_diag inet_diag tcp_lp nls_utf8 isofs tun devlink bridge stp llc ip_set vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw snd_ens1371 gf128mul snd_rawmidi glue_helper ablk_helper snd_ac97_codec vmw_balloon cryptd ac97_bus snd_seq snd_seq_device joydev snd_pcm pcspkr snd_timer snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ttm crct10dif_common nfit mptspi crc32c_intel ata_piix scsi_transport_spi drm libnvdimm libata e1000 serio_raw mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ip_tables] CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Not tainted 3.10.0-1160.119.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020 task: ffff9aa3f65b5280 ti: ffff9aa3f6678000 task.ti: ffff9aa3f6678000 RIP: 0010:[<ffffffffc00b38e7>] [<ffffffffc00b38e7>] e1000_xmit_frame+0x7f7/0x1120 [e1000] RSP: 0018:ffff9aa3f667b550 EFLAGS: 00000286 RAX: ffffb4b2c0703818 RBX: 0000000000000001 RCX: 0000000000000061 RDX: 0000000000000061 RSI: ffff9aa2f5538f80 RDI: ffff9aa3f43a0000 RBP: ffff9aa3f667b5d0 R08: 0000000000000100 R09: 0000000022100000 R10: 0000000080000000 R11: 0000000000000000 R12: 0000000000000202 R13: 0000000130e334ce R14: 00000001f667b540 R15: ffffffffbc1c9150 FS: 0000000000000000(0000) GS:ffff9aa3f9600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000008c6d86a2320 CR3: 0000000105c1c000 CR4: 0000000000760ff0 PKRU: 00000000 Call Trace: [<ffffffffbc47edc6>] dev_hard_start_xmit+0x246/0x3b0 [<ffffffffbc4add72>] sch_direct_xmit+0x122/0x250 [<ffffffffbc481dd1>] __dev_queue_xmit+0x491/0x650 [<ffffffffbbedd37f>] ? ttwu_do_activate+0x6f/0x80 [<ffffffffbc481fa0>] dev_queue_xmit+0x10/0x20 [<ffffffffbc4d260b>] ip_finish_output+0x67b/0x8d0 [<ffffffffbc4d2b9b>] ip_output+0x7b/0x100 [<ffffffffbc0ad1e8>] ? ep_poll_callback+0xf8/0x220 [<ffffffffbc4d0587>] ip_local_out_sk+0x37/0x40 [<ffffffffbc4d0924>] ip_queue_xmit+0x144/0x3d0 [<ffffffffbc4eb5f4>] tcp_transmit_skb+0x4e4/0x9e0 [<ffffffffbc4ed83f>] tcp_send_ack+0x11f/0x180 [<ffffffffbc4e1c6b>] __tcp_ack_snd_check+0x6b/0xa0 [<ffffffffbc4e7f85>] tcp_rcv_established+0x425/0x7a0 [<ffffffffbc4f30f2>] tcp_v4_do_rcv+0x112/0x360 [<ffffffffbc4f498c>] tcp_v4_rcv+0x7dc/0x9e0 [<ffffffffbc4f30f2>] ? tcp_v4_do_rcv+0x112/0x360 [<ffffffffbc4cc34d>] ip_local_deliver_finish+0xbd/0x210 [<ffffffffbc4cc660>] ip_local_deliver+0x60/0xe0 [<ffffffffbc4f4162>] ? tcp_v4_early_demux+0x122/0x170 [<ffffffffbc4cbfa0>] ip_rcv_finish+0x90/0x380 [<ffffffffbc4cc9a0>] ip_rcv+0x2c0/0x420 [<ffffffffbc47f669>] __netif_receive_skb_core+0x739/0xa20 [<ffffffffbc4f9b00>] ? tcp4_gro_receive+0x30/0x1a0 [<ffffffffbbf0eabf>] ? __getnstimeofday64+0x3f/0xd0 [<ffffffffbc47f968>] __netif_receive_skb+0x18/0x70 [<ffffffffbc47fa00>] netif_receive_skb_internal+0x40/0xc0 [<ffffffffbc4806b8>] napi_gro_receive+0xd8/0x100 [<ffffffffc00b44cb>] e1000_clean_rx_irq+0x2bb/0x510 [e1000] [<ffffffffc00b5180>] e1000_clean+0x280/0x8e0 [e1000] [<ffffffffbc480037>] net_rx_action+0x277/0x3a0 [<ffffffffbbea9595>] __do_softirq+0xf5/0x290 [<ffffffffbbea9768>] run_ksoftirqd+0x38/0x60 [<ffffffffbbed43fc>] smpboot_thread_fn+0x14c/0x1b0 [<ffffffffbbed42b0>] ? lg_double_unlock+0x40/0x40 [<ffffffffbbecb621>] kthread+0xd1/0xe0 [<ffffffffbbecb550>] ? insert_kthread_work+0x40/0x40 [<ffffffffbc5c51dd>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffffbbecb550>] ? insert_kthread_work+0x40/0x40 Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 25s! [ksoftirqd/8:49] Modules linked in: tcp_lp rfcomm binfmt_misc nfnetlink_queue nfnetlink_log br_netfilter tun devlink overlay(T) bridge stp llc ip_set nfnetlink bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod kvm_intel kvm snd_seq_midi btusb snd_seq_midi_event irqbypass crc32_pclmul btrtl btbcm ghash_clmulni_intel aesni_intel btintel bluetooth lrw snd_ens1371 gf128mul glue_helper ablk_helper rfkill snd_rawmidi cryptd snd_ac97_codec sg pcspkr joydev ppdev ac97_bus snd_seq parport_pc vmw_balloon parport snd_seq_device snd_pcm snd_timer snd soundcore vmw_vmci i2c_piix4 xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul nfit crct10dif_common crc32c_intel libnvdimm libata mptspi e1000 scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks fuse [last unloaded: ip_tables] CPU: 8 PID: 49 Comm: ksoftirqd/8 Kdump: loaded Tainted: G L ------------ T 3.10.0-1160.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020 task: ffff97c11fbe8000 ti: ffff97c11fbe4000 task.ti: ffff97c11fbe4000 RIP: 0010:[<ffffffffc01ce877>] [<ffffffffc01ce877>] e1000_xmit_frame+0x7f7/0x1120 [e1000] RSP: 0018:ffff97c11fbe7550 EFLAGS: 00000286 RAX: ffffaa2a41c83818 RBX: 0000000000000001 RCX: 00000000000000bc RDX: 00000000000000bc RSI: ffff97c3fada1140 RDI: ffff97c11ff88000 RBP: ffff97c11fbe75d0 R08: 0000000000000100 R09: 0000000022100000 R10: 77a4280008450008 R11: 0000000000000000 R12: 0000000000000202 R13: 00000003b76e14da R14: 000000011ff88000 R15: ffffffffa7bb1650 FS: 0000000000000000(0000) GS:ffff97c3fd800000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fff3c5dbf55 CR3: 0000000034fac000 CR4: 00000000007607e0 PKRU: 00000000 Call Trace: [<ffffffffa7e3c6f7>] ? kfree_skbmem+0x37/0x90 [<ffffffffa7e54386>] dev_hard_start_xmit+0x246/0x3b0 [<ffffffffa7e8242a>] sch_direct_xmit+0x11a/0x250 [<ffffffffa7e57301>] __dev_queue_xmit+0x491/0x650 [<ffffffffa7e574d0>] dev_queue_xmit+0x10/0x20 [<ffffffffa7ea545b>] ip_finish_output+0x67b/0x8d0 [<ffffffffa7ea59db>] ip_output+0x7b/0xf0 [<ffffffffa78d2fd2>] ? __wake_up_common+0x82/0x120 [<ffffffffa7ea3407>] ip_local_out_sk+0x37/0x40 [<ffffffffa7ea37a4>] ip_queue_xmit+0x144/0x3c0 [<ffffffffa7ebde94>] tcp_transmit_skb+0x4e4/0x9e0 [<ffffffffa7ec00af>] tcp_send_ack+0x11f/0x170 [<ffffffffa7eb4c7b>] __tcp_ack_snd_check+0x6b/0x90 [<ffffffffa7eba7a4>] tcp_rcv_established+0x264/0x770 [<ffffffffa7ec57aa>] tcp_v4_do_rcv+0x10a/0x350 [<ffffffffa7ec6ffc>] tcp_v4_rcv+0x7dc/0x9e0 [<ffffffffa7e9f28d>] ip_local_deliver_finish+0xbd/0x200 [<ffffffffa7e9f580>] ip_local_deliver+0x60/0xe0 [<ffffffffa7ec67d2>] ? tcp_v4_early_demux+0x112/0x160 [<ffffffffa7e9eef0>] ip_rcv_finish+0x90/0x370 [<ffffffffa7e9f8c0>] ip_rcv+0x2c0/0x420 [<ffffffffa7f5f902>] ? packet_rcv+0x32/0x430 [<ffffffffa7e9ee60>] ? inet_del_offload+0x40/0x40 [<ffffffffa7e54c19>] __netif_receive_skb_core+0x729/0xa10 [<ffffffffa79078df>] ? __getnstimeofday64+0x3f/0xd0 [<ffffffffa7e54f18>] __netif_receive_skb+0x18/0x60 [<ffffffffa7e54fa0>] netif_receive_skb_internal+0x40/0xc0 [<ffffffffa7e55c28>] napi_gro_receive+0xd8/0x100 [<ffffffffc01cf45b>] e1000_clean_rx_irq+0x2bb/0x510 [e1000] [<ffffffffc01d0100>] e1000_clean+0x280/0x8e0 [e1000] [<ffffffffa78e43cc>] ? update_curr+0x14c/0x1e0 [<ffffffffa7e555bf>] net_rx_action+0x26f/0x390 [<ffffffffa78a4b95>] __do_softirq+0xf5/0x280 [<ffffffffa78a4d58>] run_ksoftirqd+0x38/0x50 [<ffffffffa78ce674>] smpboot_thread_fn+0x144/0x1a0 [<ffffffffa78ce530>] ? lg_double_unlock+0x40/0x40 [<ffffffffa78c5c21>] kthread+0xd1/0xe0 [<ffffffffa78c5b50>] ? insert_kthread_work+0x40/0x40 [<ffffffffa7f93ddd>] ret_from_fork_nospec_begin+0x7/0x21 [<ffffffffa78c5b50>] ? insert_kthread_work+0x40/0x40 Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [khugepaged:48] [<ffffffff90a8242a>] sch_direct_xmit+0x11a/0x250 [<ffffffff90a57301>] __dev_queue_xmit+0x491/0x650 [<ffffffff90a574d0>] dev_queue_xmit+0x10/0x20 [<ffffffff90aa545b>] ip_finish_output+0x67b/0x8d0 [<ffffffff90aa59db>] ip_output+0x7b/0xf0 [<ffffffff90aa4de0>] ? __ip_append_data.isra.50+0xa60/0xa60 [<ffffffff90aa3407>] ip_local_out_sk+0x37/0x40 [<ffffffff90aa37a4>] ip_queue_xmit+0x144/0x3c0 [<ffffffff90abde94>] tcp_transmit_skb+0x4e4/0x9e0 [<ffffffff90ac00af>] tcp_send_ack+0x11f/0x170 [<ffffffff90ab4c7b>] __tcp_ack_snd_check+0x6b/0x90 [<ffffffff90aba7a4>] tcp_rcv_established+0x264/0x770 [<ffffffff90ac57aa>] tcp_v4_do_rcv+0x10a/0x350 [<ffffffff90ac6ffc>] tcp_v4_rcv+0x7dc/0x9e0 [<ffffffff905e0468>] ? zone_statistics+0x88/0xa0 [<ffffffff90a9f28d>] ip_local_deliver_finish+0xbd/0x200 [<ffffffff90a9f580>] ip_local_deliver+0x60/0xe0 [<ffffffff90ac67d2>] ? tcp_v4_early_demux+0x112/0x160 [<ffffffff90a9eef0>] ip_rcv_finish+0x90/0x370 [<ffffffff90a9f8c0>] ip_rcv+0x2c0/0x420 [<ffffffff90a54c19>] __netif_receive_skb_core+0x729/0xa10 [<ffffffff90acc100>] ? tcp4_gro_receive+0x80/0x1a0 [<ffffffff905078df>] ? __getnstimeofday64+0x3f/0xd0 [<ffffffff90a54f18>] __netif_receive_skb+0x18/0x60 [<ffffffff90a54fa0>] netif_receive_skb_internal+0x40/0xc0 [<ffffffff90a55c28>] napi_gro_receive+0xd8/0x100 [<ffffffffc047245b>] e1000_clean_rx_irq+0x2bb/0x510 [e1000] [<ffffffffc0473100>] e1000_clean+0x280/0x8e0 [e1000] [<ffffffff904e43cc>] ? update_curr+0x14c/0x1e0 [<ffffffff90a555bf>] net_rx_action+0x26f/0x390 [<ffffffff904a4b95>] __do_softirq+0xf5/0x280 [<ffffffff904a4d58>] run_ksoftirqd+0x38/0x50 [<ffffffff904ce674>] smpboot_thread_fn+0x144/0x1a0 [<ffffffff904ce530>] ? lg_double_unlock+0x40/0x40 [<ffffffff904c5c21>] kthread+0xd1/0xe0 [<ffffffff904c5b50>] ? insert_kthread_work+0x40/0x40 [<ffffffff90b93df7>] ret_from_fork_nospec_begin+0x21/0x21 [<ffffffff904c5b50>] ? insert_kthread_work+0x40/0x40 Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [ksoftirqd/3:24] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel nls_utf8 isofs snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev ac97_bus snd_seq snd_seq_device vmw_balloon snd_pcm snd_timer snd joydev pcspkr sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata crct10dif_pclmul crct10dif_common crc32c_intel nfit drm libnvdimm serio_raw mptspi scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod CPU: 3 PID: 24 Comm: ksoftirqd/3 Kdump: loaded Not tainted 3.10.0-1127.13.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018 task: ffff8cb4fb0cb150 ti: ffff8cb4fb0d8000 task.ti: ffff8cb4fb0d8000 RIP: 0010:[<ffffffffc0262877>] [<ffffffffc0262877>] e1000_xmit_frame+0x7f7/0x1120 [e1000] RSP: 0018:ffff8cb4fb0db4f8 EFLAGS: 00000286 RAX: ffffae2f41983818 RBX: 0000000000000042 RCX: 00000000000000a5 RDX: 00000000000000a5 RSI: ffff8cb62e653100 RDI: ffff8cb632106000 RBP: ffff8cb4fb0db578 R08: 0000000000000100 R09: 0000000022100000 R10: 34000045000870b5 R11: 0000000000000000 R12: 00000001cd7becce R13: 0000000100000000 R14: ffffffffadfb0830 R15: ffff8cb4fb0db4b0 FS: 0000000000000000(0000) GS:ffff8cb6336c0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000011156a8 CR3: 000000007ea10000 CR4: 00000000000607e0 Call Trace: [<ffffffffae253386>] dev_hard_start_xmit+0x246/0x3b0 [<ffffffffae28131a>] sch_direct_xmit+0x11a/0x250 [<ffffffffae2814de>] __qdisc_run+0x8e/0x360 [<ffffffffae256319>] __dev_queue_xmit+0x4a9/0x650 [<ffffffffae2564d0>] dev_queue_xmit+0x10/0x20 [<ffffffffae2a429b>] ip_finish_output+0x67b/0x8d0 [<ffffffffae2a481b>] ip_output+0x7b/0xf0 [<ffffffffae2a3c20>] ? __ip_append_data.isra.50+0xa60/0xa60 [<ffffffffae2a2247>] ip_local_out_sk+0x37/0x40 [<ffffffffae2a25e4>] ip_queue_xmit+0x144/0x3c0 [<ffffffffae2bccd4>] tcp_transmit_skb+0x4e4/0x9e0 [<ffffffffae2beedf>] tcp_send_ack+0x11f/0x170 [<ffffffffae2b3abb>] __tcp_ack_snd_check+0x6b/0x90 [<ffffffffae2b9797>] tcp_rcv_established+0x417/0x770 [<ffffffffae2c46ca>] tcp_v4_do_rcv+0x10a/0x350 [<ffffffffae2c5ddc>] tcp_v4_rcv+0x7dc/0x9e0 [<ffffffffc07e32ce>] ? ipv4_confirm+0x4e/0x100 [nf_conntrack_ipv4] [<ffffffffae29e0fd>] ip_local_deliver_finish+0xbd/0x200 [<ffffffffae29e3f0>] ip_local_deliver+0x60/0xe0 [<ffffffffae29e040>] ? ip_rcv_finish+0x370/0x370 [<ffffffffae29dd60>] ip_rcv_finish+0x90/0x370 [<ffffffffae29e730>] ip_rcv+0x2c0/0x420 [<ffffffffae29dcd0>] ? inet_del_offload+0x40/0x40 [<ffffffffae253c19>] __netif_receive_skb_core+0x729/0xa10 [<ffffffffae2caf00>] ? tcp4_gro_receive+0xa0/0x1a0 [<ffffffffadd0825f>] ? __getnstimeofday64+0x3f/0xd0 [<ffffffffae253f18>] __netif_receive_skb+0x18/0x60 [<ffffffffae253fa0>] netif_receive_skb_internal+0x40/0xc0 [<ffffffffae254c28>] napi_gro_receive+0xd8/0x100 [<ffffffffc026345b>] e1000_clean_rx_irq+0x2bb/0x510 [e1000] [<ffffffffc0264100>] e1000_clean+0x280/0x8e0 [e1000] [<ffffffffae10efa0>] ? usb_hcd_poll_rh_status+0x180/0x180 [<ffffffffae2545bf>] net_rx_action+0x26f/0x390 [<ffffffffadca5695>] __do_softirq+0xf5/0x280 [<ffffffffadca5858>] run_ksoftirqd+0x38/0x50 [<ffffffffadccf0e4>] smpboot_thread_fn+0x144/0x1a0 [<ffffffffadccefa0>] ? lg_double_unlock+0x40/0x40 [<ffffffffadcc6691>] kthread+0xd1/0xe0 [<ffffffffadcc65c0>] ? insert_kthread_work+0x40/0x40 [<ffffffffae392d37>] ret_from_fork_nospec_begin+0x21/0x21 [<ffffffffadcc65c0>] ? insert_kthread_work+0x40/0x40 Code: 00 00 20 74 10 48 8b 83 00 03 00 00 f6 80 98 00 00 00 03 74 16 48 8b 5d c8 41 0f b7 47 2a 41 8b 57 18 48 03 83 90 0c 00 00 89 10 <48> 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 84 00
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 41s! [ksoftirqd/7:44] Modules linked in: udp_diag tcp_diag inet_diag binfmt_misc vmw_vsock_vmci_transport vsock sunrpc vfat fat sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd sg vmw_balloon pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul mptspi crct10dif_common scsi_transport_spi crc32c_intel mptscsih nfit serio_raw mptbase libnvdimm vmxnet3 drm_panel_orientation_quirks floppy CPU: 7 PID: 44 Comm: ksoftirqd/7 Kdump: loaded Tainted: G W L ------------ 3.10.0-1127.18.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018 task: ffff8baff9d01070 ti: ffff8baff9d0c000 task.ti: ffff8baff9d0c000 RIP: 0010:[<ffffffffc01773dc>] [<ffffffffc01773dc>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0000:ffff8baff9d0f5e0 EFLAGS: 00000282 RAX: 0000000000000000 RBX: 0101000006ff9202 RCX: 000000000000011e RDX: ffffaaa4418b0638 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff8baff9d0f670 R08: 65770a0501013a65 R09: 0000000000c8404e R10: 0101000012ff9205 R11: ffff8bb2addf31d0 R12: ffff8baff9d0f5f0 R13: ffff8baff9d0f5d0 R14: 0000000000000283 R15: ffff8bb291facc00 FS: 0000000000000000(0000) GS:ffff8bb2bfdc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f59b39f9018 CR3: 00000000ba926000 CR4: 00000000000407e0 Call Trace: [<ffffffff8dc53386>] dev_hard_start_xmit+0x246/0x3b0 [<ffffffff8dc8131a>] sch_direct_xmit+0x11a/0x250 [<ffffffff8dc56301>] __dev_queue_xmit+0x491/0x650 [<ffffffff8dc564d0>] dev_queue_xmit+0x10/0x20 [<ffffffff8dca429b>] ip_finish_output+0x67b/0x8d0 [<ffffffff8dca481b>] ip_output+0x7b/0xf0 [<ffffffff8d6d3c52>] ? __wake_up_common+0x82/0x120 [<ffffffff8d825fe0>] ? __slab_free+0x1b0/0x290 [<ffffffff8dca2247>] ip_local_out_sk+0x37/0x40 [<ffffffff8dca25e4>] ip_queue_xmit+0x144/0x3c0 [<ffffffff8dcbccd4>] tcp_transmit_skb+0x4e4/0x9e0 [<ffffffff8dcbeedf>] tcp_send_ack+0x11f/0x170 [<ffffffff8dcb3abb>] __tcp_ack_snd_check+0x6b/0x90 [<ffffffff8dcb95e4>] tcp_rcv_established+0x264/0x770 [<ffffffff8dcc46ca>] tcp_v4_do_rcv+0x10a/0x350 [<ffffffff8dcc5ddc>] tcp_v4_rcv+0x7dc/0x9e0 [<ffffffff8d7c5efa>] ? __rmqueue+0x8a/0x460 [<ffffffff8d7e05c8>] ? zone_statistics+0x88/0xa0 [<ffffffff8dc9e0fd>] ip_local_deliver_finish+0xbd/0x200 [<ffffffff8dc9e3f0>] ip_local_deliver+0x60/0xe0 [<ffffffff8dcc55b2>] ? tcp_v4_early_demux+0x112/0x160 [<ffffffff8dc9dd60>] ip_rcv_finish+0x90/0x370 [<ffffffff8dc9e730>] ip_rcv+0x2c0/0x420 [<ffffffff8dc53c19>] __netif_receive_skb_core+0x729/0xa10 [<ffffffff8dc53f18>] __netif_receive_skb+0x18/0x60 [<ffffffff8dc53fa0>] netif_receive_skb_internal+0x40/0xc0 [<ffffffff8dc5403c>] netif_receive_skb+0x1c/0x70 [<ffffffffc0178639>] vmxnet3_rq_rx_complete+0x8a9/0xef0 [vmxnet3] [<ffffffffc01790e6>] vmxnet3_poll_rx_only+0x36/0xa0 [vmxnet3] [<ffffffff8dc545bf>] net_rx_action+0x26f/0x390 [<ffffffff8d6a5695>] __do_softirq+0xf5/0x280 [<ffffffff8d6a5858>] run_ksoftirqd+0x38/0x50 [<ffffffff8d6cf0e4>] smpboot_thread_fn+0x144/0x1a0 [<ffffffff8d6cefa0>] ? lg_double_unlock+0x40/0x40 [<ffffffff8d6c6691>] kthread+0xd1/0xe0 [<ffffffff8d6c65c0>] ? insert_kthread_work+0x40/0x40 [<ffffffff8dd92d37>] ret_from_fork_nospec_begin+0x21/0x21 [<ffffffff8d6c65c0>] ? insert_kthread_work+0x40/0x40 Code: ff c7 02 00 00 00 00 41 8b 94 24 e8 00 00 00 48 8b 4d d0 8d 14 d5 00 06 00 00 48 63 d2 48 03 91 b8 23 00 00 41 8b 4c 24 34 89 0a <e9> d0 f7 ff ff 0f 1f 80 00 00 00 00 48 c7 45 c8 00 40 00 00 41
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ksoftirqd/0:6] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter tcp_diag inet_diag vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul mptspi crct10dif_common scsi_transport_spi crc32c_intel mptscsih libata serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G L ------------ 3.10.0-1062.9.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013 task: ffff8da6274eb150 ti: ffff8da627500000 task.ti: ffff8da627500000 RIP: 0010:[<ffffffffc02233dc>] [<ffffffffc02233dc>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0018:ffff8da6275035e0 EFLAGS: 00000282 RAX: 0000000000000000 RBX: ffff8da627503638 RCX: 0000000000000197 RDX: ffffb7bb43180608 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff8da627503670 R08: 0000000000000050 R09: 0000000000c84042 R10: 3400004500082443 R11: ffff8da61b037960 R12: ffff8da627503628 R13: ffff8da67a146d90 R14: ffffffffa9a24a99 R15: ffffffffc069cf8d FS: 0000000000000000(0000) GS:ffff8da628800000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f41d10e03b8 CR3: 0000000305610000 CR4: 00000000000407f0 Call Trace: [<ffffffffc0320719>] ? ipt_do_table+0x359/0x740 [ip_tables] [<ffffffffa9e4f346>] dev_hard_start_xmit+0x246/0x3b0 [<ffffffffa9e7d0ba>] sch_direct_xmit+0x11a/0x250 [<ffffffffa9e522b1>] __dev_queue_xmit+0x491/0x650 [<ffffffffa9e52480>] dev_queue_xmit+0x10/0x20 [<ffffffffa9ea097b>] ip_finish_output+0x67b/0x8d0 [<ffffffffa9ea0efb>] ip_output+0x7b/0xf0 [<ffffffffa9ea0300>] ? __ip_append_data.isra.50+0xa60/0xa60 [<ffffffffa9e9e927>] ip_local_out_sk+0x37/0x40 [<ffffffffa9e9ecc4>] ip_queue_xmit+0x144/0x3c0 [<ffffffffa9eb9374>] tcp_transmit_skb+0x4e4/0x9e0 [<ffffffffa9ebb56f>] tcp_send_ack+0x11f/0x170 [<ffffffffa9eb016b>] __tcp_ack_snd_check+0x6b/0x90 [<ffffffffa9eb5e37>] tcp_rcv_established+0x417/0x770 [<ffffffffa9ec0d5a>] tcp_v4_do_rcv+0x10a/0x350 [<ffffffffa9ec246c>] tcp_v4_rcv+0x7dc/0x9e0 [<ffffffffc06b72ce>] ? ipv4_confirm+0x4e/0x100 [nf_conntrack_ipv4] [<ffffffffa9e9a86d>] ip_local_deliver_finish+0xbd/0x200 [<ffffffffa9e9ab60>] ip_local_deliver+0x60/0xe0 [<ffffffffa9e9a7b0>] ? ip_rcv_finish+0x370/0x370 [<ffffffffa9e9a4d0>] ip_rcv_finish+0x90/0x370 [<ffffffffa9e9aea0>] ip_rcv+0x2c0/0x420 [<ffffffffa9e9a440>] ? inet_del_offload+0x40/0x40 [<ffffffffa9e4fbd9>] __netif_receive_skb_core+0x729/0xa10 [<ffffffffa9ec7600>] ? tcp4_gro_receive+0x110/0x1a0 [<ffffffffa99076bf>] ? __getnstimeofday64+0x3f/0xd0 [<ffffffffa9e4fed8>] __netif_receive_skb+0x18/0x60 [<ffffffffa9e4ff60>] netif_receive_skb_internal+0x40/0xc0 [<ffffffffa9e50be8>] napi_gro_receive+0xd8/0x100 [<ffffffffc02246b4>] vmxnet3_rq_rx_complete+0x924/0xef0 [vmxnet3] [<ffffffffc02250e6>] vmxnet3_poll_rx_only+0x36/0xa0 [vmxnet3] [<ffffffffa9e5057f>] net_rx_action+0x26f/0x390 [<ffffffffa98a5305>] __do_softirq+0xf5/0x280 [<ffffffffa98a54c8>] run_ksoftirqd+0x38/0x50 [<ffffffffa98cec24>] smpboot_thread_fn+0x144/0x1a0 [<ffffffffa98ceae0>] ? lg_double_unlock+0x40/0x40 [<ffffffffa98c61f1>] kthread+0xd1/0xe0 [<ffffffffa98c6120>] ? insert_kthread_work+0x40/0x40 [<ffffffffa9f8dd37>] ret_from_fork_nospec_begin+0x21/0x21 [<ffffffffa98c6120>] ? insert_kthread_work+0x40/0x40 Code: ff c7 02 00 00 00 00 41 8b 94 24 e8 00 00 00 48 8b 4d d0 8d 14 d5 00 06 00 00 48 63 d2 48 03 91 b8 23 00 00 41 8b 4c 24 34 89 0a <e9> d0 f7 ff ff 0f 1f 80 00 00 00 00 48 c7 45 c8 00 40 00 00 41
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:3] Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun lp ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus ppdev snd_seq snd_seq_device pcspkr snd_pcm snd_timer vmw_balloon snd soundcore joydev nfit sg libnvdimm parport_pc parport i2c_piix4 shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt crc32c_intel fb_sys_fops ttm drm serio_raw ata_piix mptspi scsi_transport_spi mptscsih libata mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 3 Comm: ksoftirqd/0 Tainted: G OE ------------ 3.10.0-693.2.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020 task: ffff88017a251fa0 ti: ffff88017a360000 task.ti: ffff88017a360000 RIP: 0010:[<ffffffffc00554dd>] [<ffffffffc00554dd>] vmxnet3_xmit_frame+0xb5d/0xed0 [vmxnet3] RSP: 0018:ffff88017a363640 EFLAGS: 00000286 RAX: 0000000000000000 RBX: ffffffff81e87180 RCX: 0000000000000095 RDX: ffffc900018b4600 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff88017a3636d0 R08: 0c00bc92fc565000 R09: 0000000000c84036 R10: ffff88041a76c940 R11: 0040298528001045 R12: ffffffff81e87180 R13: 0000000000000000 R14: ffffffffc068374d R15: ffff88017a363670 FS: 0000000000000000(0000) GS:ffff88042d600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fac7d76b2e6 CR3: 0000000420efb000 CR4: 00000000003407f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Stack: ffff880428b20000 ffff88017a363658 ffffffff815710d7 ffff88017a363678 ffff88041a76c940 0000000000000283 ffff880428b208e0 000088017a3636d0 0000002200000014 ffff8800a74bc800 0000000000000036 ffff880400000002 Call Trace: [<ffffffff815710d7>] ? kfree_skbmem+0x37/0x90 [<ffffffff81586771>] dev_hard_start_xmit+0x171/0x3b0 [<ffffffff815af52a>] sch_direct_xmit+0x11a/0x240 [<ffffffff81589566>] __dev_queue_xmit+0x226/0x550 [<ffffffff815898a0>] dev_queue_xmit+0x10/0x20 [<ffffffff815cfd5e>] ip_finish_output+0x52e/0x780 [<ffffffff815d02b3>] ip_output+0x73/0xe0 [<ffffffff815cf830>] ? __ip_append_data.isra.48+0xa00/0xa00 [<ffffffff815cdf51>] ip_local_out_sk+0x31/0x40 [<ffffffff815ce2c3>] ip_queue_xmit+0x143/0x3a0 [<ffffffff815e7c2f>] tcp_transmit_skb+0x4cf/0x9b0 [<ffffffff815e9c4c>] tcp_send_ack+0x11c/0x170 [<ffffffff815defcb>] __tcp_ack_snd_check+0x6b/0xa0 [<ffffffff815e46b4>] tcp_rcv_established+0x264/0x760 [<ffffffff815ef17a>] tcp_v4_do_rcv+0x10a/0x340 [<ffffffff815f08c8>] tcp_v4_rcv+0x798/0x9a0 [<ffffffffc06b4306>] ? ipv4_confirm+0x86/0x100 [nf_conntrack_ipv4] [<ffffffff815ca074>] ip_local_deliver_finish+0xb4/0x1f0 [<ffffffff815ca359>] ip_local_deliver+0x59/0xd0 [<ffffffff815c9fc0>] ? ip_rcv_finish+0x350/0x350 [<ffffffff815c9cfa>] ip_rcv_finish+0x8a/0x350 [<ffffffff815ca686>] ip_rcv+0x2b6/0x410 [<ffffffff815c9c70>] ? inet_del_offload+0x40/0x40 [<ffffffff81586f22>] __netif_receive_skb_core+0x572/0x7c0 [<ffffffff810ea8ca>] ? __getnstimeofday64+0x3a/0xd0 [<ffffffff81587188>] __netif_receive_skb+0x18/0x60 [<ffffffff81587210>] netif_receive_skb_internal+0x40/0xc0 [<ffffffff81588318>] napi_gro_receive+0xd8/0x130 [<ffffffffc0053fac>] vmxnet3_rq_rx_complete+0x82c/0xc20 [vmxnet3] [<ffffffffc005450a>] vmxnet3_poll_rx_only+0x3a/0xb0 [vmxnet3] [<ffffffff8158799d>] net_rx_action+0x16d/0x380 [<ffffffff81090b3f>] __do_softirq+0xef/0x280 [<ffffffff81090d08>] run_ksoftirqd+0x38/0x50 [<ffffffff810b909f>] smpboot_thread_fn+0x12f/0x180 [<ffffffff810b8f70>] ? lg_double_unlock+0x40/0x40 [<ffffffff810b098f>] kthread+0xcf/0xe0 [<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40 [<ffffffff816b4f58>] ret_from_fork+0x58/0x90 [<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40 Code: f7 ff ff c7 02 00 00 00 00 41 8b 95 e8 00 00 00 48 8b 4d d0 8d 14 d5 00 06 00 00 48 63 d2 48 03 91 b8 21 00 00 41 8b 4d 34 89 0a <e9> 6b f7 ff ff 48 8b 5d 80 e9 9a fc ff ff 0f b7 b3 c0 00 00 00