NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kubelet:6316]
Modules linked in: ipt_REJECT nf_reject_ipv4 xt_multiport ipt_rpfilter xt_set iptable_raw ip_set_hash_ip ip_set_hash_net ip_set vxlan ip6_udp_tunnel udp_tunnel ip6t_MASQUERADE nf_nat_masquerade_ipv6 xt_mark xt_comment ip6table_filter ip6table_mangle iptable_mangle ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables nf_tables veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq sb_edac iosf_mbi btusb crc32_pclmul snd_seq_device ghash_clmulni_intel btrtl btbcm snd_pcm aesni_intel
lrw btintel gf128mul snd_timer glue_helper snd ablk_helper bluetooth soundcore cryptd ppdev joydev vmw_vmci i2c_piix4 rfkill sg pcspkr vmw_balloon parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common nfit crc32c_intel libata nvme serio_raw libnvdimm e1000 nvme_core drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod synosnap(OE)
CPU: 2 PID: 6316 Comm: kubelet Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.95.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff92131019b180 ti: ffff921309ae4000 task.ti: ffff921309ae4000
RIP: 0010:[<ffffffffb5f1eacd>]  [<ffffffffb5f1eacd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff921309ae7dc0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff921309ae7d98 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffdca48c40cc70
RBP: ffff921309ae7dc0 R08: 0000000000000000 R09: ffffdca493f318c0
R10: ffff921309ae7e20 R11: ffff92134b2c2f30 R12: ffff921176198c40
R13: 0000000000000001 R14: ffff92134b3a0c00 R15: ffffdca493f31880
FS:  00007f124affd640(0000) GS:ffff921601e80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c002e6d000 CR3: 0000000310388000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb65ac21a>] queued_spin_lock_slowpath+0xb/0x13
[<ffffffffb65ba734>] _raw_spin_lock+0x24/0x30
[<ffffffffb600241e>] handle_mm_fault+0xd3e/0x1190
[<ffffffffb65bf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb65bfb05>] do_page_fault+0x35/0x90
[<ffffffffb65bb7b8>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 cc 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [x:4981]
Modules linked in: xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) binfmt_misc vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi snd_ens1371 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper snd_rawmidi snd_ac97_codec ac97_bus cryptd snd_seq ppdev snd_seq_device vmw_balloon snd_pcm snd_timer nfit pcspkr joydev sg snd soundcore libnvdimm parport_pc parport i2c_piix4 shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic
pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul crct10dif_common sysimgblt fb_sys_fops ttm crc32c_intel ata_piix drm mptspi scsi_transport_spi libata serio_raw mptscsih mptbase e1000 i2c_core floppy
CPU: 3 PID: 4981 Comm: x Tainted: G             L ------------ T 3.10.0-693.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff88000f238000 ti: ffff88000fa64000 task.ti: ffff88000fa64000
RIP: 0010:[<ffffffff810fbf42>]  [<ffffffff810fbf42>] native_queued_spin_lock_slowpath+0x22/0x1e0
RSP: 0000:ffff88000fa67e00  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffea00003454f0
RBP: ffff88000fa67e00 R08: 000000000000008f R09: 3e03bc38819e5761
R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000080000
R13: 0000000000000001 R14: 0000000000000258 R15: 0000000000000077
FS:  00007f8fafb94b30(0000) GS:ffff88042e6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f903648f6c0 CR3: 000000012b9a2000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff816a070f>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff816ade20>] _raw_spin_lock+0x20/0x30
[<ffffffff811b3f5a>] handle_mm_fault+0x81a/0xfa0
[<ffffffff810cf411>] ? update_curr+0x71/0x190
[<ffffffff816b37e4>] __do_page_fault+0x154/0x450
[<ffffffff816b3b15>] do_page_fault+0x35/0x90
[<ffffffff816af8f8>] page_fault+0x28/0x30
Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 ba 01 00 00 00 8b 07 85 c0 75 0a f0 0f b1 17 85 c0 75 f2 5d c3 f3 90 <eb> ec 0f 1f 40 00 81 fe 00 01 00 00 66 90 0f 84 5c 01 00 00 41 

NMI watchdog: BUG: soft lockup - CPU#8 stuck for 21s! [java:19940]
Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 tun xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype br_netfilter bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter nfnetlink_queue nfnetlink_log bluetooth rfkill tampercore(OE) vmw_vsock_vmci_transport vsock overlay(T) snd_seq_midi snd_seq_midi_event snd_ens1371 iosf_mbi crc32_pclmul snd_rawmidi snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq aesni_intel lrw gf128mul snd_seq_device ppdev glue_helper ablk_helper vmw_balloon cryptd snd_pcm joydev snd_timer pcspkr snd sg soundcore i2c_piix4 vmw_vmci dm_mirror dm_region_hash dm_log dm_mod parport_pc parport
nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix nfit scsi_transport_spi drm mptscsih libata libnvdimm crct10dif_pclmul crct10dif_common mptbase crc32c_intel e1000 serio_raw drm_panel_orientation_quirks fuse
CPU: 8 PID: 19940 Comm: java Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff90c66de0e300 ti: ffff90c4f019c000 task.ti: ffff90c4f019c000
RIP: 0010:[<ffffffff99317acd>]  [<ffffffff99317acd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff90c4f019fdc0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff90c4f019fd98 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: fffffce9792caaf0
RBP: ffff90c4f019fdc0 R08: 0000000000000000 R09: fffffce979187640
R10: ffff90c4f019fe20 R11: ffff90c4e9222e58 R12: ffff90b7e3ac3b00
R13: 0000000000000001 R14: ffff90bec06dd800 R15: fffffce979187600
FS:  00007f27805cb700(0000) GS:ffff90c67d800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f277ee7c300 CR3: 0000000e656bc000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff99980d63>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9998ed00>] _raw_spin_lock+0x20/0x30
[<ffffffff993f6903>] handle_mm_fault+0xd83/0xfb0
[<ffffffff9927dff5>] ? switch_mm_irqs_off+0x115/0x290
[<ffffffff99994653>] __do_page_fault+0x213/0x500
[<ffffffff99994975>] do_page_fault+0x35/0x90
[<ffffffff99990778>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 48s! [VM Periodic Tas:15434]
Modules linked in: veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) overlay(T) sunrpc snd_seq_midi snd_seq_midi_event sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm joydev snd_timer snd pcspkr sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic
pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ata_piix mptspi libata scsi_transport_spi e1000 nfit mptscsih libnvdimm mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 15434 Comm: VM Periodic Tas Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8b76c7f01040 ti: ffff8b76b8320000 task.ti: ffff8b76b8320000
RIP: 0010:[<ffffffff9f511fff>]  [<ffffffff9f511fff>] native_queued_spin_lock_slowpath+0x1f/0x200
RSP: 0018:ffff8b76b8323d48  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffefb40fe5e730
RBP: ffff8b76b8323d48 R08: ffff8b76b979cda8 R09: 0000000000000001
R10: ffffefb402cd1640 R11: 00003ffffffff000 R12: 00007f7b943b5000
R13: 0000000000000001 R14: ffff8b76000000a9 R15: 0000000000000002
FS:  00007f7b41efb700(0000) GS:ffff8b76ed600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555abeaeaeb0 CR3: 00000003f8ba4000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9fb5bf4b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9fb6a400>] _raw_spin_lock+0x20/0x30
[<ffffffff9f5e8fa0>] handle_pte_fault+0x160/0xd10
[<ffffffff9fb67bc9>] ? schedule+0x29/0x70
[<ffffffff9f5ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9fb6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9fb6f915>] do_page_fault+0x35/0x90
[<ffffffff9fb6b758>] page_fault+0x28/0x30
Code: ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 8b 17 <85> d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 1f 80 

NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [java:24163]
Modules linked in: binfmt_misc vmw_vsock_vmci_transport vsock sunrpc vfat fat ppdev crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper sg cryptd vmw_vmci pcspkr joydev parport_pc parport i2c_piix4 tpm_crb ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic e1000e crct10dif_pclmul crct10dif_common vmwgfx crc32c_intel serio_raw mptsas ptp pps_core scsi_transport_sas drm_kms_helper mptscsih mptbase syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm ata_piix nfit libata libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 8 PID: 24163 Comm: java Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.17369862.B64.2012240522 12/24/2020
task: ffff924df4b8d140 ti: ffff92520164c000 task.ti: ffff92520164c000
RIP: 0010:[<ffffffff9e911ffd>]  [<ffffffff9e911ffd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff92520164fd48  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff92520164fd18 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: fffff14356c7a6f0
RBP: ffff92520164fd48 R08: ffff925231e9b5c8 R09: 0000000000000001
R10: 0000000000000001 R11: 00003ffffffff000 R12: ffff92520164fd00
R13: 0000000000000001 R14: 0000000000000001 R15: ffff92549f25ab80
FS:  00007fe4bd1a9700(0000) GS:ffff92549f400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc92ecb7000 CR3: 00000005886c8000 CR4: 0000000000760fe0
PKRU: 55555554
Call Trace:
[<ffffffff9ef5bf4b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9ef6a400>] _raw_spin_lock+0x20/0x30
[<ffffffff9e9e8fa0>] handle_pte_fault+0x160/0xd10
[<ffffffff9ef67bc9>] ? schedule+0x29/0x70
[<ffffffff9e9ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9ef6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9ef6f915>] do_page_fault+0x35/0x90
[<ffffffff9ef6b758>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 51s! [VM Thread:3394]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi snd_ens1371 crc32_pclmul snd_rawmidi snd_ac97_codec ghash_clmulni_intel btusb ac97_bus btrtl
btbcm btintel aesni_intel snd_seq bluetooth ppdev lrw snd_seq_device gf128mul snd_pcm glue_helper vmw_balloon ablk_helper snd_timer cryptd snd rfkill joydev soundcore pcspkr vmw_vmci sg i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic crc_t10dif pata_acpi crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ttm ata_piix crct10dif_common mptspi nfit crc32c_intel drm libata libnvdimm scsi_transport_spi serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks synosnap(OE) fuse
CPU: 3 PID: 3394 Comm: VM Thread Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9451573a0000 ti: ffff9450aab4c000 task.ti: ffff9450aab4c000
RIP: 0010:[<ffffffffae517acd>]  [<ffffffffae517acd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff9450aab4fdc0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffe4c89daba970
RBP: ffff9450aab4fdc0 R08: ffff9450a9caa580 R09: 0000000000000001
R10: ffff9450aab4fe20 R11: ffff9450ad8ad878 R12: ffffb2cd032aa840
R13: 00007f257c4c0b54 R14: 0000000000000001 R15: ffff9450aab4fde8
FS:  00007f252014c700(0000) GS:ffff94515d6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f258474b600 CR3: 000000074dc36000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaeb80d63>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffaeb8ed00>] _raw_spin_lock+0x20/0x30
[<ffffffffae5f64b5>] handle_mm_fault+0x935/0xfb0
[<ffffffffaeb94653>] __do_page_fault+0x213/0x500
[<ffffffffaeb94975>] do_page_fault+0x35/0x90
[<ffffffffaeb90778>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#18 stuck for 49s! [java:28070]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13109(E) falcon_lsm_pinned_13005(E) falcon_lsm_pinned_12905(E) binfmt_misc cmac arc4 md4 nls_utf8 cifs ccm dns_resolver ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment xt_mark iptable_filter iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_netlink nfnetlink iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi overlay(T) vmw_vsock_vmci_transport vsock falcon_lsm_pinned_12803(E) ext4 sb_edac mbcache iosf_mbi jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper pcc_cpufreq cryptd ppdev vmw_balloon pcspkr joydev parport_pc parport vmw_vmci sg i2c_piix4 br_netfilter bridge stp
llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi nfit crct10dif_pclmul libata crct10dif_common crc32c_intel mptscsih libnvdimm serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 18 PID: 28070 Comm: java Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff89080f05b150 ti: ffff8904274b4000 task.ti: ffff8904274b4000
RIP: 0010:[<ffffffff8311757f>]  [<ffffffff8311757f>] native_queued_spin_lock_slowpath+0x1f/0x200
RSP: 0018:ffff8904274b7dc0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffc4febfeab7f0
RBP: ffff8904274b7dc0 R08: ffff89033b5e9f40 R09: 0000000000000001
R10: ffff8904274b7e20 R11: ffff890926757bd8 R12: 0000000000000000
R13: 00007f85a35e8000 R14: 0000000000000000 R15: ffff8904000000a9
FS:  00007f707477a700(0000) GS:ffff89133f280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c001279078 CR3: 000000029ce8a000 CR4: 00000000001607e0
Call Trace:
[<ffffffff837754ee>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff83783b20>] _raw_spin_lock+0x20/0x30
[<ffffffff831f3ae5>] handle_mm_fault+0x935/0xfb0
[<ffffffff83788653>] __do_page_fault+0x213/0x500
[<ffffffff83788975>] do_page_fault+0x35/0x90
[<ffffffff83784778>] page_fault+0x28/0x30
Code: ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 8b 17 <85> d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 1f 80 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [C2 CompilerThre:123448]
Modules linked in: veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache overlay nf_conntrack_netbios_ns nf_conntrack_broadcast xt_nat xt_mark ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash
dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul snd_ens1371 glue_helper snd_rawmidi ablk_helper snd_ac97_codec ppdev cryptd ac97_bus snd_seq joydev snd_seq_device snd_pcm vmw_balloon pcspkr snd_timer snd soundcore sg vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sch_fq_codel sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi nfit ata_piix crct10dif_pclmul crct10dif_common scsi_transport_spi libata mptscsih crc32c_intel libnvdimm serio_raw drm_panel_orientation_quirks e1000 mptbase fuse
CPU: 0 PID: 123448 Comm: C2 CompilerThre Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff8c19486bd280 ti: ffff8c174315c000 task.ti: ffff8c174315c000
RIP: 0010:[<ffffffff94f177ad>]  [<ffffffff94f177ad>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff8c174315fdc0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffd2ecc851dcb0
RBP: ffff8c174315fdc0 R08: ffff8c196db1d780 R09: 0000000000000001
R10: ffff8c174315fe20 R11: ffff8c1845eac288 R12: ffffffff94ed757f
R13: ffff8c174315fd50 R14: 0000000000000000 R15: ffff8c197561acc0
FS:  00007f23fc157700(0000) GS:ffff8c1975600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2415646208 CR3: 000000020943e000 CR4: 00000000003607f0
Call Trace:
[<ffffffff9557d155>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9558b870>] _raw_spin_lock+0x20/0x30
[<ffffffff94ff5d95>] handle_mm_fault+0x935/0xfb0
[<ffffffff95590653>] __do_page_fault+0x213/0x500
[<ffffffff95590975>] do_page_fault+0x35/0x90
[<ffffffff9558c778>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kube-apiserver:27426]
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kube-apiserver:27442]
Modules linked in: xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 tun ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_conntrack ebtable_filter ebtables ip6table_filter ip6_tables devlink xt_addrtype xt_set xt_mark ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set nfnetlink dummy xt_comment iptable_nat nf_nat_ipv4 nf_nat overlay(T) nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 parport_pc parport vmw_vmci br_netfilter
bridge stp llc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel mptscsih libata serio_raw nfit mptbase libnvdimm vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 27442 Comm: kube-apiserver Tainted: G             L ------------ T 3.10.0-957.27.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9647cef75140 ti: ffff964713658000 task.ti: ffff964713658000
RIP: 0010:[<ffffffffa3d1342d>]  [<ffffffffa3d1342d>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff96471365bd48  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffee0b8fa1d230
RBP: ffff96471365bd48 R08: 0000000000000000 R09: ffffee0b84c12c80
R10: 00000000000043b5 R11: 00003ffffffff000 R12: 0000000000000001
R13: ffff9645d7349000 R14: ffffee0b84c12c40 R15: ffff9647a723d140
FS:  000000c007f6e410(0000) GS:ffff9647ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbe1057480c CR3: 00000003b23a2000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa435e2cb>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa436c7a0>] _raw_spin_lock+0x20/0x30
[<ffffffffa3deaf57>] handle_pte_fault+0x9d7/0xd10
[<ffffffffa3ded3ad>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa4371603>] __do_page_fault+0x203/0x4f0
[<ffffffffa4371925>] do_page_fault+0x35/0x90
[<ffffffffa436d768>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [java:2287]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev pcspkr joydev cryptd sg vmw_balloon parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt libata fb_sys_fops ttm nfit libnvdimm drm e1000 mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common crc32c_intel mptscsih serio_raw mptbase drm_panel_orientation_quirks
dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 2287 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9da5ee63c1c0 ti: ffff9da5e5d0c000 task.ti: ffff9da5e5d0c000
RIP: 0010:[<ffffffffacf17ecd>]  [<ffffffffacf17ecd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff9da5e5d0fd60  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffcd0682defdf0
RBP: ffff9da5e5d0fd60 R08: ffffcd0682defdc0 R09: ffffcd068230629c
R10: ffff9da5e5d0fe20 R11: ffff9da634e22f30 R12: 0000000000000002
R13: ffff9da5e5d0fda8 R14: 000000001c9c2bf8 R15: ffff9da5e5d0fcf8
FS:  00007f0b1530d700(0000) GS:ffff9da63a600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0b667813b0 CR3: 00000000a7430000 CR4: 00000000003607f0
Call Trace:
[<ffffffffad57a004>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffad5886b0>] _raw_spin_lock+0x20/0x30
[<ffffffffacfee558>] do_read_fault.isra.63+0x108/0x1b0
[<ffffffffacff5d00>] handle_mm_fault+0xa20/0xfb0
[<ffffffffad58d653>] __do_page_fault+0x213/0x500
[<ffffffffad58d975>] do_page_fault+0x35/0x90
[<ffffffffad589778>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 27s! [java:61355]
Modules linked in: tcp_lp fuse tun ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac edac_core snd_ens1371 iosf_mbi crc32_pclmul snd_rawmidi snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq aesni_intel lrw snd_seq_device gf128mul glue_helper snd_pcm ablk_helper vmw_balloon joydev pcspkr ppdev cryptd snd_timer i2c_piix4 vmw_vmci sg snd soundcore shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi drm ata_piix crct10dif_pclmul e1000 crct10dif_common mptscsih libata i2c_core serio_raw crc32c_intel mptbase dm_mirror dm_region_hash
dm_log dm_mod [last unloaded: ip_tables]
CPU: 4 PID: 61355 Comm: java Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff880381b22f70 ti: ffff8803bd7a8000 task.ti: ffff8803bd7a8000
RIP: 0010:[<ffffffff810fa232>]  [<ffffffff810fa232>] native_queued_spin_lock_slowpath+0x22/0x1e0
RSP: 0000:ffff8803bd7abe00  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00005a656c05d737 RCX: 8000000000000825
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffea000e29a7f0
RBP: ffff8803bd7abe00 R08: 000000000000015a R09: 0000000000018452
R10: 0000000000000000 R11: 800000040502a825 R12: ffff88042d74fe80
R13: ffff8803bd7abd68 R14: ffffc90001bc0800 R15: 00007fdc602eb754
FS:  00007fdc397d7700(0000) GS:ffff88042d700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdc6a55a420 CR3: 00000003fc608000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8803bd7abe10 ffffffff8169e5ff ffff8803bd7abe20 ffffffff816abb50
ffff8803bd7abeb8 ffffffff811b1ced ffff880428c95780 0000000000000728
00000000005906ca 00007fdc602eb728 8000000000000825 800000040502a825
Call Trace:
[<ffffffff8169e5ff>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff816abb50>] _raw_spin_lock+0x20/0x30
[<ffffffff811b1ced>] handle_mm_fault+0x46d/0xfa0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b02e5>] do_page_fault+0x35/0x90
[<ffffffff816ac508>] page_fault+0x28/0x30
Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 ba 01 00 00 00 8b 07 85 c0 75 0a f0 0f b1 17 85 c0 75 f2 5d c3 f3 90 <eb> ec 0f 1f 40 00 81 fe 00 01 00 00 66 90 0f 84 5c 01 00 00 41 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sleep:11514]
Modules linked in: tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock bnep sunrpc ppdev coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd snd_seq_midi snd_seq_midi_event pcspkr btusb snd_ens1371 btrtl btbcm btintel bluetooth snd_rawmidi snd_ac97_codec ac97_bus snd_seq joydev snd_seq_device snd_pcm rfkill snd_timer snd soundcore sg parport_pc parport vmw_vmci i2c_piix4 xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks floppy ata_generic pata_acpi ata_piix libata dm_mirror dm_region_hash
dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 11514 Comm: sleep Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
task: ffff8f0d9d405140 ti: ffff8f0d8f1a8000 task.ti: ffff8f0d8f1a8000
RIP: 0010:[<ffffffff81f11ffd>]  [<ffffffff81f11ffd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0000:ffff8f0d8f1abce8  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff8f0da5811b00 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: fffff8d9403c6130
RBP: ffff8f0d8f1abce8 R08: fffff8d9403c6100 R09: fffff8d940c8a89c
R10: fffff8d940c8a880 R11: 00003ffffffff000 R12: ffff8f0d8f1abd30
R13: 000000002062fd68 R14: ffff8f0dbd61ab80 R15: 0000000081ed2eb2
FS:  00007f914380f740(0000) GS:ffff8f0dbd640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000607188 CR3: 000000000f1b8000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8255bf4b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff8256a400>] _raw_spin_lock+0x20/0x30
[<ffffffff81fe4848>] do_read_fault.isra.61+0x108/0x1b0
[<ffffffff81fe9134>] handle_pte_fault+0x2f4/0xd10
[<ffffffff8256770f>] ? __schedule+0x3ff/0x890
[<ffffffff81febc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8256f5e3>] __do_page_fault+0x203/0x500
[<ffffffff8256f915>] do_page_fault+0x35/0x90
[<ffffffff8256b758>] page_fault+0x28/0x30
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f