NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [containerd-shim:52254]
Modules linked in: tcp_lp veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq crc32_pclmul ghash_clmulni_intel snd_seq_device snd_pcm aesni_intel lrw gf128mul snd_timer glue_helper ablk_helper snd soundcore cryptd ppdev vmw_balloon sg pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm libnvdimm ata_piix mptspi crct10dif_pclmul crct10dif_common crc32c_intel libata scsi_transport_spi mptscsih e1000 mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 52254 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffffa01c55504200 ti: ffffa01c278d8000 task.ti: ffffa01c278d8000
RIP: 0010:[<ffffffffad51dba2>]  [<ffffffffad51dba2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffffa01c278dbbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffffa01c278dbbb0 RCX: 0000000000000002
RDX: ffffffffadc13c50 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffffa01c278dbc30 R08: ffffffffadc13c48 R09: 00003ffffffff000
R10: ffffa01c2403c1b0 R11: 0000000000000246 R12: ffffa01c4f1f75c0
R13: ffffa01c4f1f75c0 R14: 00000000000003f5 R15: ffffa01475ed9600
FS:  000000c000080090(0000) GS:ffffa01c5d6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00035b010 CR3: 00000007cf30c000 CR4: 0000000000760fe0
PKRU: 55555554
Call Trace:
[<ffffffffad481b10>] ? leave_mm+0x130/0x130
[<ffffffffad481b10>] ? leave_mm+0x130/0x130
[<ffffffffad481b10>] ? leave_mm+0x130/0x130
[<ffffffffad51dcbf>] smp_call_function_single+0x5f/0xa0
[<ffffffffad79c615>] ? cpumask_next_and+0x35/0x50
[<ffffffffad51e27b>] smp_call_function_many+0x22b/0x270
[<ffffffffad481fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffad48201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffad5fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffad5fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffad5fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffad5ff2ca>] zap_page_range+0xea/0x140
[<ffffffffad4e1118>] ? wake_up_q+0x58/0x90
[<ffffffffad5196b0>] ? futex_wake+0x90/0x190
[<ffffffffad5f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffad4e8699>] ? pick_next_entity+0xa9/0x190
[<ffffffffad4e918c>] ? set_next_entity+0x3c/0x100
[<ffffffffadbc539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 c0 19 15 ae 48 89 df e8 b7 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [node_exporter:1684]
Modules linked in: binfmt_misc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd sb_edac soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd sg pcspkr vmw_balloon joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata nfit drm mptspi scsi_transport_spi mptscsih mptbase crct10dif_pclmul crct10dif_common libnvdimm crc32c_intel e1000 serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 1684 Comm: node_exporter Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff96ed3a94e2a0 ti: ffff96ed3a00c000 task.ti: ffff96ed3a00c000
RIP: 0010:[<ffffffff8ed16f62>]  [<ffffffff8ed16f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff96ed3a00fc30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff96ed3a00fc00 RCX: 0000000000000001
RDX: ffffffff8f413610 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff96ed3a00fc80 R08: ffffffff8f413608 R09: 00003ffffffff000
R10: ffff96edb4c7eaf8 R11: 0000000000000202 R12: 0000000000000001
R13: ffff96edb96dacc0 R14: ffff96edb96dad40 R15: ffff96ed3a94e308
FS:  000000c00004ac90(0000) GS:ffff96edb96c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0004ad000 CR3: 00000000b118c000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8ec7e730>] ? leave_mm+0x120/0x120
[<ffffffff8ec7e730>] ? leave_mm+0x120/0x120
[<ffffffff8ec7e730>] ? leave_mm+0x120/0x120
[<ffffffff8ed1707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8ef84ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8ed1762b>] smp_call_function_many+0x22b/0x270
[<ffffffff8ec7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ec7e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8edf2e58>] zap_page_range+0xd8/0x150
[<ffffffff8ecdb948>] ? wake_up_q+0x58/0x80
[<ffffffff8eded36d>] SyS_madvise+0x49d/0xab0
[<ffffffff8ecdb990>] ? wake_up_state+0x20/0x20
[<ffffffff8f392ed2>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 0f 95 8f 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [ActiveMQ Inacti:126546]
Modules linked in: nf_conntrack_netlink xt_nat veth nfnetlink ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_addrtype overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer pcspkr joydev snd soundcore sg parport_pc parport i2c_piix4 vmw_vmci sch_fq_codel ip_tables xfs libcrc32c
sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper syscopyarea sysfillrect ata_piix sysimgblt fb_sys_fops ttm e1000 libata drm mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 126546 Comm: ActiveMQ Inacti Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff971c08f6a080 ti: ffff971c07e2c000 task.ti: ffff971c07e2c000
RIP: 0010:[<ffffffffa111109a>]  [<ffffffffa111109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff971c07e2fbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff971c07e2fbc0 RCX: 0000000000000006
RDX: ffffffffa1813a10 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff971c07e2fc38 R08: ffffffffa1813a08 R09: 00003ffffffff000
R10: ffff971b056aed80 R11: 00007fe87b5ba000 R12: ffff971d320f9ba0
R13: ffffe63488515ac0 R14: 0000000000000000 R15: ffffe63488515ac0
FS:  00007fe87b6b6700(0000) GS:ffff971d35780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe9aa1152b0 CR3: 0000000210e3e000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa107a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa107a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa107a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa11111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa1375465>] ? cpumask_next_and+0x35/0x50
[<ffffffffa111175b>] smp_call_function_many+0x22b/0x270
[<ffffffffa107a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa107a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa11e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffa11e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffa11e87c2>] zap_page_range+0x142/0x190
[<ffffffffa10d6768>] ? wake_up_q+0x58/0x80
[<ffffffffa110cca0>] ? futex_wake+0x90/0x180
[<ffffffffa11e368d>] SyS_madvise+0x49d/0xac0
[<ffffffffa177505b>] tracesys+0xa3/0xc9
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 d4 a1 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sdatad:24280]
Modules linked in: tcp_lp fuse sfs(OE) binfmt_misc nls_utf8 isofs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc xfs libcrc32c ppdev pcspkr joydev parport_pc sg parport virtio_balloon i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom virtio_blk virtio_net virtio_console virtio_scsi net_failover failover ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: dattobd]
CPU: 1 PID: 24280 Comm: sdatad Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9aefb4848000 ti: ffff9af59e86c000 task.ti: ffff9af59e86c000
RIP: 0010:[<ffffffff98116f62>]  [<ffffffff98116f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9af59e86fc30  EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff9af59e86fc00 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000286
RBP: ffff9af59e86fc80 R08: ffffffff98813600 R09: 0000000001a4f000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000001
R13: ffff9af1aea5acc0 R14: ffff9af1aea5ad40 R15: ffff9aefb4848068
FS:  00007fbc9b7e6700(0000) GS:ffff9af1aea40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8f7bffe008 CR3: 000000056fdb4000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9807e730>] ? leave_mm+0x120/0x120
[<ffffffff9807e730>] ? leave_mm+0x120/0x120
[<ffffffff9807e730>] ? leave_mm+0x120/0x120
[<ffffffff9811707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff98384ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9811762b>] smp_call_function_many+0x22b/0x270
[<ffffffff9807e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9807e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff981f2e58>] zap_page_range+0xd8/0x150
[<ffffffff98112a60>] ? futex_wake+0x90/0x180
[<ffffffff981ed36d>] SyS_madvise+0x49d/0xab0
[<ffffffff98792e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff98792e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff98792ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff98792e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 0f d5 98 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [containerd-shim:24953]
Modules linked in: veth xt_TCPMSS ip_set_hash_ip ip_set_hash_net nf_tables_ipv4 nf_tables nfnetlink_cttimeout xt_set ip_set_hash_ipportnet openvswitch ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport dummy ip_set ip6table_mangle ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel vmw_balloon sg aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr parport_pc i2c_piix4 vmw_vmci parport
br_netfilter bridge stp llc ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs geneve ip6_udp_tunnel udp_tunnel iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables xfs libcrc32c ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod ahci crc_t10dif crct10dif_generic ata_piix nfit libahci drm libnvdimm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 24953 Comm: containerd-shim Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8893abf7d280 ti: ffff8890cf250000 task.ti: ffff8890cf250000
RIP: 0010:[<ffffffff90116562>]  [<ffffffff90116562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8890cf253be0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8890cf253bb0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8890cf253c30 R08: ffffffff90813c80 R09: ffffed72442cf300
R10: 000000000000154c R11: 0000000000000202 R12: ffffffff901c8aa6
R13: ffff8890cf253bf0 R14: 0000000000020200 R15: 0000000000000000
FS:  000000c000040490(0000) GS:ffff8895ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000137010 CR3: 0000000111d68000 CR4: 00000000007607f0
PKRU: 55555554
Call Trace:
[<ffffffff9007e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9007e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9007e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9011667f>] smp_call_function_single+0x5f/0xa0
[<ffffffff90385de5>] ? cpumask_next_and+0x35/0x50
[<ffffffff90116c2b>] smp_call_function_many+0x22b/0x270
[<ffffffff9007e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9007e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff901edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff901ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff901ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffff901f2dba>] zap_page_range+0xea/0x130
[<ffffffff900daaa8>] ? wake_up_q+0x58/0x80
[<ffffffff901ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffff90786e76>] ? __schedule+0x596/0x860
[<ffffffff90786e82>] ? __schedule+0x5a2/0x860
[<ffffffff90786e76>] ? __schedule+0x596/0x860
[<ffffffff90786e82>] ? __schedule+0x5a2/0x860
[<ffffffff90793f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 13 d5 90 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kubelet:1282]
Modules linked in: xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc edac_mce_amd crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg pcspkr joydev vmw_balloon nfit libnvdimm shpchp vmw_vmci parport_pc parport i2c_piix4 nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops crct10dif_pclmul crct10dif_common ttm ahci crc32c_intel drm serio_raw libahci ata_piix libata vmxnet3 vmw_pvscsi i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 1282 Comm: kubelet Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9ac237e26eb0 ti: ffff9ac107c88000 task.ti: ffff9ac107c88000
RIP: 0010:[<ffffffff9ab0779a>]  [<ffffffff9ab0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9ac107c8bbf0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff9ac107c8bbc0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff9ac107c8bc38 R08: ffffffff9b213160 R09: ffff9ac19da5b1b0
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000091
R13: ffff9ac23fd98b80 R14: ffff9ac1bbb9b780 R15: 0000000000000001
FS:  00007efd8a2e4700(0000) GS:ffff9ac23fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efd880ad0d8 CR3: 0000000003110000 CR4: 00000000000407e0
Call Trace:
[<ffffffff9aa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9aa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9aa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9ab078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9ad4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff9ab07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff9aa77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9aa77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9abc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff9abc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff9abc4a02>] zap_page_range+0x142/0x190
[<ffffffff9aacee38>] ? wake_up_q+0x58/0x80
[<ffffffff9ab033a0>] ? futex_wake+0x90/0x180
[<ffffffff9abbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff9aacee80>] ? wake_up_state+0x20/0x20
[<ffffffff9b11f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9b11f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff9b11f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff9b11f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 73 9b 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [docker-containe:15950]
Modules linked in: binfmt_misc cts 8021q garp mrp ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_17005(E) nf_nat_tftp nf_conntrack_tftp nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) tun vmw_vsock_vmci_transport
vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport nfsd nfs_acl lockd grace auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper nfit ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata libnvdimm drm crct10dif_pclmul crct10dif_common mptspi crc32c_intel serio_raw scsi_transport_spi mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 15950 Comm: docker-containe Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff93b1d4971080 ti: ffff93b1d4fac000 task.ti: ffff93b1d4fac000
RIP: 0010:[<ffffffffa6b1dba2>]  [<ffffffffa6b1dba2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff93b1d4fafbe0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff93b1d4fafbb0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff93b1d4fafc30 R08: ffffffffa7214140 R09: 00003ffffffff000
R10: ffff93b1d513d510 R11: 0000000000000246 R12: ffff93b1d4971080
R13: ffffffffa6aeab04 R14: ffff93b1d4fafb80 R15: ffff93b1d4fafc60
FS:  00007fd2a642a700(0000) GS:ffff93b1ffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000db6578 CR3: 00000002148ac000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa6a81b10>] ? leave_mm+0x130/0x130
[<ffffffffa6a81b10>] ? leave_mm+0x130/0x130
[<ffffffffa6a81b10>] ? leave_mm+0x130/0x130
[<ffffffffa6b1dcbf>] smp_call_function_single+0x5f/0xa0
[<ffffffffa6d9c615>] ? cpumask_next_and+0x35/0x50
[<ffffffffa6b1e27b>] smp_call_function_many+0x22b/0x270
[<ffffffffa6a81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa6a8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa6bfa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa6bfbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa6bfbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffa6bff2ca>] zap_page_range+0xea/0x140
[<ffffffffa6ae1118>] ? wake_up_q+0x58/0x90
[<ffffffffa6bf96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffa71c539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 c0 19 75 a7 48 89 df e8 b7 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [containerd-shim:30877]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype br_netfilter bridge stp llc iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon sg pcspkr joydev vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm nfit sd_mod libahci ata_piix crc_t10dif crct10dif_generic libnvdimm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod fuse
CPU: 3 PID: 30877 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.118.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9d13a3cc6300 ti: ffff9d13a1128000 task.ti: ffff9d13a1128000
RIP: 0010:[<ffffffffb191dba2>]  [<ffffffffb191dba2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9d13a112bbe0  EFLAGS: 00000202
RAX: 0000000000000006 RBX: ffff9d13a112bbb0 RCX: 0000000000000830
RDX: 0000000000000006 RSI: 0000000000000006 RDI: 0000000000000282
RBP: ffff9d13a112bc30 R08: ffffffffb2014640 R09: 00003ffffffff000
R10: ffff9d13ea810e58 R11: 0000000000000246 R12: ffff9d146a6730c0
R13: ffff9d146a6730c0 R14: 000000000000023a R15: ffff9d11c8faa000
FS:  000000c000200490(0000) GS:ffff9d147fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00021b010 CR3: 00000004aa8c0000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb1881b10>] ? leave_mm+0x130/0x130
[<ffffffffb1881b10>] ? leave_mm+0x130/0x130
[<ffffffffb1881b10>] ? leave_mm+0x130/0x130
[<ffffffffb191dcbf>] smp_call_function_single+0x5f/0xa0
[<ffffffffb1b9c615>] ? cpumask_next_and+0x35/0x50
[<ffffffffb191e27b>] smp_call_function_many+0x22b/0x270
[<ffffffffb1881fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb188201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb19fa0ce>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffb19fbc4f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb19fbd83>] tlb_finish_mmu+0x23/0x40
[<ffffffffb19ff2ca>] zap_page_range+0xea/0x140
[<ffffffffb18e1118>] ? wake_up_q+0x58/0x90
[<ffffffffb19f96bd>] SyS_madvise+0x49d/0xac0
[<ffffffffb18e8699>] ? pick_next_entity+0xa9/0x190
[<ffffffffb18e918c>] ? set_next_entity+0x3c/0x100
[<ffffffffb1fc539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 c0 19 55 b2 48 89 df e8 b7 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [kube-apiserver:54766]
Modules linked in: ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_CHECKSUM tun ebtable_filter ebtables ip6table_filter devlink ip6_tables xt_multiport xt_nat veth nf_tables(T) vxlan ip6_udp_tunnel udp_tunnel ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipportnet ip_set_hash_ipport dummy iptable_mangle xt_comment xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_nat_ipv4 ceph libceph dns_resolver overlay(T) toa(OE) vmw_vsock_vmci_transport vsock bnep sunrpc dm_mirror dm_region_hash dm_log dm_mod crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev btusb btrtl btbcm btintel bluetooth rfkill snd_seq_midi snd_seq_midi_event snd_ens1371
snd_rawmidi snd_ac97_codec ac97_bus parport_pc parport snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg vmw_vmci i2c_piix4 nf_conntrack_ipv4 nf_defrag_ipv4 ipip tunnel4 ip_tunnel ipt_REJECT nf_reject_ipv4 ipt_rpfilter xt_set ip_set nfnetlink ip_vs_ftp nf_nat ip_vs_sed ip_vs_nq ip_vs_dh ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs_lblcr ip_vs_lblc ip_vs_lc ip_vs nf_conntrack br_netfilter bridge stp llc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw nfit libnvdimm ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix e1000 libata mptspi scsi_transport_spi drm_panel_orientation_quirks mptscsih mptbase
CPU: 3 PID: 54766 Comm: kube-apiserver Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff88caf3326180 ti: ffff88ca08b98000 task.ti: ffff88ca08b98000
RIP: 0010:[<ffffffff8851109a>]  [<ffffffff8851109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff88ca08b9bbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88ca08b9bbc0 RCX: 0000000000000003
RDX: ffffffff88c13290 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff88ca08b9bc38 R08: ffffffff88c13288 R09: 00003ffffffff000
R10: ffff88c9fe802948 R11: 0000000000000246 R12: ffff88caf965ab80
R13: ffff88ca616c90d0 R14: ffff88caf965ab80 R15: 00000000060b69cb
FS:  000000c000068c90(0000) GS:ffff88caf96c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000c4a020 CR3: 0000000044be0000 CR4: 0000000000360fe0
Call Trace:
[<ffffffff8847a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8847a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8847a3d0>] ? leave_mm+0x110/0x110
[<ffffffff885111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff88775465>] ? cpumask_next_and+0x35/0x50
[<ffffffff8851175b>] smp_call_function_many+0x22b/0x270
[<ffffffff8847a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8847a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff885e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff885e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff885e87c2>] zap_page_range+0x142/0x190
[<ffffffff884d6768>] ? wake_up_q+0x58/0x80
[<ffffffff885e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff88b7505b>] tracesys+0xa3/0xc9
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 89 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 39s! [kubelet:83601]
Modules linked in: xt_nat ipt_REJECT nf_reject_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables xt_conntrack nf_nat nf_conntrack br_netfilter overlay(T) tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock bnep sunrpc iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel snd_seq_midi snd_seq_midi_event aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_ens1371 btusb btrtl btbcm btintel snd_rawmidi bluetooth snd_ac97_codec ac97_bus snd_seq rfkill snd_seq_device snd_pcm snd_timer snd soundcore sg parport_pc parport vmw_vmci i2c_piix4 xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic vmwgfx drm_kms_helper syscopyarea pata_acpi sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi ata_piix drm e1000 mptscsih libata drm_panel_orientation_quirks nfit mptbase libnvdimm dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 83601 Comm: kubelet Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff980a0b36b0c0 ti: ffff980a0b388000 task.ti: ffff980a0b388000
RIP: 0010:[<ffffffff94d1109a>]  [<ffffffff94d1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff980a0b38bbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff980a0b38bbc0 RCX: 0000000000000002
RDX: ffffffff95413010 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff980a0b38bc38 R08: ffffffff95413008 R09: 00003ffffffff000
R10: ffff980af09ffcb0 R11: 0000000000000246 R12: ffffffff94cdd70b
R13: ffff980a0b38bbd0 R14: 0000000000000046 R15: ffff980a0b38bb58
FS:  00007fdca65e9700(0000) GS:ffff980af9600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f245a1a9000 CR3: 0000000135028000 CR4: 00000000003607f0
Call Trace:
[<ffffffff94c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94d111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff94f75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff94d1175b>] smp_call_function_many+0x22b/0x270
[<ffffffff94c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff94c7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff94de4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff94de5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff94de87c2>] zap_page_range+0x142/0x190
[<ffffffff94cd6768>] ? wake_up_q+0x58/0x80
[<ffffffff94de368d>] SyS_madvise+0x49d/0xac0
[<ffffffff94e640a4>] ? mntput+0x24/0x40
[<ffffffff95374ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 94 95 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [ipc_listen:61078]
Modules linked in: binfmt_misc fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul snd_ens1371 snd_rawmidi snd_ac97_codec glue_helper ablk_helper cryptd ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore joydev
ppdev pcspkr sg vmw_balloon nfit libnvdimm parport_pc parport i2c_piix4 shpchp vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common mptspi crc32c_intel libata scsi_transport_spi e1000 mptscsih mptbase serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 61078 Comm: ipc_listen Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8827d2f3dee0 ti: ffff88291566c000 task.ti: ffff88291566c000
RIP: 0010:[<ffffffffbe50779a>]  [<ffffffffbe50779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff88291566fbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88291566fbc0 RCX: 0000000000000008
RDX: ffffffffbec142f0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff88291566fc38 R08: ffffffffbec142e8 R09: ffff882602bd6ca8
R10: 0000000000000002 R11: 0000000000000246 R12: 000280da00000000
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fe049ffb700(0000) GS:ffff882ad1a80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe018022558 CR3: 00000002ea602000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffbe476e50>] ? leave_mm+0x110/0x110
[<ffffffffbe476e50>] ? leave_mm+0x110/0x110
[<ffffffffbe476e50>] ? leave_mm+0x110/0x110
[<ffffffffbe5078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffbe74a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffbe507e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffbe477018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbe477088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbe5c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffbe5c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffbe5c4a02>] zap_page_range+0x142/0x190
[<ffffffffbe4cee38>] ? wake_up_q+0x58/0x80
[<ffffffffbe5033a0>] ? futex_wake+0x90/0x180
[<ffffffffbe5bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffbeb1f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 13 bf 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [containerd-shim:20456]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon
pcspkr joydev snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer sg snd soundcore parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx e1000 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi mptscsih libata mptbase nfit drm_panel_orientation_quirks libnvdimm
CPU: 1 PID: 20456 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9415f2b330c0 ti: ffff9415f5bb0000 task.ti: ffff9415f5bb0000
RIP: 0010:[<ffffffff85b1109a>]  [<ffffffff85b1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9415f5bb3bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9415f5bb3bc0 RCX: 0000000000000001
RDX: ffffffff86212d90 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9415f5bb3c38 R08: ffffffff86212d88 R09: fffff88240a785c0
R10: 000000000000312b R11: 0000000000000202 R12: ffff94163ffd8000
R13: ffffffff85bc0c06 R14: ffff9415f5bb3c00 R15: 0000000000020200
FS:  000000c000080890(0000) GS:ffff941635640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f25e7e42a80 CR3: 00000001f9a12000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff85a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff85a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff85a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff85b111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff85d75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff85b1175b>] smp_call_function_many+0x22b/0x270
[<ffffffff85a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff85be4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff85be5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff85be87c2>] zap_page_range+0x142/0x190
[<ffffffff85ad6768>] ? wake_up_q+0x58/0x80
[<ffffffff85be368d>] SyS_madvise+0x49d/0xac0
[<ffffffff85adfff6>] ? update_curr+0x86/0x1e0
[<ffffffff86174ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 74 86 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [threads_sampler:2483]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi vmw_balloon aesni_intel snd_ac97_codec lrw gf128mul glue_helper ac97_bus ablk_helper snd_seq cryptd snd_seq_device snd_pcm joydev pcspkr sg snd_timer snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm libnvdimm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata mptspi scsi_transport_spi e1000 mptscsih serio_raw mptbase drm_panel_orientation_quirks fuse
CPU: 4 PID: 2483 Comm: threads_sampler Kdump: loaded Tainted: G             L ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9c33aaeb4200 ti: ffff9c32b6834000 task.ti: ffff9c32b6834000
RIP: 0010:[<ffffffff99f16b82>]  [<ffffffff99f16b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c32b6837be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9c32b6837bb0 RCX: 0000000000000001
RDX: ffffffff9a6137d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9c32b6837c30 R08: ffffffff9a6137c8 R09: 00003ffffffff000
R10: ffff9c32aef22288 R11: 0000000000000206 R12: 00006ff9930a5717
R13: ffffffff99ee497c R14: ffff9c32b6837b80 R15: ffff9c30f59ee000
FS:  00007f5208dff700(0000) GS:ffff9c33ad700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff7db18c860 CR3: 00000000916e8000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff99e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff99e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff99e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff99f16c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9a186825>] ? cpumask_next_and+0x35/0x50
[<ffffffff99f1724b>] smp_call_function_many+0x22b/0x270
[<ffffffff99e7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff99e7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff99fee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff99ff012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff99ff0253>] tlb_finish_mmu+0x23/0x30
[<ffffffff99ff371a>] zap_page_range+0xea/0x130
[<ffffffff99f127c0>] ? futex_wake+0x90/0x180
[<ffffffff99fedc0d>] SyS_madvise+0x49d/0xac0
[<ffffffff99ee64cb>] ? put_prev_entity+0x8b/0x400
[<ffffffff99e7dfe9>] ? switch_mm_irqs_off+0x109/0x290
[<ffffffff99e2b621>] ? __switch_to+0x151/0x580
[<ffffffff9a599f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 60 18 b5 9a 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#11 stuck for 24s! [containerd-shim:3019]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel snd_ens1371 aesni_intel snd_rawmidi lrw snd_ac97_codec gf128mul glue_helper ablk_helper ac97_bus cryptd snd_seq snd_seq_device snd_pcm snd_timer snd soundcore ppdev sg vmw_balloon pcspkr joydev nfit parport_pc libnvdimm parport i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod
sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common mptspi libata crc32c_intel scsi_transport_spi e1000 mptscsih serio_raw mptbase i2c_core
CPU: 11 PID: 3019 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9d1138bc1fa0 ti: ffff9d185aa60000 task.ti: ffff9d185aa60000
RIP: 0010:[<ffffffffb730779a>]  [<ffffffffb730779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9d185aa63bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9d185aa63bc0 RCX: 0000000000000009
RDX: ffffffffb7a14570 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9d185aa63c38 R08: ffffffffb7a14568 R09: fffff6c72009f5c0
R10: fffff6c72009f5c0 R11: 0000000000000202 R12: ffff9d1138bc1fa0
R13: ffff9d1138bc1fa0 R14: ffff9d185aa63ba0 R15: 0000000000000001
FS:  000000c000200090(0000) GS:ffff9d18bd8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc1e24dc580 CR3: 00000007d0fea000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb7276e50>] ? leave_mm+0x110/0x110
[<ffffffffb7276e50>] ? leave_mm+0x110/0x110
[<ffffffffb7276e50>] ? leave_mm+0x110/0x110
[<ffffffffb73078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb754a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffb7307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffb7277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb7277088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb73c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffb73c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffb73c4a02>] zap_page_range+0x142/0x190
[<ffffffffb72cee38>] ? wake_up_q+0x58/0x80
[<ffffffffb73bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffb72d80a6>] ? update_curr+0x86/0x1e0
[<ffffffffb791f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 f3 b7 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [minio:112566]
Modules linked in: nfnetlink_queue nfnetlink_log bluetooth rfkill xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq joydev snd_seq_device snd_pcm pcspkr snd_timer snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic
pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm e1000 ata_piix mptspi scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 112566 Comm: minio Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8a6af8281040 ti: ffff8a6d7d774000 task.ti: ffff8a6d7d774000
RIP: 0010:[<ffffffffb991109a>]  [<ffffffffb991109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8a6d7d777bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8a6d7d777b80 RCX: 0000000000000004
RDX: ffffffffba013510 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8a6d7d777c38 R08: ffffffffba013508 R09: 00003ffffffff000
R10: ffff8a6a2d1540d8 R11: 0000000000000202 R12: 0000000000000010
R13: ffffffffb98611c6 R14: ffffffffffffff10 R15: 0000000000000282
FS:  000000c00137a890(0000) GS:ffff8a6dc5140000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fff735538c8 CR3: 00000003f9eb4000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb987a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb987a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb987a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb99111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb9b75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffb991175b>] smp_call_function_many+0x22b/0x270
[<ffffffffb987a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb987a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb99e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffb99e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffb99e87c2>] zap_page_range+0x142/0x190
[<ffffffffb98d6768>] ? wake_up_q+0x58/0x80
[<ffffffffb99e368d>] SyS_madvise+0x49d/0xac0
[<ffffffffb9f7505b>] tracesys+0xa3/0xc9
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 54 ba 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [pushgateway:25938]
Modules linked in: xt_comment xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink tun vmw_vsock_vmci_transport vsock sunrpc vfat fat sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common mptspi libata crc32c_intel scsi_transport_spi mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ipmi_msghandler]
CPU: 1 PID: 25938 Comm: pushgateway Tainted: G        W    L ------------   3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1704120155 04/12/2017
task: ffff9d12b65ce300 ti: ffff9d0de2008000 task.ti: ffff9d0de2008000
RIP: 0010:[<ffffffffa3116a32>]  [<ffffffffa3116a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9d0de200bbe0  EFLAGS: 00000202
RAX: 0000000000000007 RBX: ffff9d0de200bbb0 RCX: 0000000000000039
RDX: 000000000000007f RSI: 0000000000000007 RDI: 0000000000000282
RBP: ffff9d0de200bc30 R08: ffffffffa3813cc0 R09: 00003ffffffff000
R10: ffff9d133ca29e58 R11: 0000000000000202 R12: ffff9d0de200bb60
R13: 0000000000000000 R14: ffff9d157fc9acc0 R15: ffffffffa30d3c1a
FS:  0000000001051b70(0063) GS:ffff9d157fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa751568e50 CR3: 0000000095d24000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa307ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa307ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa307ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa3116b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa3386a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffa31170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffa307e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa307e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa31ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa31f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa31f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffa31f372a>] zap_page_range+0xea/0x130
[<ffffffffa30dada8>] ? wake_up_q+0x58/0x80
[<ffffffffa31edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffa302b59e>] ? __switch_to+0xce/0x580
[<ffffffffa3795f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 60 15 d5 a3 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [containerd:8941]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg nfit vmw_vmci libnvdimm i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm ata_piix drm mptspi libata scsi_transport_spi crct10dif_pclmul mptscsih crct10dif_common crc32c_intel mptbase e1000 serio_raw i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 8941 Comm: containerd Tainted: G               ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8801b0e33f40 ti: ffff8801bb278000 task.ti: ffff8801bb278000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8801bb27bc20  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffff88017c9e5000 RCX: 0000000000000005
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000286
RBP: ffff8801bb27bc70 R08: ffffffff816cee28 R09: 000000c000bce000
R10: 00000000000001e7 R11: 0000000000000202 R12: ffff88017fc19000
R13: 000000005076d999 R14: 0000000000000000 R15: 0000000000000000
FS:  00007fa40ffff700(0000) GS:ffff8802367c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa43c5b2098 CR3: 0000000206367000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff8801bb27bcf8
0000000000000003 000000005076d999 ffff8801bb27bc88 0000000000000004
ffffffff8106e4a0 ffffffff8106e4a0 ffff8801bb27bca0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b08fa>] zap_page_range+0x13a/0x180
[<ffffffff810c47c8>] ? wake_up_q+0x58/0x80
[<ffffffff811ac71d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff810f8040>] ? SyS_futex+0x80/0x180
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 68s! [JS Helper:59749]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi aesni_intel snd_ac97_codec lrw ac97_bus
gf128mul ppdev glue_helper ablk_helper snd_seq snd_seq_device cryptd vmw_balloon snd_pcm snd_timer snd joydev pcspkr soundcore sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops pata_acpi mptspi ttm scsi_transport_spi crct10dif_pclmul mptscsih crct10dif_common ata_piix drm mptbase nfit e1000 crc32c_intel serio_raw drm_panel_orientation_quirks libata libnvdimm fuse
CPU: 3 PID: 59749 Comm: JS Helper Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff99a68398e300 ti: ffff99a678398000 task.ti: ffff99a678398000
RIP: 0010:[<ffffffff8c916562>]  [<ffffffff8c916562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff99a67839bbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff99a67839bbb0 RCX: 0000000000000003
RDX: ffffffff8d013c90 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff99a67839bc30 R08: ffffffff8d013c88 R09: 00003ffffffff000
R10: ffff99a643702e58 R11: 0000000000000246 R12: ffff99a76f53e400
R13: ffff99a68398e368 R14: ffffffff8c8e4759 R15: ffff99a67839bb60
FS:  00007f5bd0f7c700(0000) GS:ffff99a7796c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f79a335242c CR3: 0000000046c48000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8c87e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8c87e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8c87e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8c91667f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8cb85de5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8c916c2b>] smp_call_function_many+0x22b/0x270
[<ffffffff8c87e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c87e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8c9edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8c9ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8c9ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffff8c9f2dba>] zap_page_range+0xea/0x130
[<ffffffff8c912040>] ? futex_wake+0x90/0x180
[<ffffffff8c9ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffff8cf86e82>] ? __schedule+0x5a2/0x860
[<ffffffff8cf86e76>] ? __schedule+0x596/0x860
[<ffffffff8cf86e82>] ? __schedule+0x5a2/0x860
[<ffffffff8cf93f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 13 55 8d 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [ganesha.nfsd:24380]
Modules linked in: tcp_diag inet_diag iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod xfs libcrc32c sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw vmw_balloon gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg i2c_piix4 parport_pc parport vmw_vmci ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci ata_piix drm nfit libata crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks fuse
CPU: 5 PID: 24380 Comm: ganesha.nfsd Kdump: loaded Tainted: G        W      ------------   3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8ad6b04d9080 ti: ffff8addf888c000 task.ti: ffff8addf888c000
RIP: 0010:[<ffffffffb6d16a36>]  [<ffffffffb6d16a36>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff8addf888fbe0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8addf888fbb0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8addf888fc30 R08: ffffffffb74146c0 R09: 00007fd1771f5000
R10: 00007fd1772f2ba0 R11: 0000000000000206 R12: 000042957e50596b
R13: ffffffffb6ce482c R14: ffff8addf888fb80 R15: ffff8ade3ffd9008
FS:  00007fd1772f4700(0000) GS:ffff8ade3fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000022a09d0 CR3: 000000080c7c6000 CR4: 00000000001607e0
Call Trace:
 [<ffffffffb6c7ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffb6c7ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffb6c7ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffb6d16b4f>] smp_call_function_single+0x5f/0xa0
 [<ffffffffb6f86a55>] ? cpumask_next_and+0x35/0x50
 [<ffffffffb6d170fb>] smp_call_function_many+0x22b/0x270
 [<ffffffffb6c7e258>] native_flush_tlb_others+0xb8/0xc0
 [<ffffffffb6c7e2cb>] flush_tlb_mm_range+0x6b/0x140
 [<ffffffffb6dee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
 [<ffffffffb6df013f>] arch_tlb_finish_mmu+0x3f/0x80
 [<ffffffffb6df0263>] tlb_finish_mmu+0x23/0x30
 [<ffffffffb6df372a>] zap_page_range+0xea/0x130
 [<ffffffffb6d12670>] ? futex_wake+0x90/0x180
 [<ffffffffb6dedc1d>] SyS_madvise+0x49d/0xac0
 [<ffffffffb7399f92>] system_call_fastpath+0x25/0x2a
Code: 14 c5 60 18 95 b7 48 89 df e8 97 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [popen.rb:42:13111]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink arc4 md4 nls_utf8 cifs ccm dns_resolver sunrpc ppdev pcspkr virtio_balloon sg i2c_piix4 parport_pc parport joydev ip_tables xfs libcrc32c sr_mod cdrom virtio_net virtio_blk net_failover failover virtio_console virtio_scsi ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci drm_panel_orientation_quirks virtio_ring floppy virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 13111 Comm: popen.rb:42 Tainted: G             L ------------   3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff8abe2a11e300 ti: ffff8abd79da0000 task.ti: ffff8abd79da0000
RIP: 0010:[<ffffffffb0b16b62>]  [<ffffffffb0b16b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8abd79da3be0  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffff8abd79da3bb0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff8abd79da3c30 R08: ffffffffb12137c0 R09: 00003ffffffff000
R10: ffff8abef660f290 R11: 0000000000000206 R12: 0000360ec6e8bfcf
R13: ffffffffb0ae495c R14: ffff8abd79da3b80 R15: ffff8ab4bfd11000
FS:  00007f09626d0700(0000) GS:ffff8abc5aec0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0986fdd000 CR3: 0000000be30fe000 CR4: 00000000000006e0
Call Trace:
[<ffffffffb0a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb0a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb0a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb0b16c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb0d86865>] ? cpumask_next_and+0x35/0x50
[<ffffffffb0b1722b>] smp_call_function_many+0x22b/0x270
[<ffffffffb0a7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb0a7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb0bee6c6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb0bf018f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb0bf02b3>] tlb_finish_mmu+0x23/0x30
[<ffffffffb0bf377a>] zap_page_range+0xea/0x130
[<ffffffffb0b127a0>] ? futex_wake+0x90/0x180
[<ffffffffb0bedc6d>] SyS_madvise+0x49d/0xac0
[<ffffffffb1199ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb1199ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb1199f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb1199ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 75 b1 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [mariadbd:5480]
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [mariadbd:6029]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg ppdev joydev pcspkr vmw_balloon i2c_piix4 parport_pc parport vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper ahci ata_piix syscopyarea sysfillrect sysimgblt libahci
fb_sys_fops ttm sd_mod crc_t10dif libata crct10dif_generic drm crct10dif_pclmul crct10dif_common serio_raw crc32c_intel nfit vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 6029 Comm: mariadbd Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8ebb02c00000 ti: ffff8ebc3896c000 task.ti: ffff8ebc3896c000
RIP: 0010:[<ffffffffaa116f72>]  [<ffffffffaa116f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8ebc3896fc30  EFLAGS: 00000202
RAX: 0000000000000006 RBX: ffff8ebc3896fc00 RCX: 000000000000003a
RDX: 000000000000003f RSI: 0000000000000006 RDI: 0000000000000286
RBP: ffff8ebc3896fc80 R08: ffffffffaa813e40 R09: 00007f5385f67000
R10: 00005613fff16760 R11: 0000000000000206 R12: ffff8ebc3fc5acc0
R13: ffffffffaa0e103c R14: ffff8ebc3896fbb8 R15: ffffffffaa0e2e89
FS:  00007f5386766700(0000) GS:ffff8ebc3fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc8f4012000 CR3: 00000000b1e22000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaa07e730>] ? leave_mm+0x120/0x120
[<ffffffffaa07e730>] ? leave_mm+0x120/0x120
[<ffffffffaa07e730>] ? leave_mm+0x120/0x120
[<ffffffffaa11708f>] smp_call_function_single+0x5f/0xa0
[<ffffffffaa385005>] ? cpumask_next_and+0x35/0x50
[<ffffffffaa11763b>] smp_call_function_many+0x22b/0x270
[<ffffffffaa07e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaa07e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffaa1f2ed8>] zap_page_range+0xd8/0x150
[<ffffffffaa112a70>] ? futex_wake+0x90/0x180
[<ffffffffaa1ed3ed>] SyS_madvise+0x49d/0xab0
[<ffffffffaa792ed2>] system_call_fastpath+0x25/0x2a

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [docker-proxy:2758]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi ext4 snd_ac97_codec ac97_bus mbcache snd_seq jbd2 snd_seq_device snd_pcm snd_timer crc32_pclmul ghash_clmulni_intel snd aesni_intel
lrw gf128mul glue_helper ppdev soundcore ablk_helper cryptd sg vmw_balloon joydev pcspkr parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper nfit syscopyarea sysfillrect sysimgblt fb_sys_fops libnvdimm ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel mptspi scsi_transport_spi mptscsih mptbase libata serio_raw e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 2758 Comm: docker-proxy Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8aebde783180 ti: ffff8ae7f9388000 task.ti: ffff8ae7f9388000
RIP: 0010:[<ffffffff93116a12>]  [<ffffffff93116a12>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8ae7f938bbe0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8ae7f938bbb0 RCX: 0000000000000001
RDX: ffffffff938137d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8ae7f938bc30 R08: ffffffff938137c8 R09: 00003ffffffff000
R10: ffff8aebdee495f0 R11: 0000000000000202 R12: ffff8aebf7ef8b40
R13: ffff8aebf7ef8b40 R14: 00000000000003ad R15: ffff8aec05650200
FS:  00007f64d8de0700(0000) GS:ffff8aec0b440000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000141010 CR3: 000000049ecb4000 CR4: 0000000000360fe0
Call Trace:
[<ffffffff9307ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9307ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9307ddd0>] ? leave_mm+0x120/0x120
[<ffffffff93116b2f>] smp_call_function_single+0x5f/0xa0
[<ffffffff933869f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff931170db>] smp_call_function_many+0x22b/0x270
[<ffffffff9307e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9307e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff931ee656>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff931f011f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff931f0243>] tlb_finish_mmu+0x23/0x30
[<ffffffff931f370a>] zap_page_range+0xea/0x130
[<ffffffff930dad88>] ? wake_up_q+0x58/0x80
[<ffffffff931edbfd>] SyS_madvise+0x49d/0xac0
[<ffffffff93795f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 60 15 d5 93 48 89 df e8 57 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#8 stuck for 67s! [ThreadPoolForeg:40657]
Modules linked in: tcp_lp tun devlink nfsv3 nfs_acl cmac rpcsec_gss_krb5 auth_rpcgss arc4 md4 nfsv4 nls_utf8 cifs nfs lockd grace fscache ccm dns_resolver ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip_set ebtable_filter ebtables ip6table_filter ip6_tables ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_CHECKSUM iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul snd_ens1371
glue_helper ablk_helper cryptd snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device joydev snd_pcm pcspkr sg snd_timer snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit crct10dif_pclmul crct10dif_common crc32c_intel libata libnvdimm e1000 nvme serio_raw nvme_core drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 8 PID: 40657 Comm: ThreadPoolForeg Not tainted 3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9b8beba83180 ti: ffff9b8aed3f0000 task.ti: ffff9b8aed3f0000
RIP: 0010:[<ffffffff8ad16a32>]  [<ffffffff8ad16a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9b8aed3f3be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9b8aed3f3bb0 RCX: 0000000000000004
RDX: ffffffff8b413f50 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9b8aed3f3c30 R08: ffffffff8b413f48 R09: 00003ffffffff000
R10: ffff9b8b9d0e0000 R11: 0000000000000206 R12: 0000005c0607eebb
R13: ffffffff8ace482c R14: ffff9b8aed3f3b80 R15: ffff9b8af8a8e000
FS:  00007fcf03aad700(0000) GS:ffff9b8c33800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb0e000bb98 CR3: 00000000a4bf8000 CR4: 00000000003407e0
Call Trace:
[<ffffffff8ac7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8ac7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8ac7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8ad16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8af86a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff8ad170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff8ac7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ac7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8adee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8adf013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8adf0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff8adf372a>] zap_page_range+0xea/0x130
[<ffffffff8ad12670>] ? futex_wake+0x90/0x180
[<ffffffff8adedc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff8b395f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 60 15 95 8b 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [popen.rb:41:25208]
Modules linked in: tcp_lp btrfs raid6_pq xor vfat fat tcp_diag inet_diag ext4 mbcache jbd2 fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc uas coretemp usb_storage crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon
sg pcspkr joydev vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi ata_piix scsi_transport_spi mptscsih libata mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 25208 Comm: popen.rb:41 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
task: ffff9c85815c20e0 ti: ffff9c84ef00c000 task.ti: ffff9c84ef00c000
RIP: 0010:[<ffffffff92916f62>]  [<ffffffff92916f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c84ef00fc30  EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffff9c85815c2148 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9c84ef00fc80 R08: 0000000000000001 R09: 00007f3c1cfac000
R10: 00007f3c39f0a140 R11: 0000000000000206 R12: ffff9c85bfc1acc0
R13: ffffffff928e103c R14: ffff9c84ef00fbb8 R15: ffffffff928e2e89
FS:  00007f3c1d0ab700(0000) GS:ffff9c85bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3c2e6f13e0 CR3: 00000000a5918000 CR4: 00000000000007f0
Call Trace:
[<ffffffff9287e730>] ? leave_mm+0x120/0x120
[<ffffffff9287e730>] ? leave_mm+0x120/0x120
[<ffffffff9287e730>] ? leave_mm+0x120/0x120
[<ffffffff9291707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff92b84ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9291762b>] smp_call_function_many+0x22b/0x270
[<ffffffff9287e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9287e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff929f2e58>] zap_page_range+0xd8/0x150
[<ffffffff92912a60>] ? futex_wake+0x90/0x180
[<ffffffff929ed36d>] SyS_madvise+0x49d/0xab0
[<ffffffff92f92e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff92f92e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff92f92ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff92f92e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 0f 55 93 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [Plex Media Serv:1571]
Modules linked in: arc4 md4 rpcsec_gss_krb5 nfsv4 nfs fscache nls_utf8 cifs dns_resolver ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ppdev pcspkr sg joydev virtio_balloon shpchp parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net virtio_scsi bochs_drm drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw virtio_pci virtio_ring virtio ata_piix libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1571 Comm: Plex Media Serv Tainted: G             L ------------   3.10.0-693.17.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff88005989cf10 ti: ffff8800610cc000 task.ti: ffff8800610cc000
RIP: 0010:[<ffffffff810fb21a>]  [<ffffffff810fb21a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8800610cfbf0  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff8800610cfbc0 RCX: 0000000000000830
RDX: 0000000000000002 RSI: 0000000000000002 RDI: 0000000000000282
RBP: ffff8800610cfc40 R08: ffffffff816d69e0 R09: 00007f7adf8e2000
R10: 0000000000000000 R11: 0000000000000206 R12: ffffffff816b9bff
R13: ffffffff816b9c06 R14: ffffffff816b9c0d R15: ffffffff816b9c14
FS:  00007f7ad7ff7700(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7adc1c6138 CR3: 0000000078b56000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8106f340>] ? leave_mm+0x110/0x110
[<ffffffff8106f340>] ? leave_mm+0x110/0x110
[<ffffffff8106f340>] ? leave_mm+0x110/0x110
[<ffffffff810fb31f>] smp_call_function_single+0x5f/0xa0
[<ffffffff81324bd5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810fb8b3>] smp_call_function_many+0x223/0x260
[<ffffffff8106f508>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106f5d8>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811af0f3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b0415>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b299a>] zap_page_range+0x13a/0x180
[<ffffffff810f6e60>] ? futex_wake+0x90/0x180
[<ffffffff811ae7bd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816b895a>] ? system_call_after_swapgs+0x187/0x214
[<ffffffff816b89fd>] system_call_fastpath+0x16/0x1b
[<ffffffff816b889d>] ? system_call_after_swapgs+0xca/0x214
Code: 40 95 01 00 48 89 de 48 03 14 c5 20 8e b1 81 48 89 df e8 5a e0 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [containerd-shim:24728]
Modules linked in: binfmt_misc xt_multiport ipt_rpfilter iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth xt_statistic xt_recent xt_nat ipt_REJECT nf_reject_ipv4 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_filter rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache overlay(T) xt_set ip_set nfnetlink xt_addrtype iptable_nat nf_nat_ipv4 nf_nat xt_comment xt_mark xt_conntrack vmw_vsock_vmci_transport vsock iptable_mangle dm_mirror dm_region_hash dm_log dm_mod sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd i2c_piix4 ppdev joydev pcspkr vmw_balloon sg vmw_vmci parport_pc parport nfsd nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh
ip_vs_wrr ip_vs_rr ip_vs auth_rpcgss nfs_acl lockd grace sunrpc nf_conntrack libcrc32c br_netfilter bridge stp llc ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops sd_mod ttm crc_t10dif crct10dif_generic drm ahci ata_piix libahci crct10dif_pclmul crct10dif_common crc32c_intel nfit libata serio_raw libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks fuse
CPU: 5 PID: 24728 Comm: containerd-shim Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
task: ffff97cbbeb49080 ti: ffff97c8b55c0000 task.ti: ffff97c8b55c0000
RIP: 0010:[<ffffffffae116802>]  [<ffffffffae116802>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff97c8b55c3be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff97c8b55c3bb0 RCX: 0000000000000001
RDX: ffffffffae8137d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff97c8b55c3c30 R08: ffffffffae8137c8 R09: ffffd391066359c0
R10: 0000000000004b29 R11: 0000000000000246 R12: ffffffffae1c8c76
R13: ffff97c8b55c3bf0 R14: 0000000000020200 R15: 0000000000000000
FS:  000000c000040b10(0000) GS:ffff97cc5d740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4cef4e9220 CR3: 000000069cf92000 CR4: 00000000001607e0
Call Trace:
[<ffffffffae07e6a0>] ? leave_mm+0x120/0x120
[<ffffffffae07e6a0>] ? leave_mm+0x120/0x120
[<ffffffffae07e6a0>] ? leave_mm+0x120/0x120
[<ffffffffae11691f>] smp_call_function_single+0x5f/0xa0
[<ffffffffae385fe5>] ? cpumask_next_and+0x35/0x50
[<ffffffffae116ecb>] smp_call_function_many+0x22b/0x270
[<ffffffffae07e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffae07e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffae1edee6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffae1ef9af>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffae1efad3>] tlb_finish_mmu+0x23/0x30
[<ffffffffae1f2f9a>] zap_page_range+0xea/0x130
[<ffffffffae0db148>] ? wake_up_q+0x58/0x80
[<ffffffffae1ed48d>] SyS_madvise+0x49d/0xac0
[<ffffffffae1154c0>] ? SyS_futex+0x80/0x190
[<ffffffffae793f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 13 d5 ae 48 89 df e8 57 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 25s! [splunkd:1176]
Modules linked in: vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm snd_ens1371 snd_rawmidi irqbypass snd_ac97_codec crc32_pclmul ac97_bus snd_seq ghash_clmulni_intel ppdev snd_seq_device btusb btrtl snd_pcm vmw_balloon aesni_intel btbcm lrw btintel gf128mul glue_helper ablk_helper bluetooth cryptd snd_timer joydev pcspkr rfkill snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ahci drm ata_piix libahci crc32c_intel serio_raw nfit libnvdimm libata mptspi scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks
CPU: 4 PID: 1176 Comm: splunkd Kdump: loaded Not tainted 3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff98f67bbaa0e0 ti: ffff98f6c73bc000 task.ti: ffff98f6c73bc000
RIP: 0010:[<ffffffff95316f62>]  [<ffffffff95316f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff98f6c73bfc30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff98f6c73bfc00 RCX: 0000000000000004
RDX: ffffffff95a13d90 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff98f6c73bfc80 R08: ffffffff95a13d88 R09: 00007f7e8e400000
R10: 0000000000000000 R11: 0000000000000206 R12: ffff98f7b371acc0
R13: ffffffff952e103c R14: ffff98f6c73bfbb8 R15: ffffffff952e2e89
FS:  00007f7e8e5ff700(0000) GS:ffff98f7b3700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000211e178 CR3: 00000001cd2ee000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9527e730>] ? leave_mm+0x120/0x120
[<ffffffff9527e730>] ? leave_mm+0x120/0x120
[<ffffffff9527e730>] ? leave_mm+0x120/0x120
[<ffffffff9531707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff95584ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9531762b>] smp_call_function_many+0x22b/0x270
[<ffffffff9527e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9527e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff953f2e58>] zap_page_range+0xd8/0x150
[<ffffffff952db948>] ? wake_up_q+0x58/0x80
[<ffffffff95312a60>] ? futex_wake+0x90/0x180
[<ffffffff953ed36d>] SyS_madvise+0x49d/0xab0
[<ffffffff95992ed2>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 0f f5 95 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [containerd-shim:16624]
Modules linked in: xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper serio_raw crc32c_intel ata_generic nfit libnvdimm syscopyarea pata_acpi sysfillrect sysimgblt
fb_sys_fops mptspi ttm ata_piix scsi_transport_spi drm mptscsih libata mptbase e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 16624 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8b7ac6f32080 ti: ffff8b7a83fd8000 task.ti: ffff8b7a83fd8000
RIP: 0010:[<ffffffff96b111ea>]  [<ffffffff96b111ea>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8b7a83fdbbf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8b7a83fdbbc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8b7a83fdbc38 R08: ffffffff97212dc0 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: ffff8b7a83fdbbb8
R13: ffff8b7a83fdbb80 R14: ffff8b7adb2e5b40 R15: 00000000f4e858cc
FS:  000000c4200d9490(0000) GS:ffff8b7afcd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4202ad010 CR3: 00000006c6af0000 CR4: 00000000003607e0
Call Trace:
[<ffffffff96a7a400>] ? leave_mm+0x110/0x110
[<ffffffff96a7a400>] ? leave_mm+0x110/0x110
[<ffffffff96a7a400>] ? leave_mm+0x110/0x110
[<ffffffff96b112ff>] smp_call_function_single+0x5f/0xa0
[<ffffffff96d75af5>] ? cpumask_next_and+0x35/0x50
[<ffffffff96b118ab>] smp_call_function_many+0x22b/0x270
[<ffffffff96a7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff96a7a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff96be44c7>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff96be5dc5>] tlb_finish_mmu+0x55/0x60
[<ffffffff96be8c12>] zap_page_range+0x142/0x190
[<ffffffff96ad67a8>] ? wake_up_q+0x58/0x80
[<ffffffff96be3add>] SyS_madvise+0x49d/0xac0
[<ffffffff97175d9b>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 ba 74 97 48 89 df e8 6a b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [ThreadPoolForeg:9745]
Modules linked in: tcp_lp rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmhgfs(OE) bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev btusb btrtl vmw_balloon btbcm pcspkr btintel bluetooth vmw_vmci joydev snd_timer snd sg rfkill soundcore shpchp i2c_piix4 nfit libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul drm crct10dif_common crc32c_intel libata mptspi scsi_transport_spi serio_raw e1000 mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 9745 Comm: ThreadPoolForeg Tainted: G           OE  ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff932ef3670000 ti: ffff932f3afa4000 task.ti: ffff932f3afa4000
RIP: 0010:[<ffffffffbd50779a>]  [<ffffffffbd50779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff932f3afa7bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff932f3afa7bc0 RCX: 0000000000000006
RDX: ffffffffbdc13df0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff932f3afa7c38 R08: ffffffffbdc13de8 R09: ffff932f10712948
R10: 00007fd9303ea8a0 R11: 0000000000000206 R12: ffff932f75718b00
R13: ffffffffbd4d865c R14: ffff932f3afa7ba0 R15: 0000000000000001
FS:  00007fd9303eb700(0000) GS:ffff932f757c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001e71c30 CR3: 00000001f8fba000 CR4: 00000000003607e0
Call Trace:
[<ffffffffbd476e50>] ? leave_mm+0x110/0x110
[<ffffffffbd476e50>] ? leave_mm+0x110/0x110
[<ffffffffbd476e50>] ? leave_mm+0x110/0x110
[<ffffffffbd5078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffbd74a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffbd507e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffbd477018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbd477088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbd5c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffbd5c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffbd5c4a02>] zap_page_range+0x142/0x190
[<ffffffffbd5033a0>] ? futex_wake+0x90/0x180
[<ffffffffbd5bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffbdb1f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 13 be 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mysqld:13075]
Modules linked in: tcp_diag inet_diag talpa_vfshook(OE) talpa_pedconnector(OE) talpa_pedevice(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack xt_multiport iptable_filter vmw_vsock_vmci_transport vsock sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper nfit ablk_helper cryptd libnvdimm joydev pcspkr sg parport_pc parport vmw_vmci shpchp i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic ahci drm_kms_helper ata_piix libahci syscopyarea sysfillrect crct10dif_pclmul crct10dif_common sysimgblt fb_sys_fops libata ttm crc32c_intel serio_raw drm vmxnet3 vmw_pvscsi i2c_core dm_mirror dm_region_hash
dm_log dm_mod
CPU: 1 PID: 13075 Comm: mysqld Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff919e2c361fa0 ti: ffff919da4ac0000 task.ti: ffff919da4ac0000
RIP: 0010:[<ffffffff94d0a9fa>]  [<ffffffff94d0a9fa>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff919da4ac3bf0  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff919da4ac3bc0 RCX: 000000000000003e
RDX: 0000000000000003 RSI: 0000000000000002 RDI: 0000000000000282
RBP: ffff919da4ac3c38 R08: ffffffff95413320 R09: 00003ffffffff000
R10: ffff919e2a7da5e8 R11: 00007f052307c000 R12: ffffffff95318978
R13: ffffffff95318984 R14: ffffffff95318978 R15: ffffffff95318984
FS:  00007f05230c4700(0000) GS:ffff919e3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559654b89000 CR3: 0000000035b70000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff94c79c90>] ? leave_mm+0x110/0x110
[<ffffffff94c79c90>] ? leave_mm+0x110/0x110
[<ffffffff94c79c90>] ? leave_mm+0x110/0x110
[<ffffffff94d0ab0f>] smp_call_function_single+0x5f/0xa0
[<ffffffff94f4ec95>] ? cpumask_next_and+0x35/0x50
[<ffffffff94d0b0bb>] smp_call_function_many+0x22b/0x270
[<ffffffff94c79e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff94c79ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffff94dc33c7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff94dc4cc5>] tlb_finish_mmu+0x55/0x60
[<ffffffff94dc7b12>] zap_page_range+0x142/0x190
[<ffffffff94cd1f98>] ? wake_up_q+0x58/0x80
[<ffffffff94dc2a1d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff9532579b>] system_call_fastpath+0x22/0x27
Code: c0 96 01 00 48 89 de 48 03 14 c5 60 54 93 95 48 89 df e8 7a a0 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [prometheus:22200]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_lsm_pinned_10703(E) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock falcon_lsm_pinned_10404(E) sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon nfit pcspkr joydev sg libnvdimm vmw_vmci shpchp i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel mptspi serio_raw libata scsi_transport_spi mptscsih mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_lsm_serviceable]
CPU: 5 PID: 22200 Comm: prometheus Kdump: loaded Tainted: P            EL ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9b7a3a322f70 ti: ffff9b7acba94000 task.ti: ffff9b7acba94000
RIP: 0010:[<ffffffffa810aa2a>]  [<ffffffffa810aa2a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9b7acba97bf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9b7acba97bc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff9b7acba97c38 R08: ffffffffa88141a0 R09: 00003ffffffff000
R10: ffff9b7cda2ad878 R11: 0000000000000246 R12: ffffffffa834edb5
R13: ffff9b7acba97b78 R14: ffff9b7a39d29800 R15: 00000000a1cc8b98
FS:  00007f54df9c8700(0000) GS:ffff9b7cffd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f603834d000 CR3: 0000000421b8e000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa8079c90>] ? leave_mm+0x110/0x110
[<ffffffffa8079c90>] ? leave_mm+0x110/0x110
[<ffffffffa8079c90>] ? leave_mm+0x110/0x110
[<ffffffffa810ab3f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa834edb5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa810b0eb>] smp_call_function_many+0x22b/0x270
[<ffffffffa8079e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa8079ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa81c3487>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa81c4d85>] tlb_finish_mmu+0x55/0x60
[<ffffffffa81c7bd2>] zap_page_range+0x142/0x190
[<ffffffffa80d1fc8>] ? wake_up_q+0x58/0x80
[<ffffffffa81c2add>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa80d2010>] ? wake_up_state+0x20/0x20
[<ffffffffa8725a1b>] tracesys+0xa3/0xc9
Code: c0 96 01 00 48 89 de 48 03 14 c5 a0 54 d3 a8 48 89 df e8 6a a1 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [containerd-shim:3538]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ppdev snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec ac97_bus aesni_intel
vmw_balloon snd_seq lrw gf128mul snd_seq_device glue_helper ablk_helper cryptd snd_pcm snd_timer snd sg joydev pcspkr soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel nfit mptspi libnvdimm ata_piix scsi_transport_spi drm mptscsih e1000 libata serio_raw mptbase drm_panel_orientation_quirks
CPU: 1 PID: 3538 Comm: containerd-shim Tainted: G             L ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9b157412b150 ti: ffff9b18be730000 task.ti: ffff9b18be730000
RIP: 0010:[<ffffffff9a116f72>]  [<ffffffff9a116f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9b18be733c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9b18be733c00 RCX: 0000000000000001
RDX: ffffffff9a8136d0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9b18be733c80 R08: ffffffff9a8136c8 R09: 00003ffffffff000
R10: ffff9b18c207cf30 R11: 0000000000000246 R12: ffff9b18ed65acc0
R13: ffffffff9a0e103c R14: ffff9b18be733bb8 R15: ffffffff9a0e2e89
FS:  000000c4200f8890(0000) GS:ffff9b18ed640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c42019b010 CR3: 00000003ff628000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9a07e730>] ? leave_mm+0x120/0x120
[<ffffffff9a07e730>] ? leave_mm+0x120/0x120
[<ffffffff9a07e730>] ? leave_mm+0x120/0x120
[<ffffffff9a11708f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9a385005>] ? cpumask_next_and+0x35/0x50
[<ffffffff9a11763b>] smp_call_function_many+0x22b/0x270
[<ffffffff9a07e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9a07e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9a1f2ed8>] zap_page_range+0xd8/0x150
[<ffffffff9a0db948>] ? wake_up_q+0x58/0x80
[<ffffffff9a1ed3ed>] SyS_madvise+0x49d/0xab0
[<ffffffff9a792ed2>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 a0 10 d5 9a 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [mysqld:56338]
Modules linked in: xt_nat ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) binfmt_misc iptable_filter nls_utf8 isofs vmw_vsock_vmci_transport vsock raid0 coretemp crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod ata_piix crc_t10dif crct10dif_generic drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3
vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 56338 Comm: mysqld Tainted: G             L ------------ T 3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff898f5e9da100 ti: ffff8990e44c0000 task.ti: ffff8990e44c0000
RIP: 0010:[<ffffffff85f16a32>]  [<ffffffff85f16a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8990e44c3be0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8990e44c3bb0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8990e44c3c30 R08: ffffffff86613a40 R09: 00003ffffffff000
R10: ffff898edd8115f0 R11: 0000000000000202 R12: ffff898f5e9da100
R13: 0000000300000048 R14: ffff8990e44c3cc0 R15: ffffffff85fc862f
FS:  00007f3b66fcd700(0000) GS:ffff8991ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd87fff8a08 CR3: 0000000330c12000 CR4: 00000000000007e0
Call Trace:
[<ffffffff85e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff85e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff85e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff85f16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff86186a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff85f170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff85e7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85e7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff85fee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff85ff013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff85ff0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff85ff372a>] zap_page_range+0xea/0x130
[<ffffffff85edada8>] ? wake_up_q+0x58/0x80
[<ffffffff85fedc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff85e2b621>] ? __switch_to+0x151/0x580
[<ffffffff86595ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff86595ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff86595f92>] system_call_fastpath+0x25/0x2a
[<ffffffff86595ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 b5 86 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [docker:5910]
Modules linked in: vhost_net vhost macvtap macvlan hci_vhci bluetooth vfio_iommu_type1 vfio uinput uhid ppp_generic slhc lp loop btrfs raid6_pq xor veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) tcp_lp tcp_diag inet_diag vboxsf(OE) xt_CHECKSUM iptable_mangle cfg80211 ipt_MASQUERADE nf_nat_masquerade_ipv4 rfkill iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_timer snd iosf_mbi joydev vboxguest(OE) sg video pcspkr soundcore parport_pc i2c_piix4 parport sch_fq_codel ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci serio_raw libahci drm drm_panel_orientation_quirks libata crc32c_intel e1000 dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 5910 Comm: docker Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff88e43f3a0000 ti: ffff88e440904000 task.ti: ffff88e440904000
RIP: 0010:[<ffffffff84116b62>]  [<ffffffff84116b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff88e440907be0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: ffffffff840e108e RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff88e440907c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff88e595a6ad80 R11: 0000000000000202 R12: 000003804e0d5fe7
R13: ffffffff840e495c R14: ffff88e440907b80 R15: 0000000000000087
FS:  00007fd16d54f700(0000) GS:ffff88e59fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f283f3d99d0 CR3: 00000001f5fc4000 CR4: 00000000000206e0
Call Trace:
[<ffffffff8407ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8407ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8407ddd0>] ? leave_mm+0x120/0x120
[<ffffffff84116c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffff84386865>] ? cpumask_next_and+0x35/0x50
[<ffffffff8411722b>] smp_call_function_many+0x22b/0x270
[<ffffffff8407e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8407e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff841ee6c6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff841f018f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff841f02b3>] tlb_finish_mmu+0x23/0x30
[<ffffffff841f377a>] zap_page_range+0xea/0x130
[<ffffffff840daed8>] ? wake_up_q+0x58/0x80
[<ffffffff841edc6d>] SyS_madvise+0x49d/0xac0
[<ffffffff84799ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff84799ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff84799f92>] system_call_fastpath+0x25/0x2a
[<ffffffff84799ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 d5 84 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [docker:17513]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_timer snd soundcore sg pcspkr parport_pc iosf_mbi video i2c_piix4 parport ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea ahci sysfillrect sysimgblt fb_sys_fops ttm libahci ata_piix drm serio_raw crc32c_intel drm_panel_orientation_quirks libata e1000
dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 17513 Comm: docker Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9e0d6bda5280 ti: ffff9e0d83a60000 task.ti: ffff9e0d83a60000
RIP: 0010:[<ffffffffb9116b82>]  [<ffffffffb9116b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9e0d83a63be0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 00000000000000fb RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9e0d83a63c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9e0d9313c438 R11: 0000000000000202 R12: 0000000000000000
R13: 00000000000008fd R14: 0000000000000001 R15: 00003ffffffff000
FS:  00007fadebfff700(0000) GS:ffff9e0e1fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056034ff5e000 CR3: 00000000d75e6000 CR4: 00000000000206f0
Call Trace:
[<ffffffffb907ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb907ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb907ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb9116c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb9386825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb911724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb907e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb907e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb91ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb91f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb91f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffffb91f371a>] zap_page_range+0xea/0x130
[<ffffffffb90daef8>] ? wake_up_q+0x58/0x80
[<ffffffffb91edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffffb9799ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb9799ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb9799f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb9799ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 d5 b9 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [containerd:4049]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) tcp_lp xt_owner xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter sunrpc joydev ppdev parport_pc vboxguest(OE) iosf_mbi i2c_piix4 sg video parport pcspkr binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx virtio_net net_failover failover drm_kms_helper ahci virtio_pci ata_piix libahci syscopyarea sysfillrect sysimgblt virtio_ring fb_sys_fops serio_raw ttm drm crc32c_intel virtio libata e1000 drm_panel_orientation_quirks
dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 4049 Comm: containerd Kdump: loaded Tainted: G        W  OE  ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff89e8b8085280 ti: ffff89e8d4f5c000 task.ti: ffff89e8d4f5c000
RIP: 0010:[<ffffffffb2916b82>]  [<ffffffffb2916b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff89e8d4f5fbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: ffff89e8d4f5fbd8 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff89e8d4f5fc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff89e8d4dfaaf8 R11: 0000000000000202 R12: ffff89e8d4f5fb60
R13: 0000000000000001 R14: ffff89e92069acc0 R15: ffffffffb28d3d6a
FS:  00007feb118e7700(0000) GS:ffff89e920680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7060785000 CR3: 0000000481a06000 CR4: 00000000000206e0
Call Trace:
[<ffffffffb287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb2916c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb2b86825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb291724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb287e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb287e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb29ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb29f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb29f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffffb29f371a>] zap_page_range+0xea/0x130
[<ffffffffb28daef8>] ? wake_up_q+0x58/0x80
[<ffffffffb29edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffffb28daf40>] ? wake_up_state+0x20/0x20
[<ffffffffb2f99ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb2f99ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb2f9a226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 60 18 55 b3 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [go:6288]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device ppdev snd_pcm snd_timer iosf_mbi snd pcspkr soundcore vboxguest(OE) sg i2c_piix4 joydev parport_pc parport video ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx drm_kms_helper ahci syscopyarea
sysfillrect sr_mod sysimgblt cdrom fb_sys_fops ttm ata_generic pata_acpi drm crc32c_intel serio_raw libahci ata_piix e1000 libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 6288 Comm: go Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9946443bb180 ti: ffff9946433e8000 task.ti: ffff9946433e8000
RIP: 0010:[<ffffffffb6916b82>]  [<ffffffffb6916b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9946433ebbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: ffffffffb68e10ae RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9946433ebc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9946435b7e58 R11: 0000000000000202 R12: 0000007c13cfc520
R13: ffffffffb68e497c R14: ffff9946433ebb80 R15: ffff99464439b180
FS:  00007fc1eccd3700(0000) GS:ffff99475fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000174a00e31c20 CR3: 000000000323a000 CR4: 00000000000206e0
Call Trace:
[<ffffffffb687ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb687ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb687ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb6916c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb6b86825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb691724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb687e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb687e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb69ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb69f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb69f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffffb69f371a>] zap_page_range+0xea/0x130
[<ffffffffb68daef8>] ? wake_up_q+0x58/0x80
[<ffffffffb69edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffffb6f99ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb6f99ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb6f99f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb6f99ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 55 b7 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [prometheus:770]
Modules linked in: tcp_lp vboxsf(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter tun sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer ppdev joydev sg snd parport_pc vboxguest(OE) parport soundcore pcspkr video i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci ttm ata_piix libahci libata drm serio_raw e1000 crc32c_intel drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 770 Comm: prometheus Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8f6291965280 ti: ffff8f628f5d8000 task.ti: ffff8f628f5d8000
RIP: 0010:[<ffffffffb8116a32>]  [<ffffffffb8116a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8f628f5dbbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 0000000000000000 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff8f628f5dbc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff8f5fb5409360 R11: 0000000000000202 R12: 0000000000000002
R13: ffff8f629fd1acc0 R14: ffff8f60f83f1c00 R15: ffff8f60f83f3800
FS:  000000c00006c890(0000) GS:ffff8f629fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f267111f268 CR3: 00000003109be000 CR4: 00000000000206f0
Call Trace:
[<ffffffffb807ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb807ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb807ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb8116b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb8386a55>] ? cpumask_next_and+0x35/0x50
[<ffffffffb81170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffb807e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb807e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb81ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb81f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb81f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffb81f372a>] zap_page_range+0xea/0x130
[<ffffffffb80dada8>] ? wake_up_q+0x58/0x80
[<ffffffffb81edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffb80dadf0>] ? wake_up_state+0x20/0x20
[<ffffffffb8799ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb8799ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb8799f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb8799ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 d5 b8 48 89 df e8 97 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [dockerd:3005]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc snd_hda_codec_idt snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer ppdev snd parport_pc soundcore joydev sg i2c_piix4 video parport pcspkr ip_tables
xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ahci libahci crc32c_intel libata serio_raw e1000 dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 3005 Comm: dockerd Kdump: loaded Tainted: G               ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9094c363b180 ti: ffff909598340000 task.ti: ffff909598340000
RIP: 0010:[<ffffffffad916562>]  [<ffffffffad916562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff909598343be0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 00000000000000fb RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff909598343c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9094d5177948 R11: 0000000000000202 R12: 0000000000000000
R13: 00000000000008fd R14: 0000000000000001 R15: 00003ffffffff000
FS:  00007fce567fc700(0000) GS:ffff90959f500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9c66338e40 CR3: 0000000198130000 CR4: 00000000000006e0
Call Trace:
[<ffffffffad87e6a0>] ? leave_mm+0x120/0x120
[<ffffffffad87e6a0>] ? leave_mm+0x120/0x120
[<ffffffffad87e6a0>] ? leave_mm+0x120/0x120
[<ffffffffad91667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffadb85de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffad916c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffad87e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffad87e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffad9edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffad9ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffad9ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffad9f2dba>] zap_page_range+0xea/0x130
[<ffffffffad8daaa8>] ? wake_up_q+0x58/0x80
[<ffffffffad9ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffad8daaf0>] ? wake_up_state+0x20/0x20
[<ffffffffadf93ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffadf93ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffadf93f92>] system_call_fastpath+0x25/0x2a
[<ffffffffadf93ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 13 55 ae 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [xfsaild/dm-0:545]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [containerd-shim:27711]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel nfsv4 dns_resolver nfs ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 fscache rpcsec_gss_krb5 nf_conntrack_netlink br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype xt_set tun bridge stp llc xt_mark ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set nfnetlink dummy xt_comment iptable_nat nf_nat_ipv4 ebtable_filter ebtables nf_nat ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg parport_pc
parport vmw_balloon joydev pcspkr vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci ata_piix mptspi i2c_core libata scsi_transport_spi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptscsih vmxnet3 mptbase dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 27711 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff949967f9eeb0 ti: ffff9498af738000 task.ti: ffff9498af738000
RIP: 0010:[<ffffffff85d0779a>]  [<ffffffff85d0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9498af73bbf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9498af73bbc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff9498af73bc38 R08: ffffffff86413b60 R09: ffffdf82d0948340
R10: ffffdf82d0948340 R11: 0000000000000246 R12: ffff949967f9eeb0
R13: ffff949967f9eeb0 R14: ffff9498af73bba0 R15: 0000000000000001
FS:  000000c420031890(0000) GS:ffff94997fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c420091010 CR3: 00000003ee3f8000 CR4: 00000000000007f0
Call Trace:
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85d078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff85f4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff85d07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff85c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85c77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff85dc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff85dc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff85dc4a02>] zap_page_range+0x142/0x190
[<ffffffff85ccee38>] ? wake_up_q+0x58/0x80
[<ffffffff85dbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff8631f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8631f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8631f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff8631f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 93 86 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kubelet:2073]
Modules linked in: xt_statistic nf_conntrack_netlink nfnetlink xt_nat ipt_REJECT nf_reject_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc vmw_vsock_vmci_transport vsock overlay(T) sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq pcspkr ppdev vmw_balloon snd_seq_device snd_pcm snd_timer snd joydev soundcore sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit
crc32c_intel mptspi libnvdimm scsi_transport_spi mptscsih ata_piix e1000 mptbase serio_raw libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 2073 Comm: kubelet Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff91029bfce2a0 ti: ffff91029a810000 task.ti: ffff91029a810000
RIP: 0010:[<ffffffff92916612>]  [<ffffffff92916612>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff91029a813c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff91029a813c00 RCX: 0000000000000004
RDX: ffffffff93013cd0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff91029a813c80 R08: ffffffff93013cc8 R09: 00003ffffffff000
R10: ffff9102b60d41b0 R11: 0000000000000246 R12: ffff91029a813bc0
R13: 0000000000000000 R14: ffff9102bb69ac80 R15: ffffffff928de7f5
FS:  00007fd3a12c7700(0000) GS:ffff9102bb680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fce31bafab8 CR3: 0000000078962000 CR4: 00000000000007e0
Call Trace:
[<ffffffff9287e440>] ? leave_mm+0x110/0x110
[<ffffffff9287e440>] ? leave_mm+0x110/0x110
[<ffffffff9287e440>] ? leave_mm+0x110/0x110
[<ffffffff9291672f>] smp_call_function_single+0x5f/0xa0
[<ffffffff92b81845>] ? cpumask_next_and+0x35/0x50
[<ffffffff92916cdb>] smp_call_function_many+0x22b/0x270
[<ffffffff9287e608>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9287e67b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff929f0d28>] zap_page_range+0xd8/0x150
[<ffffffff928db5b8>] ? wake_up_q+0x58/0x80
[<ffffffff929eb22d>] SyS_madvise+0x49d/0xac0
[<ffffffff928db600>] ? wake_up_state+0x20/0x20
[<ffffffff92f8de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff92f8de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff92f8dede>] system_call_fastpath+0x25/0x2a
[<ffffffff92f8de21>] ? system_call_after_swapgs+0xae/0x146
Code: 89 de 48 03 14 c5 60 e8 54 93 48 89 df e8 b7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00