NMI watchdog: BUG: soft lockup - CPU#6 stuck for 27s! [node_exporter:14419]
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [node_exporter:4362]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev virtio_balloon i2c_piix4 pcspkr ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net virtio_scsi crct10dif_pclmul crct10dif_common crc32c_intel bochs_drm drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm serio_raw libata virtio_pci drm_panel_orientation_quirks virtio_ring
virtio
CPU: 5 PID: 4362 Comm: node_exporter Kdump: loaded Tainted: G             L ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
task: ffff8af096a130c0 ti: ffff8af090364000 task.ti: ffff8af090364000
RIP: 0010:[<ffffffff887113ea>]  [<ffffffff887113ea>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8af090367bf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8af090367bc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8af090367c38 R08: ffffffff88e13dc0 R09: ffffe2169d82fc40
R10: 0000000000004aac R11: 0000000000000246 R12: ffff8af0bffda000
R13: ffffffff887c11c6 R14: ffff8af090367c00 R15: 0000000000020200
FS:  000000c00004a090(0000) GS:ffff8af0bfd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000562f00dfa78c CR3: 0000000790298000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffff8867a4e0>] ? leave_mm+0x110/0x110
[<ffffffff8867a4e0>] ? leave_mm+0x110/0x110
[<ffffffff8867a4e0>] ? leave_mm+0x110/0x110
[<ffffffff887114ff>] smp_call_function_single+0x5f/0xa0
[<ffffffff88975cd5>] ? cpumask_next_and+0x35/0x50
[<ffffffff88711aab>] smp_call_function_many+0x22b/0x270
[<ffffffff8867a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8867a718>] flush_tlb_mm_range+0x68/0x140
[<ffffffff887e4687>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff887e5f85>] tlb_finish_mmu+0x55/0x60
[<ffffffff887e8dd2>] zap_page_range+0x142/0x190
[<ffffffff887e3c9d>] SyS_madvise+0x49d/0xac0
[<ffffffff886c5950>] ? hrtimer_get_res+0x50/0x50
[<ffffffff88d75ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 bc 34 89 48 89 df e8 4a b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [s1-agent:23838]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg i2c_piix4 parport_pc parport vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm ata_piix libahci nfit libata libnvdimm drm_panel_orientation_quirks
dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 23838 Comm: s1-agent Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/03/2018
task: ffff8ac4d4c15140 ti: ffff8ac4d7908000 task.ti: ffff8ac4d7908000
RIP: 0010:[<ffffffff86b1109a>]  [<ffffffff86b1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8ac4d790bbf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8ac4d790bbc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8ac4d790bc38 R08: ffffffff87212d80 R09: 00003ffffffff000
R10: ffff8ac625aa26c0 R11: 0000000000000206 R12: ffff8ac609a44a80
R13: 0000000000000000 R14: ffff8ac54a12d000 R15: ffff8ac609a44aa8
FS:  00007ffb4e8ec700(0000) GS:ffff8ac67fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001b771b0 CR3: 00000002ddbf2000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff86a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff86a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff86a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff86b111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff86d75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff86b1175b>] smp_call_function_many+0x22b/0x270
[<ffffffff86a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff86a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff86be4077>] tlb_flush_mmu.part.76+0x37/0xe0

NMI watchdog: BUG: soft lockup - CPU#11 stuck for 22s! [calico-node:24923]
Modules linked in: xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth iptable_raw xt_CT rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_addrtype xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipport ip_set_hash_ipportip ip_set dummy ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 devlink tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev sg joydev parport_pc parport i2c_piix4 virtio_balloon pcspkr br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net cirrus drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata drm serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 24923 Comm: calico-node Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9496eac85140 ti: ffff94a5b0a04000 task.ti: ffff94a5b0a04000
RIP: 0010:[<ffffffff9871109a>]  [<ffffffff9871109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff94a5b0a07bf0  EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff94a5b0a07bc0 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000282
RBP: ffff94a5b0a07c38 R08: ffffffff98e12d80 R09: 00003ffffffff000
R10: ffff94a6212d1440 R11: 0000000000000246 R12: ffff94a67ff9e000
R13: ffffffff987c0c06 R14: ffff94a5b0a07c00 R15: 0000000000020200
FS:  000000c00007e090(0000) GS:ffff94a63ecc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4236cb000 CR3: 0000000fb1b20000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff987111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff98975465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9871175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9867a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9867a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff987e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff987e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff987e87c2>] zap_page_range+0x142/0x190
[<ffffffff987e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff98d74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff98d74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff98d74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff98d74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 34 99 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [qemu-kvm:3750]
Modules linked in: vhost_net vhost macvtap macvlan nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter bridge stp llc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 snd_rawmidi aesni_intel snd_ac97_codec lrw ac97_bus vmw_balloon gf128mul glue_helper snd_seq ablk_helper cryptd snd_seq_device snd_pcm snd_timer pcspkr sg snd soundcore i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c
sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm drm ata_piix mptspi libata scsi_transport_spi i2c_core mptscsih e1000 mptbase fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3750 Comm: qemu-kvm Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff88030bfc4e70 ti: ffff8802e1668000 task.ti: ffff8802e1668000
RIP: 0010:[<ffffffff810f925a>]  [<ffffffff810f925a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8802e166bc40  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8802e166bbd0 RCX: 0000000000000003
RDX: ffffffff816afeb0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8802e166bc88 R08: ffffffff816afea8 R09: ffffea000b63eb00
R10: 0000000000003ec4 R11: 0000000000000206 R12: 0000000000000010
R13: ffffffff81057b36 R14: ffffffffffffff10 R15: 0000000000000282
FS:  00007fc357bff700(0000) GS:ffff880331600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc3c8a76ff0 CR3: 00000002e17c5000 CR4: 00000000003427f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106c300 ffff8802e166bd10
0000000000000003 0000000055ab89eb 0000000000000002 ffffffff8106c300
ffffffff8106c300 ffff8802e166bcb8 ffffffff810f935f ffff8802e166bcb8
Call Trace:
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff810f935f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813179a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f9903>] smp_call_function_many+0x223/0x260
[<ffffffff8106c4c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c599>] flush_tlb_mm_range+0x69/0x140
[<ffffffff811ac093>] tlb_flush_mmu.part.61+0x33/0xc0
[<ffffffff811ad385>] tlb_finish_mmu+0x55/0x60
[<ffffffff811af75a>] zap_page_range+0x13a/0x180
[<ffffffff8118f898>] ? lru_cache_add+0x8/0x10
[<ffffffff811ab7ec>] SyS_madvise+0x38c/0x8d0
[<ffffffff816965c9>] system_call_fastpath+0x16/0x1b
Code: 80 77 01 00 48 89 de 48 03 14 c5 40 57 ad 81 48 89 df e8 ea 28 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

BUG: soft lockup - CPU#5 stuck for 22s! [docker-containe:30479]
Modules linked in: veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw iptable_filter crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev parport_pc virtio_rng sg parport virtio_balloon i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console
virtio_net cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper crct10dif_pclmul crct10dif_common ttm ata_piix drm virtio_pci crc32c_intel virtio_ring libata serio_raw i2c_core virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 30479 Comm: docker-containe Tainted: G        W    L ------------   3.10.0-327.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.11.0-2.el7 04/01/2014
task: ffff880428f4dc00 ti: ffff8803ea8f8000 task.ti: ffff8803ea8f8000
RIP: 0010:[<ffffffff810e675a>]  [<ffffffff810e675a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8803ea8fbc40  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8803ea8fbc10 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8803ea8fbc88 R08: ffffffff8165ece0 R09: ffffea000ca909c0
R10: 00000000000043cb R11: 0000000000000246 R12: ffff880428f4dc00
R13: ffff880428f4dc00 R14: 0000000000000001 R15: ffff880428f4dc00
FS:  000000c42003bc90(0000) GS:ffff88043fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0e93e21000 CR3: 0000000362c41000 CR4: 00000000001406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff81065c90 ffff8803ea8fbd10
0000000000000003 0000000059a1352f 0000000000000002 ffffffff81065c90
ffffffff81065c90 ffff8803ea8fbcb8 ffffffff810e685f ffff8803ea8fbcb8
Call Trace:
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff810e685f>] smp_call_function_single+0x5f/0xa0
[<ffffffff812f2785>] ? cpumask_next_and+0x35/0x50
[<ffffffff810e6e03>] smp_call_function_many+0x223/0x260
[<ffffffff81065e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81065f26>] flush_tlb_mm_range+0x66/0x140
[<ffffffff81192563>] tlb_flush_mmu.part.54+0x33/0xc0
[<ffffffff811930f5>] tlb_finish_mmu+0x55/0x60
[<ffffffff8119568a>] zap_page_range+0x12a/0x170
[<ffffffff81191db4>] SyS_madvise+0x394/0x820
[<ffffffff810e55a0>] ? SyS_futex+0x80/0x180
[<ffffffff81645909>] system_call_fastpath+0x16/0x1b
Code: 40 52 01 00 48 89 de 48 03 14 c5 e0 b8 a5 81 48 89 df e8 da fc 21 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [agent_service:18619]
Modules linked in: binfmt_misc sysmon_edr(OE) netfilter_edr(OE) ppdev sg i2c_piix4 joydev virtio_balloon parport_pc parport pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net serio_raw cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 18619 Comm: agent_service Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff9cfd8e7c1040 ti: ffff9d00d023c000 task.ti: ffff9d00d023c000
RIP: 0010:[<ffffffffaed1109a>]  [<ffffffffaed1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9d00d023fbf0  EFLAGS: 00000202
RAX: ffff9d00d963c600 RBX: 00007fccc9f60000 RCX: ffff9d00d963c610
RDX: ffff9d00dec5b700 RSI: ffff9d00d023fbf0 RDI: ffff9d00d023fbf0
RBP: ffff9d00d023fc38 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9cfd7bb0be60 R11: 0000000000000206 R12: ffff9cfb5079b7a0
R13: ffffffffaedf6e08 R14: ffff9d00d023fb78 R15: ffffffffaedc588e
FS:  00007fccbf24d700(0000) GS:ffff9d00ded00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fccc86413a0 CR3: 000000081075c000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaed111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffaef75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffaed1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffaec7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaec7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaede4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffaede5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffaede87c2>] zap_page_range+0x142/0x190
[<ffffffffaedebc6d>] ? handle_mm_fault+0x39d/0x9b0
[<ffffffffaede368d>] SyS_madvise+0x49d/0xac0
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaf374d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaf374ddb>] system_call_fastpath+0x22/0x27
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 94 af 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [evdefer/2:45966]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_15508(E) falcon_lsm_pinned_15402(E) falcon_lsm_pinned_15309(E) falcon_lsm_pinned_15110(E) falcon_lsm_pinned_15003(E) falcon_lsm_pinned_14812(E) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter binfmt_misc falcon_lsm_pinned_14810(E) tcp_lp fuse falcon_lsm_pinned_14713(E) vmw_vsock_vmci_transport vsock intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr sg vmw_vmci i2c_piix4 shpchp nfit libnvdimm parport_pc parport nfsd nfs_acl lockd grace auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul
crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ata_piix ttm serio_raw scsi_transport_spi drm mptscsih vmxnet3 libata mptbase i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 3 PID: 45966 Comm: evdefer/2 Tainted: P            EL ------------   3.10.0-514.26.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8803fa2cedd0 ti: ffff88053fee0000 task.ti: ffff88053fee0000
RIP: 0010:[<ffffffff810f98ea>]  [<ffffffff810f98ea>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff88053fee3c80  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88053fee3c50 RCX: 0000000000000007
RDX: ffffffff816b1970 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff88053fee3cd0 R08: ffffffff816b1968 R09: ffffffff811bc85c
R10: ffff8805296d9a80 R11: ffffea00005e3f80 R12: ffffffff811fe8c7
R13: ffff88053fee3c20 R14: ffff88049ef0b310 R15: ffff88018441ee70
FS:  00007f08e445c700(0000) GS:ffff8805296c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f08ef9bf000 CR3: 0000000002280000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106c430 ffff88053fee3d58
0000000000000003 000000001dd99590 ffff88002969fdf8 0000000000000006
ffffffff8106c430 ffffffff8106c430 ffff88053fee3d00 ffffffff810f99ef
Call Trace:
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff810f99ef>] smp_call_function_single+0x5f/0xa0
[<ffffffff813183b5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f9f93>] smp_call_function_many+0x223/0x260
[<ffffffff8106c5f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c6c9>] flush_tlb_mm_range+0x69/0x140
[<ffffffff811ac733>] tlb_flush_mmu.part.61+0x33/0xc0

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mysqld:6847]
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [mysqld:6846]
Modules linked in: ip_set_hash_net xt_set nf_log_ipv4 nf_log_common xt_LOG xt_limit ip_set_hash_ip ip_set nfnetlink xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables nfsv3 nfs fscache vmw_vsock_vmci_transport vsock nf_conntrack_ipv4 nf_defrag_ipv4 xt_iprange xt_conntrack nf_conntrack libcrc32c iptable_filter dm_mirror dm_region_hash dm_log dm_mod ppdev vmw_balloon edac_mce_amd joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod crct10dif_common cdrom serio_raw ata_generic pata_acpi vmwgfx floppy drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix vmxnet3 libata mptspi scsi_transport_spi drm_panel_orientation_quirks mptscsih mptbase
CPU: 3 PID: 6846 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff9c187ac30000 ti: ffff9c1a26af8000 task.ti: ffff9c1a26af8000
RIP: 0010:[<ffffffffac11117e>]  [<ffffffffac11117e>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff9c1a26afbbf0  EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffff9c187ac30000 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff9c1a26afbc38 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9c19f8eff438 R11: 00007f82941c8000 R12: ffff9c187ac30000
R13: ffffffffac6f5587 R14: ffff9c1a26afbcd0 R15: 0000000000000004
FS:  00007f8294210700(0000) GS:ffff9c1a3fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fccf3b6d000 CR3: 0000000321bf6000 CR4: 00000000000007e0
Call Trace:
[<ffffffffac07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffac07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffac07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffac11128f>] smp_call_function_single+0x5f/0xa0
[<ffffffffac3756d5>] ? cpumask_next_and+0x35/0x50
[<ffffffffac11183b>] smp_call_function_many+0x22b/0x270
[<ffffffffac07a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffac07a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffac1e42e7>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffac1e5be5>] tlb_finish_mmu+0x55/0x60
[<ffffffffac1e8a32>] zap_page_range+0x142/0x190
[<ffffffffac1e38fd>] SyS_madvise+0x49d/0xac0

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 29s! [java:26616]
Modules linked in: drbd_transport_tcp(OE) drbd(OE) tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc sg pcspkr i2c_piix4 joydev virtio_balloon sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_gpu drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm virtio_scsi virtio_net drm_panel_orientation_quirks ata_piix libata floppy serio_raw virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 26616 Comm: java Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffff8f661bf20000 ti: ffff8f6a193f0000 task.ti: ffff8f6a193f0000
RIP: 0010:[<ffffffff94b1109e>]  [<ffffffff94b1109e>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff8f6a193f3bf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8f6a193f3bc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8f6a193f3c38 R08: ffffffff95213000 R09: 00003ffffffff000
R10: ffff8f6709b32510 R11: 00007f8b3baf1000 R12: ffff8f6a3ffce000
R13: ffffffff94bc0c06 R14: ffff8f6a193f3c00 R15: 0000000000020200
FS:  00007f8b3bbee700(0000) GS:ffff8f6a3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8b40287a78 CR3: 000000042e5f4000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94b111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff94d75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff94b1175b>] smp_call_function_many+0x22b/0x270
[<ffffffff94a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff94a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff94be4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff94be5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff94be87c2>] zap_page_range+0x142/0x190
[<ffffffff94be368d>] SyS_madvise+0x49d/0xac0
[<ffffffff95174d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff95174d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff95174ddb>] system_call_fastpath+0x22/0x27
[<ffffffff95174d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 89 de 48 03 14 c5 60 b9 74 95 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [java:142686]
Modules linked in: netfilter_edr(OE) tcp_lp fuse tun devlink ip_set nfnetlink bridge stp llc sunrpc ppdev sg virtio_balloon parport_pc parport i2c_piix4 joydev pcspkr binfmt_misc xfs libcrc32c sr_mod cdrom virtio_console virtio_blk virtio_scsi virtio_net ata_generic pata_acpi floppy cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata drm serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 142686 Comm: java Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff931196145140 ti: ffff931188b98000 task.ti: ffff931188b98000
RIP: 0010:[<ffffffffbe31109a>]  [<ffffffffbe31109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff931188b9bbf0  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff931188b9bbc0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff931188b9bc38 R08: ffffffffbea13a08 R09: 00003ffffffff000
R10: ffff93198eb8caf8 R11: 00007fe8389f2000 R12: ffff9319bffbe000
R13: ffffffffbe3c0c06 R14: ffff931188b9bc00 R15: 0000000000020200
FS:  00007fe838aef700(0000) GS:ffff93199ce00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe8c597ec80 CR3: 0000000002466000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffbe27a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbe27a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbe27a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbe3111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffbe575465>] ? cpumask_next_and+0x35/0x50
[<ffffffffbe31175b>] smp_call_function_many+0x22b/0x270
[<ffffffffbe27a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbe27a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbe3e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffbe3e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffbe3e87c2>] zap_page_range+0x142/0x190
[<ffffffffbe3e368d>] SyS_madvise+0x49d/0xac0
[<ffffffffbe97505b>] tracesys+0xa3/0xc9
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 f4 be 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 33s! [kworker/0:3:7131]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 34s! [mysqld:7155]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper
 syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul mptspi crct10dif_common crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 7155 Comm: mysqld Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff881ef1f03f40 ti: ffff880501eb8000 task.ti: ffff880501eb8000
RIP: 0010:[<ffffffff810fcf5a>]  [<ffffffff810fcf5a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff880501ebbbf0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff880501ebbbc0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff880501ebbc40 R08: ffffffff816dcb60 R09: 00007f2d2414d000
R10: 00007f2d28bd6240 R11: 00007f2d2410b000 R12: ffffffff00a2bd52
R13: 0000000000007530 R14: ffffffff8158f494 R15: ffff880501ebbb80
FS:  00007f2d24152700(0000) GS:ffff881f8fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fef0f10f000 CR3: 0000001efcbc8000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 [<ffffffff810709d0>] ? leave_mm+0x110/0x110
 [<ffffffff810709d0>] ? leave_mm+0x110/0x110
 [<ffffffff810709d0>] ? leave_mm+0x110/0x110
 [<ffffffff810fd06f>] smp_call_function_single+0x5f/0xa0
 [<ffffffff813288a5>] ? cpumask_next_and+0x35/0x50
 [<ffffffff810fd61b>] smp_call_function_many+0x22b/0x270
 [<ffffffff81070b98>] native_flush_tlb_others+0xb8/0xc0
 [<ffffffff81070c68>] flush_tlb_mm_range+0x68/0x140
 [<ffffffff811b18c3>] tlb_flush_mmu.part.63+0x33/0xc0
 [<ffffffff811b2be5>] tlb_finish_mmu+0x55/0x60
 [<ffffffff811b516a>] zap_page_range+0x13a/0x180
 [<ffffffff811b0f8d>] SyS_madvise+0x3cd/0x9c0
 [<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 80 95 01 00 48 89 de 48 03 14 c5 a0 10 b2 81 48 89 df e8 7a 03 24 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:82027]
Modules linked in: af_packet_diag netlink_diag vsock_diag sctp_diag sctp tcp_diag udp_diag inet_diag unix_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink binfmt_misc bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec ppdev ac97_bus aesni_intel lrw snd_seq gf128mul btusb glue_helper ablk_helper cryptd btrtl btbcm btintel snd_seq_device bluetooth snd_pcm snd_timer joydev pcspkr snd sg rfkill soundcore nfit i2c_piix4 vmw_vmci
shpchp libnvdimm parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi drm mptscsih ata_piix crct10dif_pclmul crct10dif_common e1000 crc32c_intel libata mptbase serio_raw i2c_core
CPU: 1 PID: 82027 Comm: java Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9b2247fb8fd0 ti: ffff9b22a6f44000 task.ti: ffff9b22a6f44000
RIP: 0010:[<ffffffff9d90779a>]  [<ffffffff9d90779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9b22a6f47bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9b22a6f47bc0 RCX: 0000000000000004
RDX: ffffffff9e0138f0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9b22a6f47c38 R08: ffffffff9e0138e8 R09: ffff9b1f32e5e438
R10: 0000000000000008 R11: 00007f864b8fc000 R12: ffff9b229a7e1d00
R13: 00000000bc1d6a46 R14: 00007f864b8fc000 R15: ffff9b2247fb8fd0
FS:  00007f864b9f9700(0000) GS:ffff9b22ad640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1d74962000 CR3: 00000003e32ee000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9d876e50>] ? leave_mm+0x110/0x110
[<ffffffff9d876e50>] ? leave_mm+0x110/0x110
[<ffffffff9d876e50>] ? leave_mm+0x110/0x110
[<ffffffff9d9078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9db4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff9d907e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff9d877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9d877088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9d9c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff9d9c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff9d9c4a02>] zap_page_range+0x142/0x190

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:110144]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371
snd_rawmidi snd_ac97_codec ac97_bus snd_seq ppdev snd_seq_device vmw_balloon snd_pcm pcspkr joydev snd_timer snd i2c_piix4 soundcore vmw_vmci sg shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata mptspi scsi_transport_spi mptscsih serio_raw mptbase e1000 i2c_core
CPU: 1 PID: 110144 Comm: java Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff880305b19fa0 ti: ffff8801f7b6c000 task.ti: ffff8801f7b6c000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8801f7b6fc20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8801f7b6fbf0 RCX: 0000000000000004
RDX: ffffffff816cebb0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8801f7b6fc70 R08: ffffffff816ceba8 R09: ffffea0000a58480
R10: ffffea0000a58480 R11: 0000000000000246 R12: ffff88037fbd8000
R13: ffffffff8118caf6 R14: ffff8801f7b6fc30 R15: 0000000000020200
FS:  00007f0a8657f700(0000) GS:ffff880370240000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0a557a8000 CR3: 00000001bddb0000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff8801f7b6fcf8
0000000000000003 00000000d64c985f ffff8801f7b6fc88 0000000000000003
ffffffff8106e4a0 ffffffff8106e4a0 ffff8801f7b6fca0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b08fa>] zap_page_range+0x13a/0x180
[<ffffffff811ac71d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff8111f384>] ? __audit_syscall_entry+0xb4/0x110
[<ffffffff816b51d2>] tracesys+0xdd/0xe2
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

BUG: soft lockup - CPU#1 stuck for 23s! [server.rb:286:57944]
Modules linked in: tcp_lp nls_utf8 isofs bnep bluetooth rfkill fuse tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables lp vmhgfs(OE) vmw_vsock_vmci_transport vsock ipt_REJECT xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_CHECKSUM iptable_mangle dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw snd_seq gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm snd_timer snd pcspkr vmw_vmci ppdev soundcore vmw_balloon sg i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic sr_mod cdrom ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw ttm ata_piix libata drm mptspi scsi_transport_spi mptscsih e1000 mptbase i2c_core floppy
CPU: 1 PID: 57944 Comm: server.rb:286 Tainted: G           OEL ------------   3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
task: ffff8801a78b1700 ti: ffff88012b8c8000 task.ti: ffff88012b8c8000
RIP: 0010:[<ffffffff810e675a>]  [<ffffffff810e675a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff88012b8cbc40  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88012b8cbc10 RCX: 0000000000000001
RDX: ffffffff8165e7f0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff88012b8cbc88 R08: ffffffff8165e7e8 R09: 00007f4be95fe000
R10: 0000000000001000 R11: 0000000000000202 R12: ffff88012b8cbbc0
R13: ffff88012b8cbd63 R14: ffffffff812fc122 R15: ffff88012b8cbbb8
FS:  00007f4beaeff700(0000) GS:ffff880236620000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f315b6b4750 CR3: 00000002080fc000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff81065c90 ffff88012b8cbd10
0000000000000003 000000008f112638 0000000000000000 ffffffff81065c90
ffffffff81065c90 ffff88012b8cbcb8 ffffffff810e685f ffff88012b8cbcb8
Call Trace:
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff810e685f>] smp_call_function_single+0x5f/0xa0
[<ffffffff812f2785>] ? cpumask_next_and+0x35/0x50
[<ffffffff810e6e03>] smp_call_function_many+0x223/0x260
[<ffffffff81065e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81065f26>] flush_tlb_mm_range+0x66/0x140
[<ffffffff81192563>] tlb_flush_mmu.part.54+0x33/0xc0
[<ffffffff811930f5>] tlb_finish_mmu+0x55/0x60
[<ffffffff8119568a>] zap_page_range+0x12a/0x170
[<ffffffff81191db4>] SyS_madvise+0x394/0x820
[<ffffffff8101360b>] ? __switch_to+0x17b/0x4b0
[<ffffffff8163a2b8>] ? __schedule+0x2d8/0x900
[<ffffffff81645909>] system_call_fastpath+0x16/0x1b
Code: 40 52 01 00 48 89 de 48 03 14 c5 e0 b8 a5 81 48 89 df e8 da fc 21 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [SystemLogFlush:1503]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 iosf_mbi ppdev crc32_pclmul snd_ens1371 btusb btrtl snd_rawmidi ghash_clmulni_intel btbcm btintel snd_ac97_codec bluetooth ac97_bus vmw_balloon snd_seq aesni_intel snd_seq_device lrw snd_pcm gf128mul glue_helper ablk_helper cryptd i2c_piix4 nfit rfkill joydev pcspkr snd_timer sg snd vmw_vmci soundcore libnvdimm shpchp parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix ttm mptspi crct10dif_pclmul crct10dif_common libata crc32c_intel drm scsi_transport_spi mptscsih serio_raw e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 1503 Comm: SystemLogFlush Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff894232d3eeb0 ti: ffff894321b7c000 task.ti: ffff894321b7c000
RIP: 0010:[<ffffffffaed0779a>]  [<ffffffffaed0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff894321b7fbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff894321b7fbc0 RCX: 0000000000000001
RDX: ffffffffaf413170 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff894321b7fc38 R08: ffffffffaf413168 R09: ffff894236a69950
R10: ffff89433fbd5f40 R11: ffffffffffffffff R12: ffff894321b7fbe8
R13: ffff8943349f0ca0 R14: 0000000000000062 R15: 0000000001d2dd11
FS:  00007f21c2bfb700(0000) GS:ffff894337740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9f76d2c9e5 CR3: 00000000bb52c000 CR4: 00000000003607e0
Call Trace:
[<ffffffffaec76e50>] ? leave_mm+0x110/0x110
[<ffffffffaec76e50>] ? leave_mm+0x110/0x110
[<ffffffffaec76e50>] ? leave_mm+0x110/0x110
[<ffffffffaed078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffaef4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffaed07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffaec77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaec77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaedc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffaedc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffaedc4a02>] zap_page_range+0x142/0x190
[<ffffffffaee1a1f3>] ? do_sync_write+0x93/0xe0
[<ffffffffaedbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffaee1ad92>] ? vfs_write+0x182/0x1f0
[<ffffffffaf31f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 93 af 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [vmtoolsd:834]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmhgfs(OE) vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 snd_rawmidi ghash_clmulni_intel snd_ac97_codec ppdev ac97_bus snd_seq snd_seq_device aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_pcm snd_timer pcspkr joydev snd sg soundcore nfit libnvdimm parport_pc parport shpchp i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm ata_piix libata mptspi scsi_transport_spi
e1000 mptscsih mptbase i2c_core
CPU: 1 PID: 834 Comm: vmtoolsd Kdump: loaded Tainted: G           OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff89a1ae225ee0 ti: ffff89a03ac84000 task.ti: ffff89a03ac84000
RIP: 0010:[<ffffffff9f90779a>]  [<ffffffff9f90779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff89a03ac87c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff89a03ac87c00 RCX: 0000000000000001
RDX: ffffffffa0013170 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff89a03ac87c78 R08: ffffffffa0013168 R09: ffffffff9f9d35bc
R10: ffff89a1b565ba40 R11: fffff58a88b49240 R12: fffff58a88acf770
R13: ffff89a02e1fa000 R14: ffffffff9f9d2058 R15: ffff89a03ac87bb0
FS:  00007fb12ae60880(0000) GS:ffff89a1b5640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb12ae57000 CR3: 000000022ca5e000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9f876e50>] ? leave_mm+0x110/0x110
[<ffffffff9f876e50>] ? leave_mm+0x110/0x110
[<ffffffff9f876e50>] ? leave_mm+0x110/0x110
[<ffffffff9f9078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9fb4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff9f907e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff9f877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9f877088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9f9c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff9f9c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff9f9ca624>] unmap_region+0xf4/0x140

NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [sdatad:305214]
Modules linked in: dattobd(OE) sfs(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc nfit libnvdimm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom
crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm mptspi scsi_transport_spi mptscsih ahci vmxnet3 libahci mptbase drm_panel_orientation_quirks ata_piix libata dm_mirror dm_region_hash dm_log dm_mod [last unloaded: dattobd]
CPU: 15 PID: 305214 Comm: sdatad Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff98692b74e180 ti: ffff986919f08000 task.ti: ffff986919f08000
RIP: 0010:[<ffffffff89d1109e>]  [<ffffffff89d1109e>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff986919f0bbf0  EFLAGS: 00000202
RAX: 0000000000000014 RBX: ffff986919f0bbc0 RCX: 000000000000002c
RDX: 00000000000fffff RSI: 0000000000000014 RDI: 0000000000000282
RBP: ffff986919f0bc38 R08: ffffffff8a413f00 R09: 00003ffffffff000
R10: ffff986d27761bd8 R11: 0000000000000206 R12: ffff986919f0bbb0
R13: 0000000000000001 R14: ffff986d2cb1ac00 R15: ffff98692b74e1e8
FS:  00007f781ce7a700(0000) GS:ffff986d2cb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc44caaa000 CR3: 0000000773a3c000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff89c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff89c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff89c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff89d111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff89f75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff89d1175b>] smp_call_function_many+0x22b/0x270
[<ffffffff89c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff89c7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff89de4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff89de5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff89de87c2>] zap_page_range+0x142/0x190
[<ffffffff89de368d>] SyS_madvise+0x49d/0xac0
[<ffffffff8a374ddb>] system_call_fastpath+0x22/0x27
Code: 48 89 de 48 03 14 c5 60 b9 94 8a 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [spades-hammer:5712]
Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxsf(OE) sunrpc vboxguest(OE) snd_intel8x0 vmwgfx snd_ac97_codec ttm ac97_bus ppdev drm_kms_helper snd_seq joydev syscopyarea snd_seq_device sysfillrect snd_pcm sysimgblt iosf_mbi fb_sys_fops snd_timer snd drm sg soundcore i2c_piix4 pcspkr parport_pc video parport i2c_core ip_tables
xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod crct10dif_common cdrom ata_generic pata_acpi ata_piix libata crc32c_intel serio_raw e1000 dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 5712 Comm: spades-hammer Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.3.3.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff957741033f40 ti: ffff957757b2c000 task.ti: ffff957757b2c000
RIP: 0010:[<ffffffffa3f07ace>]  [<ffffffffa3f07ace>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff957757b2fbf0  EFLAGS: 00010202
RAX: ffff9576aa145500 RBX: ffff95775ffd5800 RCX: ffff9576aa145510
RDX: ffff95775fc19680 RSI: ffff957757b2fbf0 RDI: ffff957757b2fbf0
RBP: ffff957757b2fc38 R08: 0000000000000001 R09: ffff95770e4d9af8
R10: 0000000000000006 R11: 0000000000000202 R12: 000280da00000000
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fc9c03fd700(0000) GS:ffff95775fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc9c7c0c000 CR3: 0000000035c94000 CR4: 00000000000206e0
Call Trace:
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3f07bdf>] smp_call_function_single+0x5f/0xa0
[<ffffffffa414a6b5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa3f0818b>] smp_call_function_many+0x22b/0x270
[<ffffffffa3e772d8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa3e77348>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa3fc07d7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa3fc1fb5>] tlb_finish_mmu+0x55/0x60
[<ffffffffa3fc4d42>] zap_page_range+0x142/0x190
[<ffffffffa3fbfe3d>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa452077b>] ? system_call_after_swapgs+0xc8/0x160
[<ffffffffa452076f>] ? system_call_after_swapgs+0xbc/0x160
[<ffffffffa452082f>] system_call_fastpath+0x1c/0x21
[<ffffffffa452077b>] ? system_call_after_swapgs+0xc8/0x160
Code: 48 89 de 48 03 14 c5 e0 52 b3 a4 48 89 df e8 5a 88 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [go-jxhttp_idip:2677]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr nfit sg libnvdimm vmw_vmci i2c_piix4 shpchp parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom
crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel libata e1000 mptspi scsi_transport_spi i2c_core mptscsih mptbase serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2677 Comm: go-jxhttp_idip Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9e1fdd366eb0 ti: ffff9e1fb2644000 task.ti: ffff9e1fb2644000
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9e1fdd366eb0 ti: ffff9e1fb2644000 task.ti: ffff9e1fb2644000
RSP: 0018:ffff9e1fb2647bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9e1fb2647bc0 RCX: 0000000000000003
RDX: ffffffff87013670 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9e1fb2647c38 R08: ffffffff87013668 R09: ffff9e1c7bbebe60
R10: 00007f549ca96db8 R11: 0000000000000246 R12: ffff9e1fed6d8b00
R13: 0000000300000000 R14: ffff9e1fed618b00 R15: ffff9e1d398a3800
FS:  00007f549ca97700(0000) GS:ffff9e1fed6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007efe4d5d6e70 CR3: 00000003f1fc2000 CR4: 00000000001607e0
Call Trace:
[<ffffffff86876e50>] ? leave_mm+0x110/0x110
[<ffffffff86876e50>] ? leave_mm+0x110/0x110
[<ffffffff86876e50>] ? leave_mm+0x110/0x110
[<ffffffff869078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff86b4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff86907e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff86877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff86877088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff869c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff869c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff869c4a02>] zap_page_range+0x142/0x190
[<ffffffff869bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff868f76e2>] ? ktime_get_ts64+0x52/0xf0
[<ffffffff86f1f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 53 87 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [replexec-71:42505]
Modules linked in: ipt_rpfilter xt_set iptable_raw ip_set_hash_ip ip_set_hash_net ip_set ipip tunnel4 ip_tunnel xt_statistic xt_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc coretemp kvm_intel ppdev kvm irqbypass crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_vs_ftp nf_nat ip_vs_sed ip_vs_nq
ip_vs_sh ip_vs_dh ip_vs_lblcr ip_vs_lblc ip_vs_wrr ip_vs_rr ip_vs_wlc ip_vs_lc ip_vs nf_conntrack ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi e1000 libata mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 42505 Comm: replexec-71 Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff996e72b330c0 ti: ffff996e134c0000 task.ti: ffff996e134c0000
RIP: 0010:[<ffffffffa291109a>]  [<ffffffffa291109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff996e134c3bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff996e134c3bc0 RCX: 0000000000000001
RDX: ffffffffa3012d90 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff996e134c3c38 R08: ffffffffa3012d88 R09: 00003ffffffff000
R10: ffff996dde96cca8 R11: 00007f1ad5a39000 R12: ffff996e134c3bb0
R13: 0000000000000001 R14: ffff996df5de5c00 R15: ffff996e72b33128
FS:  00007f1ad6238700(0000) GS:ffff996ef7ac0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efe4ce7d000 CR3: 00000000adf44000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa287a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa287a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa287a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa29111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa2b75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffa291175b>] smp_call_function_many+0x22b/0x270
[<ffffffffa287a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa287a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa29e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffa29e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffa29e87c2>] zap_page_range+0x142/0x190
[<ffffffffa2a1b762>] ? kmem_cache_free+0x1e2/0x200
[<ffffffffa29e368d>] SyS_madvise+0x49d/0xac0
[<ffffffffa2f74ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 54 a3 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [DOM Worker:2915]
Modules linked in: fuse nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac edac_core iosf_mbi crc32_pclmul snd_ens1371 snd_rawmidi ghash_clmulni_intel snd_ac97_codec ppdev ac97_bus snd_seq aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm vmw_balloon pcspkr joydev snd_timer
sg snd vmw_vmci soundcore nfit i2c_piix4 shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ttm crct10dif_common crc32c_intel drm ata_piix libata serio_raw mptspi scsi_transport_spi mptscsih mptbase e1000 i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 2915 Comm: DOM Worker Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8800bad89fa0 ti: ffff8801d96c8000 task.ti: ffff8801d96c8000
RIP: 0010:[<ffffffff810fcf5a>]  [<ffffffff810fcf5a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8801d96cbbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8801d96cbbc0 RCX: 0000000000000001
RDX: ffffffff816dc8f0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8801d96cbc40 R08: ffffffff816dc8e8 R09: ffffea00088cf0c0
R10: ffffea00088cf0c0 R11: 0000000000000206 R12: ffff88023ffd8000
R13: ffffffff81191016 R14: ffff8801d96cbc00 R15: 0000000000020200
FS:  00007fdb50cff700(0000) GS:ffff880235640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdb4addb000 CR3: 00000000b163e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810fd06f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813288a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810fd61b>] smp_call_function_many+0x22b/0x270
[<ffffffff81070b98>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81070c68>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811b18c3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b2be5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b516a>] zap_page_range+0x13a/0x180
[<ffffffff811b0f8d>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816bb521>] ? __do_page_fault+0x171/0x450
[<ffffffff816bb54f>] ? __do_page_fault+0x19f/0x450
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 80 95 01 00 48 89 de 48 03 14 c5 a0 10 b2 81 48 89 df e8 7a 03 24 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

BUG: soft lockup - CPU#5 stuck for 23s! [node_exporter:1065]
Modules linked in: binfmt_misc tcp_diag inet_diag xt_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat vmw_vsock_vmci_transport nf_conntrack vsock xt_CHECKSUM iptable_mangle ext4 mbcache jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr sg parport_pc vmw_vmci parport shpchp i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ttm crct10dif_pclmul mptspi crct10dif_common crc32c_intel scsi_transport_spi ata_piix drm serio_raw mptscsih libata vmxnet3
mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 1065 Comm: node_exporter Tainted: G             L ------------   3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880421c7e780 ti: ffff8800356c4000 task.ti: ffff8800356c4000
RIP: 0010:[<ffffffff810e675e>]  [<ffffffff810e675e>] generic_exec_single+0xfe/0x1a0
RSP: 0018:ffff8800356c7c40  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8800356c7c10 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8800356c7c88 R08: ffffffff8165f1e0 R09: 000000c0006dc000
R10: 0000000000002000 R11: 0000000000000246 R12: 0000000000000000
R13: ffff88043fd54780 R14: 0000000500000003 R15: ffff88043fcd4780
FS:  0000000001116110(0063) GS:ffff88043fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc585bac000 CR3: 0000000425eb4000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff81065c90 ffff8800356c7d10
0000000000000003 00000000f90fb311 0000000000000004 ffffffff81065c90
ffffffff81065c90 ffff8800356c7cb8 ffffffff810e685f ffff8800356c7cb8
Call Trace:
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff810e685f>] smp_call_function_single+0x5f/0xa0
[<ffffffff812f2785>] ? cpumask_next_and+0x35/0x50
[<ffffffff810e6e03>] smp_call_function_many+0x223/0x260
[<ffffffff81065e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81065f26>] flush_tlb_mm_range+0x66/0x140
[<ffffffff81192563>] tlb_flush_mmu.part.54+0x33/0xc0
[<ffffffff811930f5>] tlb_finish_mmu+0x55/0x60
[<ffffffff8119568a>] zap_page_range+0x12a/0x170
[<ffffffff81191db4>] SyS_madvise+0x394/0x820
[<ffffffff810e55a0>] ? SyS_futex+0x80/0x180
[<ffffffff81645909>] system_call_fastpath+0x16/0x1b
Code: 48 89 de 48 03 14 c5 e0 b8 a5 81 48 89 df e8 da fc 21 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 80 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [java:5506]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg virtio_balloon virtio_rng parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_console virtio_scsi virtio_net crct10dif_pclmul crct10dif_common crc32c_intel ata_generic serio_raw
pata_acpi floppy qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix virtio_pci libata virtio_ring drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 5506 Comm: java Kdump: loaded Not tainted 3.10.0-957.12.2.el7.x86_64 #1
Hardware name: oVirt oVirt Node, BIOS 1.11.0-2.el7 04/01/2014
task: ffff9a4721251040 ti: ffff9a4731284000 task.ti: ffff9a4731284000
RIP: 0010:[<ffffffff91d1136a>]  [<ffffffff91d1136a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9a4731287bf0  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffff9a4731287bc0 RCX: 0000000000000030
RDX: 000000000000ffff RSI: 0000000000000010 RDI: 0000000000000282
RBP: ffff9a4731287c38 R08: ffffffff92413140 R09: 00003ffffffff000
R10: ffff9a46ffe0d878 R11: 00007fdec11b9000 R12: ffff9a477ffd6000
R13: ffffffff91dc1146 R14: ffff9a4731287c00 R15: 0000000000020200
FS:  00007fdec12b6700(0000) GS:ffff9a4777200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdf76a15530 CR3: 0000000223b52000 CR4: 00000000000206f0
Call Trace:
[<ffffffff91c7a460>] ? leave_mm+0x110/0x110
[<ffffffff91c7a460>] ? leave_mm+0x110/0x110
[<ffffffff91c7a460>] ? leave_mm+0x110/0x110
[<ffffffff91d1147f>] smp_call_function_single+0x5f/0xa0
[<ffffffff91f75c05>] ? cpumask_next_and+0x35/0x50
[<ffffffff91d11a2b>] smp_call_function_many+0x22b/0x270
[<ffffffff91c7a628>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff91c7a698>] flush_tlb_mm_range+0x68/0x140
[<ffffffff91de4607>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff91de5f05>] tlb_finish_mmu+0x55/0x60
[<ffffffff91de8d52>] zap_page_range+0x142/0x190
[<ffffffff91de3c1d>] SyS_madvise+0x49d/0xac0
[<ffffffff92375d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9237606b>] tracesys+0xa3/0xc9
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 bc 94 92 48 89 df e8 fa b6 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 67s! [systemd:1]
Modules linked in: tcp_diag inet_diag nfnetlink_queue nfnetlink_log nfnetlink bluetooth rfkill binfmt_misc ext4 mbcache jbd2 ppdev sg joydev virtio_balloon parport_pc parport i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm floppy drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1 Comm: systemd Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9da334bb0000 ti: ffff9da334bb8000 task.ti: ffff9da334bb8000
RIP: 0010:[<ffffffffa5b1109a>]  [<ffffffffa5b1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9da334bbbc30  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff9da334bbbc00 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000286
RBP: ffff9da334bbbc78 R08: ffffffffa6212d88 R09: ffffffffa5bf8298
R10: ffff9da9dce5f0a0 R11: ffffe9990026a9c0 R12: ffffffffa5be93f8
R13: ffff9da334bbbc30 R14: 00007f34957fb000 R15: ffffe99905cf0af0
FS:  00007f34957ed940(0000) GS:ffff9da9dce40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f34957fb000 CR3: 000000081da72000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa5a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa5a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa5a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa5b111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa5d75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffa5b1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffa5a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa5a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa5be4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffa5be5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffa5bef014>] unmap_region+0xf4/0x140

NMI watchdog: BUG: soft lockup - CPU#7 stuck for 28s! [com.alibaba.nac:16916]
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 32s! [com.alibaba.nac:16840]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel rpcsec_gss_krb5 auth_rpcgss ip6table_nat nfsv4 nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 dns_resolver nfs lockd grace fscache nf_conntrack_netlink br_netfilter overlay(T) xt_addrtype xt_set xt_mark ip_set_hash_ipportnet ip_set_hash_ipport ip_set_hash_ipportip ip_set_bitmap_port ip_set nfnetlink dummy xt_comment xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper
cryptd pcspkr sg joydev vmw_balloon vmw_vmci parport_pc parport i2c_piix4 shpchp sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci mptspi ata_piix scsi_transport_spi libahci drm crct10dif_pclmul crct10dif_common mptscsih libata crc32c_intel serio_raw mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 16840 Comm: com.alibaba.nac Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff909d9330cf10 ti: ffff909dd8ef4000 task.ti: ffff909dd8ef4000
RIP: 0010:[<ffffffffbbf0779a>]  [<ffffffffbbf0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff909dd8ef7c30  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff909dd8ef7c00 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffff909dd8ef7c78 R08: ffffffffbc6138e0 R09: ffffffffbbfd35bc
R10: ffffc0917fd15c70 R11: ffffdb244e2ff140 R12: ffffdb2442c13470
R13: ffff909ab0605a28 R14: ffffffffbbfd2058 R15: ffff909dd8ef7bb0
FS:  00007f9f781a6700(0000) GS:ffff909e3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9fdc02d000 CR3: 00000003f3374000 CR4: 00000000000007e0
Call Trace:
[<ffffffffbbe76e50>] ? leave_mm+0x110/0x110
[<ffffffffbbe76e50>] ? leave_mm+0x110/0x110
[<ffffffffbbe76e50>] ? leave_mm+0x110/0x110
[<ffffffffbbf078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffbc14a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffbbf07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffbbe77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbbe77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbbfc0497>] tlb_flush_mmu.part.75+0x37/0xe0

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [sge_execd:15832]
Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache 8021q garp mrp stp llc ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp joydev kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd i7core_edac dcdbas iTCO_wdt iTCO_vendor_support ipmi_si ipmi_devintf sg ipmi_msghandler lpc_ich ioatdma edac_core pcspkr i2c_i801 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 mlx4_en sd_mod crc_t10dif crct10dif_generic ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm mlx4_core ahci igb libahci libata crct10dif_pclmul crct10dif_common ptp crc32c_intel pps_core dca i2c_algo_bit i2c_core devlink
CPU: 4 PID: 15832 Comm: sge_execd Tainted: G             L ------------   3.10.0-693.5.2.el7.x86_64 #1
Hardware name: Dell     C6100       /0D61XP, BIOS 1.57 04/14/2011
task: ffff880614bc6eb0 ti: ffff8806056d4000 task.ti: ffff8806056d4000
RIP: 0010:[<ffffffff810f933e>]  [<ffffffff810f933e>] generic_exec_single+0xfe/0x1a0
RSP: 0018:ffff8806056d7c60  EFLAGS: 00000202
RAX: ffff880627a1b000 RBX: ffff880626395140 RCX: ffff880627a1b0f8
RDX: ffff880627a17800 RSI: ffff8806056d7c60 RDI: ffff8806056d7c60
RBP: ffff8806056d7cb0 R08: 0000000000000001 R09: ffffffff811bd6d8
R10: ffff880627b19bc0 R11: ffffea001778dbc0 R12: ffff8806056d7c28
R13: ffff880614bc6eb0 R14: ffff880614bc6eb0 R15: ffff880614bc6eb0
FS:  00007f4ff504d840(0000) GS:ffff880627b00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4ff5060000 CR3: 00000006053fe000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880627a1b0f8 0000000000000000 ffffffff8106e4a0 ffff8806056d7d38
0000000000000003 0000000089e938b8 0000000089e938b8 0000000000000000
ffffffff8106e4a0 ffffffff8106e4a0 ffff8806056d7ce0 ffffffff810f943f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f943f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813227c5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99d3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811acfc3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4fd8>] ret_from_fork+0x58/0x90
[<ffffffff810b08d0>] ? insert_kthread_work+0x40/0x40
Code: d5 ff 66 66 90 0f ae e8 0f 31 48 c1 e2 20 89 c0 48 09 c2 48 31 d3 e9 78 ff ff ff 90 90 90 48 89 f8 48 89 d1 48 c1 e9 03 83 e2 07 <f3> 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b 4e 08 4c 8b 56 10 4c 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [systemd-journal:1580]
Modules linked in: sunrpc vfat fat ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr virtio_balloon parport_pc parport i2c_piix4 ip_tables xfs libcrc32c virtio_blk virtio_console virtio_net crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic qxl pata_acpi drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1580 Comm: systemd-journal Tainted: G             L ------------   3.10.0-957.10.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.0.0 02/06/2015
task: ffff96c4b8e1c100 ti: ffff96c4b8e6c000 task.ti: ffff96c4b8e6c000
RIP: 0010:[<ffffffff9cf111be>]  [<ffffffff9cf111be>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff96c4b8e6fc30  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff96c4b8e6fc30 RCX: 0000000000000830
RDX: 0000000000000002 RSI: 0000000000000002 RDI: 0000000000000286
RBP: ffff96c4b8e6fc78 R08: ffffffff9d613040 R09: ffffffff9cff881c
R10: ffff96c4be21f0a0 R11: fffff1b680a23b00 R12: 0000000000000001
R13: 0000000000000001 R14: ffff96c4b8e6fd00 R15: 0000000000000001
FS:  00007f43aa596880(0000) GS:ffff96c4be200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000006614f000 CR3: 000000007c1c2000 CR4: 00000000000406f0
Call Trace:
[<ffffffff9ce7a400>] ? leave_mm+0x110/0x110
[<ffffffff9ce7a400>] ? leave_mm+0x110/0x110
[<ffffffff9ce7a400>] ? leave_mm+0x110/0x110
[<ffffffff9cf112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffff9d175b35>] ? cpumask_next_and+0x35/0x50
[<ffffffff9cf1187b>] smp_call_function_many+0x22b/0x270
[<ffffffff9ce7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9ce7a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9cfe4507>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff9cfe5e05>] tlb_finish_mmu+0x55/0x60

NMI watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [systemd:1]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq joydev pcspkr snd_seq_device snd_pcm snd_timer sg snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel
serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt e1000 fb_sys_fops ttm mptspi drm nfit ata_piix libnvdimm scsi_transport_spi mptscsih libata mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 1 Comm: systemd Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9e80fbb58000 ti: ffff9e80fbbd0000 task.ti: ffff9e80fbbd0000
RIP: 0010:[<ffffffffb5b1109a>]  [<ffffffffb5b1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9e80fbbd3c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9e80fbbd3c00 RCX: 0000000000000002
RDX: ffffffffb6213010 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9e80fbbd3c78 R08: ffffffffb6213008 R09: ffffffffb5bf838c
R10: ffff9e823379f0a0 R11: fffffb460a859400 R12: ffff9e80fbbd3bc8
R13: ffffffffb5bc588e R14: ffff9e80fbbd3ba0 R15: ffff9e7fb5df8f30
FS:  00007fd19920d940(0000) GS:ffff9e8233780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd19921f000 CR3: 00000002b363e000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb5a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb5a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb5a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb5b111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb5d75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffb5b1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffb5a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb5a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb5be4077>] tlb_flush_mmu.part.76+0x37/0xe0

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 37s! [kubelet:85762]
Modules linked in: xt_recent tcp_diag udp_diag inet_diag binfmt_misc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache xt_set ipt_rpfilter xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ip_set ipip tunnel4 ip_tunnel xt_statistic xt_comment xt_mark ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs xt_nat veth nf_conntrack_netlink nfnetlink xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntrack ipt_REJECT nf_reject_ipv4 br_netfilter tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel
lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 vmw_vmci shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih crc32c_intel mptbase serio_raw i2c_core vmxnet3 floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 85762 Comm: kubelet Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff8cff229caf70 ti: ffff8cff05f7c000 task.ti: ffff8cff05f7c000
RIP: 0010:[<ffffffff9810779a>]  [<ffffffff9810779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8cff05f7fbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8cff05f7fbc0 RCX: 0000000000000003
RDX: ffffffff98813670 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8cff05f7fc38 R08: ffffffff98813668 R09: ffff8d01bf7b80d8
R10: ffffd9ca4b17a880 R11: 0000000000000246 R12: ffff8cff229caf70
R13: ffff8cff229caf70 R14: ffff8cff05f7fb60 R15: 0000000000000081
FS:  00007f3333fff700(0000) GS:ffff8d032d740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00025f010 CR3: 000000041db86000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff98076e50>] ? leave_mm+0x110/0x110
[<ffffffff98076e50>] ? leave_mm+0x110/0x110
[<ffffffff98076e50>] ? leave_mm+0x110/0x110
[<ffffffff981078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9834a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff98107e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff98077018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff98077088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff981c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff981c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff981c4a02>] zap_page_range+0x142/0x190
[<ffffffff981bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff9871f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 d3 98 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [filebeat:42866]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr sg snd_seq_midi snd_seq_midi_event parport_pc parport snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata e1000 mptspi scsi_transport_spi mptscsih nfit mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 42866 Comm: filebeat Not tainted 3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff9555b5906180 ti: ffff95551aef0000 task.ti: ffff95551aef0000
RIP: 0010:[<ffffffffb25111ba>]  [<ffffffffb25111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff95551aef3bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff95551aef3bc0 RCX: 0000000000000002
RDX: ffffffffb2c13050 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff95551aef3c38 R08: ffffffffb2c13048 R09: 00003ffffffff000
R10: ffff955539953440 R11: 0000000000000246 R12: ffff9555bffd8000
R13: ffffffffb25c0eb6 R14: ffff95551aef3c00 R15: 0000000000020200
FS:  00007fb44ee25700(0000) GS:ffff9555b9680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f21e218a1a0 CR3: 0000000076de8000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb247a400>] ? leave_mm+0x110/0x110
[<ffffffffb247a400>] ? leave_mm+0x110/0x110
[<ffffffffb247a400>] ? leave_mm+0x110/0x110
[<ffffffffb25112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffffb2775a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffb251187b>] smp_call_function_many+0x22b/0x270
[<ffffffffb247a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb247a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb25e4427>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffb25e5d25>] tlb_finish_mmu+0x55/0x60
[<ffffffffb25e8b72>] zap_page_range+0x142/0x190
[<ffffffffb25e3a3d>] SyS_madvise+0x49d/0xac0
[<ffffffffb2b7505b>] tracesys+0xa3/0xc9
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 b3 48 89 df e8 da b6 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [Chrome_ChildIOT:4051]
INFO: rcu_sched detected stalls on CPUs/tasks: { 1} 
Modules linked in:
tcp_lp
fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmhgfs(OE) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 snd_ens1371 mbcache snd_rawmidi snd_ac97_codec iosf_mbi jbd2 kvm_intel ac97_bus kvm snd_seq irqbypass snd_seq_device snd_pcm snd_timer snd crc32_pclmul ghash_clmulni_intel soundcore aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd nfit ppdev sg vmw_balloon libnvdimm pcspkr joydev parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix drm mptspi serio_raw libata scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 4051 Comm: Chrome_ChildIOT Kdump: loaded Tainted: G           OE  ------------   3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9eb9e6336180 ti: ffff9eb9eb894000 task.ti: ffff9eb9eb894000
RIP: 0010:[<ffffffff973111ba>]  [<ffffffff973111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9eb9eb897bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9eb9eb897bc0 RCX: 0000000000000003
RDX: ffffffff97a132d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9eb9eb897c38 R08: ffffffff97a132c8 R09: 00003ffffffff000
R10: ffff9eb966ff4ca8 R11: 0000000000000206 R12: ffff9eb9fffda000
R13: ffffffff973c0f96 R14: ffff9eb9eb897c00 R15: 0000000000020200
FS:  00007f7ca5c78700(0000) GS:ffff9eb9ed740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000024adb3637ff0 CR3: 00000003efaba000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9727a400>] ? leave_mm+0x110/0x110
[<ffffffff9727a400>] ? leave_mm+0x110/0x110
[<ffffffff9727a400>] ? leave_mm+0x110/0x110
[<ffffffff973112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffff97575b35>] ? cpumask_next_and+0x35/0x50
[<ffffffff9731187b>] smp_call_function_many+0x22b/0x270
[<ffffffff9727a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9727a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff973e4507>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff973e5e05>] tlb_finish_mmu+0x55/0x60
[<ffffffff973e8c52>] zap_page_range+0x142/0x190
[<ffffffff973e3b1d>] SyS_madvise+0x49d/0xac0
[<ffffffff97975ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 b9 f4 97 48 89 df e8 ca b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#12 stuck for 21s! [java:13628]
Modules linked in: binfmt_misc intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg ppdev vmw_balloon pcspkr parport_pc parport i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea mptspi sysfillrect sysimgblt scsi_transport_spi fb_sys_fops mptscsih ttm drm crct10dif_pclmul ata_piix crct10dif_common crc32c_intel serio_raw libata mptbase vmxnet3 i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 12 PID: 13628 Comm: java Tainted: G             L ------------   3.10.0-514.10.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff881119e74e70 ti: ffff8808d9d14000 task.ti: ffff8808d9d14000
RIP: 0010:[<ffffffff810f945a>]  [<ffffffff810f945a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8808d9d17c40  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffffffff810cbe9c RCX: 0000000000000030
RDX: 0000000000000010 RSI: 0000000000000010 RDI: 0000000000000286
RBP: ffff8808d9d17c88 R08: ffffffff816b1a20 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff881119e74ed8
R13: ffff8808d9d17c68 R14: ffff8811bffda000 R15: ffffffff8118aec6
FS:  00007f367e0f0700(0000) GS:ffff8811b5c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9160f9ded8 CR3: 00000005237ab000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106c430 ffff8808d9d17d10
0000000000000003 000000009b3f17b9 000000000000000d ffffffff8106c430
ffffffff8106c430 ffff8808d9d17cb8 ffffffff810f955f ffff8808d9d17cb8
Call Trace:
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff810f955f>] smp_call_function_single+0x5f/0xa0
[<ffffffff81317bc5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f9b03>] smp_call_function_many+0x223/0x260
[<ffffffff8106c5f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c6c9>] flush_tlb_mm_range+0x69/0x140
[<ffffffff811ac293>] tlb_flush_mmu.part.61+0x33/0xc0
[<ffffffff811ad585>] tlb_finish_mmu+0x55/0x60
[<ffffffff811af95a>] zap_page_range+0x13a/0x180
[<ffffffff811ab9ec>] SyS_madvise+0x38c/0x8d0
[<ffffffff81696b09>] system_call_fastpath+0x16/0x1b
Code: 80 77 01 00 48 89 de 48 03 14 c5 40 57 ad 81 48 89 df e8 0a 29 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [pool:2590]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event intel_powerclamp coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul btusb glue_helper btrtl btbcm ablk_helper btintel cryptd snd_ens1371 snd_rawmidi bluetooth snd_ac97_codec ac97_bus sg snd_seq ppdev rfkill snd_seq_device pcspkr snd_pcm vmw_balloon i2c_piix4 snd_timer snd soundcore vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables
xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common vmwgfx crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt ata_piix fb_sys_fops ttm drm serio_raw mptspi libata scsi_transport_spi e1000 mptscsih mptbase i2c_core fjes
CPU: 3 PID: 2590 Comm: pool Not tainted 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8800a38a0fb0 ti: ffff8800a7364000 task.ti: ffff8800a7364000
RIP: 0010:[<ffffffff810f925a>]  [<ffffffff810f925a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8800a7367c40  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8800a7367c10 RCX: 0000000000000001
RDX: ffffffff816af9b0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8800a7367c88 R08: ffffffff816af9a8 R09: 00007f1a99e9e000
R10: 0000000003a5b7d0 R11: 00007f1a996a4000 R12: 0000000000000000
R13: ffff8801396d6c40 R14: 0000000300000000 R15: ffff880139616c40
FS:  00007f1a99ea3700(0000) GS:ffff8801396c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1a99690fc8 CR3: 0000000133252000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106c300 ffff8800a7367d10
0000000000000003 0000000022e1b0b1 0000000000000000 ffffffff8106c300
ffffffff8106c300 ffff8800a7367cb8 ffffffff810f935f ffff8800a7367cb8
Call Trace:
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff810f935f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813179a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f9903>] smp_call_function_many+0x223/0x260
[<ffffffff8106c4c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c599>] flush_tlb_mm_range+0x69/0x140
[<ffffffff811ac093>] tlb_flush_mmu.part.61+0x33/0xc0
[<ffffffff811ad385>] tlb_finish_mmu+0x55/0x60
[<ffffffff811af75a>] zap_page_range+0x13a/0x180
[<ffffffff811ab7ec>] SyS_madvise+0x38c/0x8d0
[<ffffffff810f7f80>] ? SyS_futex+0x80/0x180
[<ffffffff816965c9>] system_call_fastpath+0x16/0x1b
Code: 80 77 01 00 48 89 de 48 03 14 c5 40 57 ad 81 48 89 df e8 ea 28 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [java:27030]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 ipt_MASQUERADE tun nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 ebtable_filter nf_defrag_ipv4 ebtables nf_nat_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntrack ip6table_filter ip6_tables br_netfilter iptable_filter bridge stp llc devlink vmw_vsock_vmci_transport vsock overlay(T) sunrpc coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev pcspkr vmw_vmci sg i2c_piix4 floppy parport_pc parport shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix scsi_transport_spi drm crct10dif_pclmul
crct10dif_common libata mptscsih crc32c_intel e1000 serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 27030 Comm: java Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8d7e8be43f40 ti: ffff8d7f6dfb0000 task.ti: ffff8d7f6dfb0000
RIP: 0010:[<ffffffffad30779a>]  [<ffffffffad30779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8d7f6dfb3c30  EFLAGS: 00000202
RAX: 0000000000000020 RBX: 0000000000001000 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff8d7f6dfb3c78 R08: ffffffffada13660 R09: ffffffffad3d35bc
R10: ffff8d861ee5ba40 R11: ffffe0e3559d44c0 R12: 0000000000000001
R13: ffff8d7f6dfb3bc8 R14: ffffffffad91a925 R15: ffff8d7f6dfb3bb8
FS:  00007fc55a317700(0000) GS:ffff8d861ee40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc629c80000 CR3: 00000004aa324000 CR4: 00000000000007e0
Call Trace:
[<ffffffffad276e50>] ? leave_mm+0x110/0x110
[<ffffffffad276e50>] ? leave_mm+0x110/0x110
[<ffffffffad276e50>] ? leave_mm+0x110/0x110
[<ffffffffad3078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffad54a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffad307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffad277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffad277088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffad3c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffad3c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffad3ca624>] unmap_region+0xf4/0x140

BUG: soft lockup - CPU#0 stuck for 81s! [s1-perf:31145]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon sg parport_pc i2c_piix4 vmw_vmci parport shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx drm_kms_helper vmxnet3 mptspi ttm scsi_transport_spi mptscsih mptbase drm ata_piix libata i2c_core dm_mirror dm_region_hash
dm_log dm_mod
CPU: 0 PID: 31145 Comm: s1-perf Tainted: G        W    L ------------   3.10.0-327.28.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff88002014ae00 ti: ffff8800b61b4000 task.ti: ffff8800b61b4000
RIP: 0010:[<ffffffff810e69de>]  [<ffffffff810e69de>] generic_exec_single+0xfe/0x1a0
RSP: 0018:ffff8800b61b7c40  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff8800b61b7c10 RCX: 0000000000000830
RDX: 0000000000000002 RSI: 0000000000000002 RDI: 0000000000000282
RBP: ffff8800b61b7c88 R08: ffffffff8165fbe0 R09: 00007f487aaa5000
R10: 00000000007fb000 R11: 0000000000000202 R12: 0000000000000000
R13: ffff88012dc167c0 R14: 0000000000000000 R15: 0000000000000000
FS:  00007f487aaaa700(0000) GS:ffff88012dc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f487d2db000 CR3: 0000000069ea6000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff81065c90 ffff8800b61b7d10
0000000000000003 00000000f639737f 0000000000000001 ffffffff81065c90
ffffffff81065c90 ffff8800b61b7cb8 ffffffff810e6adf ffff8800b61b7cb8
Call Trace:
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff81065c90>] ? leave_mm+0x70/0x70
[<ffffffff810e6adf>] smp_call_function_single+0x5f/0xa0
[<ffffffff812f3015>] ? cpumask_next_and+0x35/0x50
[<ffffffff810e7083>] smp_call_function_many+0x223/0x260
[<ffffffff81065e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81065f26>] flush_tlb_mm_range+0x66/0x140
[<ffffffff811929d3>] tlb_flush_mmu.part.54+0x33/0xc0
[<ffffffff81193565>] tlb_finish_mmu+0x55/0x60
[<ffffffff81195afa>] zap_page_range+0x12a/0x170
[<ffffffff81192224>] SyS_madvise+0x394/0x820
[<ffffffff8110b9f4>] ? __audit_syscall_entry+0xb4/0x110
[<ffffffff81646d52>] tracesys+0xdd/0xe2
Code: 48 89 de 48 03 14 c5 20 c9 a5 81 48 89 df e8 7a 03 22 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 80 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [ActiveMQ Broker:44025]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev coretemp iosf_mbi vmw_balloon crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm sg snd_timer snd soundcore vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul
crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm e1000 ata_piix libata mptspi scsi_transport_spi mptscsih drm_panel_orientation_quirks mptbase nfit libnvdimm
CPU: 3 PID: 44025 Comm: ActiveMQ Broker Kdump: loaded Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff95c263e5d140 ti: ffff95c236f7c000 task.ti: ffff95c236f7c000
RIP: 0010:[<ffffffff9d31109a>]  [<ffffffff9d31109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff95c236f7fbf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff95c236f7fbc0 RCX: 0000000000000003
RDX: ffffffff9da13290 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff95c236f7fc38 R08: ffffffff9da13288 R09: 00003ffffffff000
R10: ffff95c2a0ac9e60 R11: 00007fa54d2db000 R12: ffff95c2bffda000
R13: ffffffff9d3c0c06 R14: ffff95c236f7fc00 R15: 0000000000020200
FS:  00007fa54d3d7700(0000) GS:ffff95c2b56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa5fe3ff2b0 CR3: 00000002292e0000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9d27a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9d27a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9d27a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9d3111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9d575465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9d31175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9d27a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9d27a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9d3e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff9d3e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff9d3e87c2>] zap_page_range+0x142/0x190
[<ffffffff9d3e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff9d974ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 f4 9d 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [mysqld:23520]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr nfit joydev vmw_balloon sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih libata e1000 crct10dif_pclmul crct10dif_common mptbase crc32c_intel i2c_core serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 23520 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8b12ddefbf40 ti: ffff8b12eea8c000 task.ti: ffff8b12eea8c000
RIP: 0010:[<ffffffffa270779a>]  [<ffffffffa270779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8b12eea8fbf0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8b12eea8fbc0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8b12eea8fc38 R08: ffffffffa2e13920 R09: ffff8b12e1259288
R10: ffffdf04c670a3c0 R11: 0000000000000206 R12: ffff8b12ddefbf40
R13: ffff8b12ddefbf40 R14: 0000000000000002 R15: 0000000000000030
FS:  00007ff026e2bb00(0000) GS:ffff8b12ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005631f8c5e000 CR3: 0000000017cb4000 CR4: 00000000000607f0
Call Trace:
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa27078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa294a215>] ? cpumask_next_and+0x35/0x50
[<ffffffffa2707e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffa2677028>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2677098>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa27c0437>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa27c1c15>] tlb_finish_mmu+0x55/0x60
[<ffffffffa27c49a2>] zap_page_range+0x142/0x190
[<ffffffffa27bfa9d>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa2d1f6e1>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa2d1f6d5>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffa2d1f795>] system_call_fastpath+0x1c/0x21
[<ffffffffa2d1f6e1>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 33 a3 48 89 df e8 ea 86 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [SystemLogFlush:12278]
Modules linked in: binfmt_misc xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE dm_thin_pool nf_nat_masquerade_ipv4 iptable_nat dm_persistent_data dm_bio_prison nf_nat_ipv4 dm_bufio nf_nat loop nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event coretemp iosf_mbi snd_ens1371 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul snd_rawmidi glue_helper ablk_helper snd_ac97_codec cryptd ac97_bus ppdev snd_seq snd_seq_device snd_pcm vmw_balloon snd_timer snd soundcore pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables
xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 12278 Comm: SystemLogFlush Kdump: loaded Not tainted 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8fd834fa5140 ti: ffff8fd9a57e0000 task.ti: ffff8fd9a57e0000
RIP: 0010:[<ffffffffaa1111ba>]  [<ffffffffaa1111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8fd9a57e3bf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8fd9a57e3bc0 RCX: 0000000000000001
RDX: ffffffffaa812dd0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8fd9a57e3c38 R08: ffffffffaa812dc8 R09: 00003ffffffff000
R10: ffff8fd82da08948 R11: 0000000000000206 R12: ffff8fd9a57e3be8
R13: ffff8fd9b25b57a0 R14: 0000000000000077 R15: 00000000073199a2
FS:  00007fd4a9b62700(0000) GS:ffff8fd9b5680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd4aa923000 CR3: 00000000ad85c000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaa07a400>] ? leave_mm+0x110/0x110
[<ffffffffaa07a400>] ? leave_mm+0x110/0x110
[<ffffffffaa07a400>] ? leave_mm+0x110/0x110
[<ffffffffaa1112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffffaa375b35>] ? cpumask_next_and+0x35/0x50
[<ffffffffaa11187b>] smp_call_function_many+0x22b/0x270
[<ffffffffaa07a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaa07a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaa1e4507>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffaa1e5e05>] tlb_finish_mmu+0x55/0x60
[<ffffffffaa1e8c52>] zap_page_range+0x142/0x190
[<ffffffffaa240c23>] ? do_sync_write+0x93/0xe0
[<ffffffffaa1e3b1d>] SyS_madvise+0x49d/0xac0
[<ffffffffaa2417d2>] ? vfs_write+0x182/0x1f0
[<ffffffffaa775ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 b9 d4 aa 48 89 df e8 ca b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00