NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [python:4409]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [python:4409]
Modules linked in: tcp_diag udp_diag inet_diag 8021q garp mrp stp llc nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter bonding nls_utf8 isofs xfs libcrc32c intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper iTCO_wdt cryptd iTCO_vendor_support sg sb_edac mei_me i2c_i801 mei edac_core i2c_core pcspkr lpc_ich shpchp acpi_power_meter ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ixgbe ahci libahci mpt3sas mdio libata ptp raid_class pps_core scsi_transport_sas scsi_ep_front(OE) dca fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 4409 Comm: python Tainted: G        W  OEL ------------   3.10.0-514.el7.x86_64 #1
Hardware name: Huawei RH2288H V3/BC11HGSA0, BIOS 3.87 02/02/2018
task: ffff881ffadece70 ti: ffff881ff9f20000 task.ti: ffff881ff9f20000
RIP: 0010:[<ffffffff810f98e6>]  [<ffffffff810f98e6>] smp_call_function_many+0x206/0x260
RSP: 0018:ffff881ff9f23bf0  EFLAGS: 00000202
RAX: 0000000000000018 RBX: 0000000000000282 RCX: ffff883fff09ae48
RDX: 0000000000000018 RSI: 0000000000000038 RDI: 0000000000000000
RBP: ffff881ff9f23c28 R08: ffff8801739abc00 R09: ffffffff81317a89
R10: ffff881fffa99b40 R11: ffffea0004928c00 R12: 0000000000000282
R13: ffff881ff9f23ba0 R14: 0000000000001592 R15: 0000000001320122
FS:  00007fad8da45700(0000) GS:ffff881fffa80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5b5e0b3980 CR3: 0000001fec274000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000100000040 ffffffff81eeef80 ffffffff81eeef80 ffffffff81189700
0000000000000000 0000000000000002 ffff88207ffda000 ffff881ff9f23c58
ffffffff810f9a2a 0000000000000037 0000000000000038 0000000000000038
Call Trace:
[<ffffffff81189700>] ? drain_pages+0xb0/0xb0
[<ffffffff810f9a2a>] on_each_cpu_mask+0x2a/0x60
[<ffffffff811865e5>] drain_all_pages+0xb5/0xc0
[<ffffffff8168199a>] __alloc_pages_slowpath+0x511/0x725
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff810830ad>] copy_process+0x1dd/0x1960
[<ffffffff810849e1>] do_fork+0x91/0x2c0

NMI watchdog: BUG: soft lockup - CPU#36 stuck for 22s! [python:4335]
NMI watchdog: BUG: soft lockup - CPU#36 stuck for 22s! [python:4335]
Modules linked in: tcp_diag udp_diag inet_diag 8021q garp mrp stp llc nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter bonding nls_utf8 isofs xfs libcrc32c intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt iTCO_vendor_support i2c_i801 lpc_ich mei_me mei sb_edac i2c_core pcspkr edac_core sg shpchp acpi_power_meter ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ixgbe ahci mpt3sas libahci mdio ptp libata raid_class pps_core scsi_ep_front(OE) scsi_transport_sas dca fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 36 PID: 4335 Comm: python Tainted: G        W  OEL ------------   3.10.0-514.el7.x86_64 #1
Hardware name: Huawei RH2288H V3/BC11HGSA0, BIOS 5.03 07/25/2018
task: ffff883ffa225e20 ti: ffff881ffc61c000 task.ti: ffff881ffc61c000
RIP: 0010:[<ffffffff810f98e6>]  [<ffffffff810f98e6>] smp_call_function_many+0x206/0x260
RSP: 0018:ffff881ffc61fbf0  EFLAGS: 00000202
RAX: 000000000000001a RBX: ffffffff81189700 RCX: ffff883fff11b398
RDX: 000000000000001a RSI: 0000000000000038 RDI: 0000000000000000
RBP: ffff881ffc61fc28 R08: ffff880173148c00 R09: ffffffff81317a89
R10: ffff881ffff99b40 R11: ffffea000103be00 R12: 0000000000000282
R13: ffff881ffc61fba0 R14: 000000000000198b R15: 0000000001320122
FS:  00007f037916b700(0000) GS:ffff881ffff80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0374000020 CR3: 0000003ff3f13000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000100000040 ffffffff81eeef80 ffffffff81eeef80 ffffffff81189700
0000000000000000 0000000000000024 ffff88207ffda000 ffff881ffc61fc58
ffffffff810f9a2a 0000000000000037 0000000000000038 0000000000000038
Call Trace:
[<ffffffff81189700>] ? drain_pages+0xb0/0xb0
[<ffffffff810f9a2a>] on_each_cpu_mask+0x2a/0x60
[<ffffffff811865e5>] drain_all_pages+0xb5/0xc0
[<ffffffff8168199a>] __alloc_pages_slowpath+0x511/0x725
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff810830ad>] copy_process+0x1dd/0x1960
[<ffffffff810849e1>] do_fork+0x91/0x2c0
[<ffffffff81084c96>] SyS_clone+0x16/0x20

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [dbus-daemon:1194]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables devlink tun ipmi_devintf ipmi_msghandler ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 xt_conntrack ip6table_filter ip6_tables iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus aesni_intel btusb snd_seq btrtl btbcm btintel lrw gf128mul glue_helper ablk_helper snd_seq_device bluetooth snd_pcm ppdev rfkill cryptd snd_timer snd soundcore joydev pcspkr vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci sch_fq_codel ip_tables
xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul crct10dif_common ttm crc32c_intel ata_piix nfit drm libnvdimm libata serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 1194 Comm: dbus-daemon Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9a173a154200 ti: ffff9a173a158000 task.ti: ffff9a173a158000
RIP: 0010:[<ffffffff9ed1720a>]  [<ffffffff9ed1720a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9a173a15bc60  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 000c00009edc7940 RCX: ffff9a18376a0600
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9a173a15bc98 R08: ffff9a17fb88b000 R09: ffffffff9ef86a19
R10: ffff9a183765f160 R11: ffffe635040d0600 R12: 0000000000000282
R13: 0000000000000282 R14: ffff9a173a15bc10 R15: 0000000001320122
FS:  00007f70dc389880(0000) GS:ffff9a1837640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000411d7a CR3: 00000001b5d92000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9edc7940>] ? drain_pages+0xb0/0xb0
[<ffffffff9ed17359>] on_each_cpu_mask+0x29/0x70
[<ffffffff9edc4825>] drain_all_pages+0xb5/0xc0
[<ffffffff9edc96ab>] __alloc_pages_nodemask+0x8cb/0xbe0
[<ffffffff9ec989bd>] copy_process+0x1dd/0x1a80
[<ffffffff9ec9a411>] do_fork+0x91/0x330
[<ffffffff9ec9a736>] SyS_clone+0x16/0x20
[<ffffffff9f39a374>] stub_clone+0x44/0x70
[<ffffffff9f399f92>] ? system_call_fastpath+0x25/0x2a

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [java:104205]
Modules linked in: vmw_vsock_vmci_transport vsock vfat fat sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg tpm_crb parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw e1000e ahci mptsas drm scsi_transport_sas ata_piix libahci mptscsih libata mptbase ptp pps_core nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 104205 Comm: java Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.17369862.B64.2012240522 12/24/2020
task: ffff98084b145140 ti: ffff98084fcdc000 task.ti: ffff98084fcdc000
RIP: 0010:[<ffffffffabd1173a>]  [<ffffffffabd1173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff98084fcdfbf0  EFLAGS: 00000202
RAX: 000000000000000e RBX: 000c0000abdbf550 RCX: ffff98105d9a0600
RDX: 000000000000000e RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff98084fcdfc28 R08: ffff9809b450d400 R09: ffffffffabf755c9
R10: ffff98105d6df160 R11: fffffc1a0e0ff800 R12: 0000000000000286
R13: 0000000000000286 R14: ffff98084fcdfba0 R15: 0000000001320122
FS:  00007f856563d700(0000) GS:ffff98105d6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b65ff9f40 CR3: 0000000324884000 CR4: 00000000001607e0
Call Trace:
[<ffffffffabdbf550>] ? drain_pages+0xb0/0xb0
[<ffffffffabd11889>] on_each_cpu_mask+0x29/0x70
[<ffffffffabdbc435>] drain_all_pages+0xb5/0xc0
[<ffffffffac35d228>] __alloc_pages_slowpath+0x510/0x724
[<ffffffffabdc0e95>] __alloc_pages_nodemask+0x405/0x420
[<ffffffffabc94cdd>] copy_process+0x1dd/0x1a40
[<ffffffffabc966f1>] do_fork+0x91/0x320
[<ffffffffabc96a06>] SyS_clone+0x16/0x20
[<ffffffffac3751a4>] stub_clone+0x44/0x70
[<ffffffffac374ddb>] ? system_call_fastpath+0x22/0x27
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 ac f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [pk-command-not-:2822]
Modules linked in: binfmt_misc xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg vmw_balloon pcspkr joydev vmw_vmci parport_pc parport i2c_piix4
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops nfit ttm sd_mod libata libnvdimm crc_t10dif crct10dif_generic drm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 2822 Comm: pk-command-not- Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8c9dfb7c2100 ti: ffff8c8b5deec000 task.ti: ffff8c8b5deec000
RIP: 0010:[<ffffffff9e7170da>]  [<ffffffff9e7170da>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8c8b5deefc60  EFLAGS: 00000202
RAX: 0000000000000008 RBX: 000800009e7c78e0 RCX: ffff8ca31f41fc10
RDX: 0000000000000008 RSI: 000000000000000a RDI: 0000000000000000
RBP: ffff8c8b5deefc98 R08: ffff8c8cbfd56800 R09: ffffffff9e986bd9
R10: ffff8ca31f01f160 R11: fffff3fbdf1ef600 R12: 0000000000000282
R13: 0000000000000282 R14: ffff8c8b5deefc10 R15: ffffffff9e7334b5
FS:  00007f35e3243880(0000) GS:ffff8ca31f000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f35d9f8ee78 CR3: 00000000ab7b6000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9e7c78e0>] ? drain_pages+0xb0/0xb0
[<ffffffff9e717229>] on_each_cpu_mask+0x29/0x70
[<ffffffff9e7c47c5>] drain_all_pages+0xb5/0xc0
[<ffffffff9e7c964b>] __alloc_pages_nodemask+0x8cb/0xbe0
[<ffffffff9e69882d>] copy_process+0x1dd/0x1a80
[<ffffffff9e69a281>] do_fork+0x91/0x330
[<ffffffff9e69a5a6>] SyS_clone+0x16/0x20
[<ffffffff9ed96374>] stub_clone+0x44/0x70
[<ffffffff9ed95f92>] ? system_call_fastpath+0x25/0x2a
Code: 48 63 35 ce 85 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 15 35 9f f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 85 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#209 stuck for 22s! [gnome-shell:54286]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14108(E) tcp_lp nfsv4 dns_resolver nfs fscache falcon_lsm_pinned_14006(E) rpcsec_gss_krb5 uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_pinned_14005(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device crc32_pclmul ppdev ghash_clmulni_intel
parport_pc aesni_intel iTCO_wdt lrw iTCO_vendor_support gf128mul glue_helper ablk_helper snd_pcm snd_timer snd cryptd qxl joydev soundcore lpc_ich parport virtio_balloon pcspkr i2c_i801 ttm virtio_rng drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net net_failover virtio_console virtio_blk failover ahci libahci libata crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel virtio_ring serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 209 PID: 54286 Comm: gnome-shell Tainted: P        W   EL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff969b8b78a100 ti: ffff969d577b0000 task.ti: ffff969d577b0000
RIP: 0010:[<ffffffffafd16e4a>]  [<ffffffffafd16e4a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff969d577b3bf8  EFLAGS: 00000202
RAX: 000000000000001d RBX: 00000000000000fc RCX: ffff987c6bd624b8
RDX: 000000000000001d RSI: 00000000000000ff RDI: 0000000000000000
RBP: ffff969d577b3c30 R08: ffff96917bb54c00 R09: ffffffffaff86479
R10: ffff987c6ea5f160 R11: ffffdccd99b29c00 R12: 0000000000000292
R13: 0000000000000018 R14: ffff969d577b3b68 R15: 0000000000000292
FS:  00007f7438b4fa80(0000) GS:ffff987c6ea40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000011d8000 CR3: 00000004b8bfe000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffffafdc7610>] ? drain_pages+0xb0/0xb0
[<ffffffffafd16f99>] on_each_cpu_mask+0x29/0x70
[<ffffffffafdc44f5>] drain_all_pages+0xb5/0xc0
[<ffffffffb037d32c>] __alloc_pages_slowpath+0x510/0x729
[<ffffffffafdc8ee6>] __alloc_pages_nodemask+0x436/0x450
[<ffffffffafc9876d>] copy_process+0x1dd/0x1a80
[<ffffffffafc9a1c1>] do_fork+0x91/0x330
[<ffffffffafc9a4e6>] SyS_clone+0x16/0x20
[<ffffffffb0395374>] stub_clone+0x44/0x70
[<ffffffffb0394f92>] ? system_call_fastpath+0x25/0x2a
Code: 48 63 35 de 86 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 95 b0 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ad 86 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#19 stuck for 23s! [python:4458]
NMI watchdog: BUG: soft lockup - CPU#19 stuck for 23s! [python:4458]
Modules linked in: af_packet_diag netlink_diag sctp_diag sctp dccp_diag dccp unix_diag tcp_diag udp_diag inet_diag 8021q garp mrp stp llc nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter xfs libcrc32c bonding nls_utf8 isofs intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sb_edac iTCO_wdt i2c_i801 edac_core mei_me iTCO_vendor_support sg mei i2c_core pcspkr lpc_ich shpchp acpi_power_meter ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ixgbe ahci libahci mpt3sas mdio libata ptp raid_class pps_core scsi_transport_sas dca scsi_ep_front(OE) fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 19 PID: 4458 Comm: python Tainted: G        W  OEL ------------   3.10.0-514.el7.x86_64 #1
Hardware name: Huawei RH2288H V3/BC11HGSA0, BIOS 5.03 07/25/2018
task: ffff88203be8ce70 ti: ffff880fe76c0000 task.ti: ffff880fe76c0000
RIP: 0010:[<ffffffff810f98e0>]  [<ffffffff810f98e0>] smp_call_function_many+0x200/0x260
RSP: 0018:ffff880fe76c3bf0  EFLAGS: 00000202
RAX: 000000000000001e RBX: 0000000000000282 RCX: ffff88203f29b0f0
RDX: 000000000000001e RSI: 0000000000000028 RDI: 0000000000000000
RBP: ffff880fe76c3c28 R08: ffff8810f9d55000 R09: ffffffff81317a89
R10: ffff88203f259b40 R11: ffffea00044ddc00 R12: 0000000000000282
R13: ffff880fe76c3ba0 R14: 000000000000061c R15: 0000000001320122
FS:  00007f2d7bfff700(0000) GS:ffff88203f240000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000112e310 CR3: 0000001031fd5000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000100000040 ffffffff81eeef80 ffffffff81eeef80 ffffffff81189700
0000000000000000 0000000000000013 ffff88207ffd7000 ffff880fe76c3c58
ffffffff810f9a2a 0000000000000027 0000000000000028 0000000000000028
Call Trace:
[<ffffffff81189700>] ? drain_pages+0xb0/0xb0
[<ffffffff810f9a2a>] on_each_cpu_mask+0x2a/0x60
[<ffffffff811865e5>] drain_all_pages+0xb5/0xc0
[<ffffffff8168199a>] __alloc_pages_slowpath+0x511/0x725
[<ffffffff8118af55>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff810830ad>] copy_process+0x1dd/0x1960
[<ffffffff810849e1>] do_fork+0x91/0x2c0
[<ffffffff81084c96>] SyS_clone+0x16/0x20
[<ffffffff81696919>] stub_clone+0x69/0x90
[<ffffffff816965c9>] ? system_call_fastpath+0x16/0x1b
Code: 23 00 48 63 35 76 9e 9e 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 40 57 ad 81 f6 41 20 01 74 cd 0f 1f 44 00 00 <f3> 90 f6 41 20 01 75 f8 48 63 35 45 9e 9e 00 eb b7 0f b6 4d cc 

NMI watchdog: BUG: soft lockup - CPU#134 stuck for 22s! [gsd-xsettings:170619]
Modules linked in: veth ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter overlay(T) tcp_lp rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13501(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device
snd_pcm snd_timer qxl ttm snd drm_kms_helper ppdev syscopyarea sysfillrect iTCO_wdt iTCO_vendor_support crc32_pclmul parport_pc ghash_clmulni_intel aesni_intel sysimgblt lrw gf128mul pcspkr fb_sys_fops glue_helper ablk_helper soundcore lpc_ich cryptd joydev parport i2c_i801 virtio_balloon virtio_rng drm drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net virtio_console virtio_blk net_failover failover ahci libahci crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel libata virtio_ring serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 134 PID: 170619 Comm: gsd-xsettings Kdump: loaded Tainted: P            EL ------------ T 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff8f8ff7329080 ti: ffff8f94ae370000 task.ti: ffff8f94ae370000
RIP: 0010:[<ffffffffb8b16e4a>]  [<ffffffffb8b16e4a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8f94ae373bf8  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00000000000000fc RCX: ffff916beb6e1900
RDX: 0000000000000003 RSI: 00000000000000ff RDI: 0000000000000000
RBP: ffff8f94ae373c30 R08: ffff8f80f1742000 R09: ffffffffb8d86479
R10: ffff916bed79f160 R11: ffffdbb45f93f600 R12: 0000000000000292
R13: 0000000000000018 R14: ffff8f94ae373b68 R15: 0000000000000292
FS:  00007f01a86d0a40(0000) GS:ffff916bed780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000169e000 CR3: 000000154dd78000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffffb8bc7610>] ? drain_pages+0xb0/0xb0
[<ffffffffb8b16f99>] on_each_cpu_mask+0x29/0x70
[<ffffffffb8bc44f5>] drain_all_pages+0xb5/0xc0
[<ffffffffb917d32c>] __alloc_pages_slowpath+0x510/0x729
[<ffffffffb8bc8ee6>] __alloc_pages_nodemask+0x436/0x450
[<ffffffffb8a9876d>] copy_process+0x1dd/0x1a80
[<ffffffffb8a9a1c1>] do_fork+0x91/0x330
[<ffffffffb8a9a4e6>] SyS_clone+0x16/0x20
[<ffffffffb9195374>] stub_clone+0x44/0x70
[<ffffffffb9194f92>] ? system_call_fastpath+0x25/0x2a
Code: 48 63 35 de 86 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 13 75 b9 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 ad 86 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [ksmtuned:904]
Modules linked in: vmsec_nfq(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nfnetlink_queue vmsecmod(OE) tun ip_set nfnetlink bridge stp llc nf_conntrack sg i2c_piix4 ppdev pcspkr virtio_balloon joydev parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_scsi virtio_blk cirrus drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata virtio_pci floppy virtio_ring i2c_core serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmsec_nfq]
CPU: 8 PID: 904 Comm: ksmtuned Tainted: G           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff880035cd0fd0 ti: ffff8800351e0000 task.ti: ffff8800351e0000
RIP: 0010:[<ffffffff810f99a2>]  [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff8800351e3bf0  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800008118b530 RCX: ffff88081da1ae80
RDX: 0000000000000000 RSI: 0000000000000070 RDI: 0000000000000000
RBP: ffff8800351e3c28 R08: ffff880174b4dc00 R09: ffffffff813227d9
R10: ffff88081dc19c80 R11: ffffea0020796c00 R12: 0000000000000286
R13: 0000000000000286 R14: ffff8800351e3ba0 R15: 0000000001320122
FS:  00007f5224415740(0000) GS:ffff88081dc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001910bc0 CR3: 0000000035d07000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000100000040 ffffffff81f404c0 ffffffff81f404c0 ffffffff8118b530
0000000000000000 0000000000000008 ffff88083ffc3000 ffff8800351e3c58
ffffffff810f9ae9 000000000000000f 0000000000000070 0000000000000070
Call Trace:
[<ffffffff8118b530>] ? drain_pages+0xb0/0xb0
[<ffffffff810f9ae9>] on_each_cpu_mask+0x29/0x60
[<ffffffff81188475>] drain_all_pages+0xb5/0xc0
[<ffffffff8169fbc4>] __alloc_pages_slowpath+0x510/0x724
[<ffffffff8118cd85>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff8108510d>] copy_process+0x1dd/0x1970
[<ffffffff81086a51>] do_fork+0x91/0x320
[<ffffffff81086d66>] SyS_clone+0x16/0x20
[<ffffffff816b5319>] stub_clone+0x69/0x90
[<ffffffff816b4fc9>] ? system_call_fastpath+0x16/0x1b
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [oracle:61941]
Modules linked in: vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd nfit ppdev vmw_balloon joydev pcspkr libnvdimm sg parport_pc parport i2c_piix4 shpchp vmw_vmci binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix mptspi scsi_transport_spi libata mptscsih serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 61941 Comm: oracle Tainted: G             L ------------   3.10.0-693.2.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
task: ffff8800b305cf10 ti: ffff8800b31f8000 task.ti: ffff8800b31f8000
RIP: 0010:[<ffffffff810f99a2>]  [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff8800b31fbbf0  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0000000000000282 RCX: ffff88081e61afe8
RDX: 0000000000000000 RSI: 0000000000000040 RDI: 0000000000000000
RBP: ffff8800b31fbc28 R08: ffff880174bb6800 R09: ffffffff813229e9
R10: ffff88081e799c80 R11: ffffea00206dfa00 R12: 0000000000000282
R13: ffff8800b31fbba0 R14: 000000000000026b R15: 0000000001320122
FS:  00007f9579d17740(0000) GS:ffff88081e780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f9577474f20 CR3: 000000074bb7a000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000100000040 ffffffff81f404c0 ffffffff81f404c0 ffffffff8118b530
0000000000000000 0000000000000006 ffff88083ffc3000 ffff8800b31fbc58
ffffffff810f9ae9 000000000000000b 0000000000000040 0000000000000040
Call Trace:
[<ffffffff8118b530>] ? drain_pages+0xb0/0xb0
[<ffffffff810f9ae9>] on_each_cpu_mask+0x29/0x60
[<ffffffff81188475>] drain_all_pages+0xb5/0xc0
[<ffffffff8169fbe4>] __alloc_pages_slowpath+0x510/0x724
[<ffffffff8118cd85>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff8108510d>] copy_process+0x1dd/0x1970
[<ffffffff81086a51>] do_fork+0x91/0x320
[<ffffffff81086d66>] SyS_clone+0x16/0x20
[<ffffffff816b5359>] stub_clone+0x69/0x90
[<ffffffff816b5009>] ? system_call_fastpath+0x16/0x1b
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#11 stuck for 23s! [abrtd:1987]
Modules linked in: udp_diag tcp_diag inet_diag binfmt_misc tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun bridge stp llc devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod
cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi libata mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 1987 Comm: abrtd Kdump: loaded Tainted: G        W    L ------------   3.10.0-1062.1.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff987d14c641c0 ti: ffff987d11b30000 task.ti: ffff987d11b30000
RIP: 0010:[<ffffffff9e9150ca>]  [<ffffffff9e9150ca>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff987d11b33bf0  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000800009e9c5210 RCX: ffff987d1d620740
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff987d11b33c28 R08: ffff98767451e400 R09: ffffffff9eb7f629
R10: ffff987d1d8df160 R11: fffff8e612a0ee00 R12: 0000000000000286
R13: 0000000000000018 R14: ffff987d11b33b60 R15: 0000000000000286
FS:  00007f27ecf04900(0000) GS:ffff987d1d8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f27ebb6de70 CR3: 0000000815b20000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9e9c5210>] ? drain_pages+0xb0/0xb0
[<ffffffff9e915219>] on_each_cpu_mask+0x29/0x70
[<ffffffff9e9c1fd5>] drain_all_pages+0xb5/0xc0
[<ffffffff9ef746b6>] __alloc_pages_slowpath+0x510/0x724
[<ffffffff9e9c6b84>] __alloc_pages_nodemask+0x404/0x420
[<ffffffff9e897e8d>] copy_process+0x1dd/0x1a50
[<ffffffff9e8998b1>] do_fork+0x91/0x330
[<ffffffff9e899bd6>] SyS_clone+0x16/0x20
[<ffffffff9ef8d2b4>] stub_clone+0x44/0x70
[<ffffffff9ef8d172>] ? tracesys+0xa6/0xcc
Code: 48 63 35 3e 68 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 d7 54 9f f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 0d 68 c4 00 eb b7 0f b6 4d cc 4c 89 

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [httpd:14512]
Modules linked in: ip6table_filter ip6_tables xt_time nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack tcp_diag inet_diag sysdig_probe(OE) ipt_REJECT nf_reject_ipv4 xt_owner xt_multiport iptable_filter ext4 mbcache jbd2 loop dm_mirror dm_region_hash dm_log dm_mod sg ppdev pcspkr parport_pc parport i2c_piix4 joydev ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper virtio_blk syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata e1000 serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy
CPU: 1 PID: 14512 Comm: httpd Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff88330b09a0e0 ti: ffff883019478000 task.ti: ffff883019478000
RIP: 0010:[<ffffffff967175ce>]  [<ffffffff967175ce>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff88301947bbf8  EFLAGS: 00000202
RAX: 0000000000000005 RBX: 00000000000000fc RCX: ffff8833a7d60400
RDX: 0000000000000005 RSI: 0000000000000006 RDI: 0000000000000000
RBP: ffff88301947bc30 R08: ffff8830fa1b1000 R09: ffffffff96985179
R10: ffff8833a7c5f160 R11: ffffe8338d1ec700 R12: 0000000000000292
R13: ffff88301947bba8 R14: 0000000001320122 R15: 000000000e2e0c07
FS:  00002aab6da23440(0000) GS:ffff8833a7c40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002ba67c2001f0 CR3: 0000000332762000 CR4: 00000000000006e0
Call Trace:
[<ffffffff967c78b0>] ? drain_pages+0xb0/0xb0
[<ffffffff96717719>] on_each_cpu_mask+0x29/0x70
[<ffffffff967c4725>] drain_all_pages+0xb5/0xc0
[<ffffffff96d7b315>] __alloc_pages_slowpath+0x510/0x729
[<ffffffff967c91a6>] __alloc_pages_nodemask+0x436/0x450
[<ffffffff9669930d>] copy_process+0x1dd/0x1a70
[<ffffffff9669ad51>] do_fork+0x91/0x330
[<ffffffff96d92e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff9669b076>] SyS_clone+0x16/0x20
[<ffffffff96d932b4>] stub_clone+0x44/0x70
[<ffffffff96d93166>] ? tracesys+0xa6/0xcc
Code: 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 10 35 97 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 ad 7b c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee