NMI watchdog: BUG: soft lockup - CPU#47 stuck for 23s! [systemd:1]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vhost_net vhost macvtap macvlan ebt_arp tun fuse bnx2fc cnic uio scsi_transport_iscsi fcoe libfcoe libfc scsi_transport_fc scsi_tgt nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack openvswitch ip_set nfnetlink ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass
crc32_pclmul ghash_clmulni_intel aesni_intel lrw iTCO_wdt gf128mul dm_multipath glue_helper iTCO_vendor_support ablk_helper cryptd dell_smbios dcdbas sparse_keymap dm_mod pcspkr joydev sg mei_me mei lpc_ich i2c_i801 shpchp wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_pad acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt crct10dif_pclmul fb_sys_fops crct10dif_common ttm crc32c_intel drm ahci tg3 bnxt_en libahci megaraid_sas libata i2c_core devlink ptp pps_core 8021q garp mrp bridge stp llc bonding
CPU: 47 PID: 1 Comm: systemd Kdump: loaded Tainted: G        W      ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.4.8 11/26/2019
task: ffff9b25e7678000 ti: ffff9ae667364000 task.ti: ffff9ae667364000
RIP: 0010:[<ffffffffabc0b89c>]  [<ffffffffabc0b89c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff9ae667367ce0  EFLAGS: 00000246
RAX: ffff9ae6bfdce440 RBX: ffff9b64bc3d5b80 RCX: ffff9b24baccfc00
RDX: ffff9b26023d0b70 RSI: 0000000000000001 RDI: ffffe1e2b9d42dc0
RBP: ffff9ae667367d20 R08: ffff9ae6bfdce400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 000000002af193f4 R14: ffff9ae667367c38 R15: ffff9ae667367c38
FS:  00007fe2e2a1b940(0000) GS:ffff9b64bc3c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000556a4b64ba40 CR3: 0000007f71602000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffabba7832>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffabba32fe>] ? lru_cache_add+0xe/0x10
[<ffffffffabc0cff8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffabc0d374>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffabb1c1da>] cgroup_destroy_locked+0xea/0x370
[<ffffffffabb1c482>] cgroup_rmdir+0x22/0x40
[<ffffffffabc2954c>] vfs_rmdir+0xdc/0x150
[<ffffffffabc2cad1>] do_rmdir+0x1f1/0x220
[<ffffffffabc1d18e>] ? ____fput+0xe/0x10
[<ffffffffabc2dd06>] SyS_rmdir+0x16/0x20
[<ffffffffac11fa11>] tracesys+0x9d/0xc3
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [slurmstepd:12349]
Modules linked in: squashfs loop overlay(T) nfs_layout_nfsv41_files osc(OE) mgc(OE) lustre(OE) lmv(OE) fld(OE) mdc(OE) fid(OE) lov(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) sha512_ssse3 sha512_generic crypto_null libcfs(OE) dell_rbu rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dm_mirror dm_region_hash dm_log dm_mod ses enclosure sd_mod sg intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd serio_raw pcspkr uas usb_storage smartpqi scsi_transport_sas nvme hpilo hpwdt lpc_ich shpchp wmi acpi_power_meter rpcrdma ib_isert iscsi_target_mod target_core_mod crc_t10dif crct10dif_generic ib_iser libiscsi scsi_transport_iscsi ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm
iw_cm nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ipmi_devintf hfi1(OE) i2c_algo_bit i2c_core crct10dif_pclmul rdmavt(OE) crct10dif_common crc32c_intel bnxt_en ib_core ipmi_si fjes ipmi_msghandler
CPU: 24 PID: 12349 Comm: slurmstepd Tainted: G        W  OEL ------------ T 3.10.0-514.10.2.el7.x86_64 #1
Hardware name: HPE ProLiant DL560 Gen10/ProLiant DL560 Gen10, BIOS U34 04/21/2022
task: ffff8a9f3e7fedd0 ti: ffff8a575f648000 task.ti: ffff8a575f648000
RIP: 0010:[<ffffffff811efaec>]  [<ffffffff811efaec>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff8a575f64bd10  EFLAGS: 00000246
RAX: ffff8bfc2f959c40 RBX: ffff89fc3f591a20 RCX: ffff8bad82f47800
RDX: ffff8a015a319b20 RSI: 0000000000000001 RDI: ffffea0b98a66c80
RBP: ffff8a575f64bd50 R08: ffff8bfc2f959c00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000aa30acbc R14: ffff8a575f64bc68 R15: ffff8a575f64bc68
FS:  00002b751702d300(0000) GS:ffff89fc3f580000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000c9ec30 CR3: 00000142d4f97000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8a575f64bd50 ffffffff81194873 ffffffffffffffff ffffea0b98a66c80
ffff8afc398c2440 ffff8b003ffda540 ffff8a015a319b20 ffffea0b98a66ca0
ffff8a575f64bdd0 ffffffff811f12d8 0000000200000002 ffff8bad82f47900
Call Trace:
[<ffffffff81194873>] ? isolate_lru_page+0x153/0x1f0
[<ffffffff811f12d8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f1654>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d257>] cgroup_destroy_locked+0xe7/0x360
[<ffffffff8110d4f2>] cgroup_rmdir+0x22/0x40
[<ffffffff8120a278>] vfs_rmdir+0xa8/0x100
[<ffffffff8120f315>] do_rmdir+0x1a5/0x200
[<ffffffff81692305>] ? do_page_fault+0x35/0x90
[<ffffffff812104c6>] SyS_rmdir+0x16/0x20
[<ffffffff81696b09>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [runc:72325]
Modules linked in: tcp_diag udp_diag inet_diag xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) binfmt_misc ppdev parport fuse dm_mirror dm_region_hash dm_log dm_mod iTCO_wdt iTCO_vendor_support skx_edac coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul xfs libcrc32c ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev ses enclosure scsi_transport_sas pcspkr sg mei_me lpc_ich i2c_i801 mei ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ast i2c_algo_bit
drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops ttm i40e drm ahci megaraid_sas libahci ptp pps_core libata drm_panel_orientation_quirks wmi nfit libnvdimm [last unloaded: vmnet]
CPU: 4 PID: 72325 Comm: runc Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: Inspur NF5280M5/YZMB-01360-102, BIOS 3.1.1 06/10/2019
task: ffff91045224a080 ti: ffff910104cc4000 task.ti: ffff910104cc4000
RIP: 0010:[<ffffffff9823181c>]  [<ffffffff9823181c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff910104cc7ce0  EFLAGS: 00000246
RAX: ffff913fe6361448 RBX: ffff9127f66d0000 RCX: ffff9127f66d0000
RDX: ffff9120f5c78cb0 RSI: 0000000000000001 RDI: ffffdffb69fe32c0
RBP: ffff910104cc7d20 R08: ffff913fe6361400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 000000006849939b R14: ffff910104cc7c38 R15: ffff910104cc7c38
FS:  00007f1ead747740(0000) GS:ffff911ffe700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000582a1d000 CR3: 000000097f21e000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff981ca0c2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff981c588e>] ? lru_cache_add+0xe/0x10
[<ffffffff98232f78>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff982332f4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff9812573a>] cgroup_destroy_locked+0xea/0x370
[<ffffffff981259e2>] cgroup_rmdir+0x22/0x40
[<ffffffff9824dc2c>] vfs_rmdir+0xdc/0x150
[<ffffffff982530d1>] do_rmdir+0x1f1/0x220
[<ffffffff98247034>] ? SYSC_newfstatat+0x24/0x60
[<ffffffff98254335>] SyS_unlinkat+0x25/0x40
[<ffffffff98774ddb>] system_call_fastpath+0x22/0x27
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#17 stuck for 22s! [runc:14008]
Modules linked in: binfmt_misc xt_nat veth nvidia_uvm(OE) fuse xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype iptable_filter br_netfilter bridge stp llc mgc(OE) lustre(OE) lmv(OE) mdc(OE) fid(OE) osc(OE) lov(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_umad(OE) mlx5_fpga_tools(OE) dm_mirror dm_region_hash dm_log dm_mod nvidia_drm(POE) nvidia_modeset(POE) iTCO_wdt iTCO_vendor_support nvidia(POE) sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel
lrw gf128mul glue_helper ablk_helper cryptd squashfs loop snd_hda_codec_hdmi snd_hda_intel snd_hda_codec pcspkr snd_hda_core snd_hwdep joydev snd_seq snd_seq_device snd_pcm snd_timer sg snd i2c_i801 soundcore lpc_ich mei_me mei ioatdma ipmi_si ipmi_devintf ipmi_msghandler acpi_pad acpi_power_meter knem(OE) auth_rpcgss sunrpc ip_tables xfs libcrc32c mlx4_ib(OE) mlx4_en(OE) sd_mod crc_t10dif crct10dif_generic mxm_wmi mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm igb mlx4_core(OE) crct10dif_pclmul crct10dif_common crc32c_intel drm ahci dca libahci i2c_algo_bit mlx5_core(OE) libata mlxfw(OE) drm_panel_orientation_quirks devlink mlx_compat(OE) ptp pps_core wmi overlay(T)
CPU: 17 PID: 14008 Comm: runc Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: Supermicro SYS-4028GR-TR2-1-EC028/X10DRG-O+-CPU, BIOS 3.1 07/13/2018
task: ffff8f17c69cc100 ti: ffff8e96de32c000 task.ti: ffff8e96de32c000
RIP: 0010:[<ffffffffae63181e>]  [<ffffffffae63181e>] mem_cgroup_move_account+0x3e/0x160
RSP: 0018:ffff8e96de32fce0  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8f943eed7c80 RCX: ffff8f794e43f800
RDX: ffff8e97b7ef14d0 RSI: 0000000000000001 RDI: ffffdfa9615c5340
RBP: ffff8e96de32fd20 R08: ffff8f13ee0d9000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 000000004b18c463 R14: ffff8e96de32fc38 R15: ffff8e96de32fc38
FS:  00007f064180d740(0000) GS:ffff8f943eec0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f21a79a3a28 CR3: 00000080bde28000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffae5ca0c2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffae5c588e>] ? lru_cache_add+0xe/0x10
[<ffffffffae632f78>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffae6332f4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffae52573a>] cgroup_destroy_locked+0xea/0x370
[<ffffffffae5259e2>] cgroup_rmdir+0x22/0x40
[<ffffffffae64dc2c>] vfs_rmdir+0xdc/0x150
[<ffffffffae6530d1>] do_rmdir+0x1f1/0x220
[<ffffffffae654335>] SyS_unlinkat+0x25/0x40
[<ffffffffaeb74ddb>] system_call_fastpath+0x22/0x27
Code: 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 19 c0 <85> c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 75 2d 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [systemd:1]
Modules linked in: squashfs loop veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c xt_addrtype br_netfilter bridge overlay(T) rpcsec_gss_krb5 nfsv4 dns_resolver nfsv3 nfs fscache 8021q garp mrp stp llc ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support rpcrdma mxm_wmi ib_isert iscsi_target_mod sb_edac edac_core coretemp intel_rapl ib_iser iosf_mbi kvm_intel libiscsi kvm scsi_transport_iscsi irqbypass ib_srpt crc32_pclmul target_core_mod ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad sg pcspkr mei_me rdma_cm mei ioatdma i2c_i801 shpchp lpc_ich ib_cm iw_cm
ipmi_si ipmi_devintf ipmi_msghandler wmi mlx4_ib acpi_power_meter nfsd ib_core auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 mlx4_en sd_mod crc_t10dif crct10dif_generic ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mpt3sas mlx4_core igb crct10dif_pclmul crct10dif_common crc32c_intel ptp pps_core raid_class dca i2c_algo_bit scsi_transport_sas devlink i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 1 Comm: systemd Tainted: G               ------------ T 3.10.0-693.5.2.el7.x86_64 #1
Hardware name: Inspur NF5170M4/YZMB-00370-104, BIOS 4.1.13 12/01/2017
task: ffff8880ceff0000 ti: ffff88014e430000 task.ti: ffff88014e430000
RIP: 0010:[<ffffffff811f1bfc>]  [<ffffffff811f1bfc>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff88014e433d10  EFLAGS: 00000246
RAX: ffff88fe7dab4440 RBX: ffff887e7f711ae0 RCX: ffff88fe7dab5c00
RDX: ffff8801fb39e5a0 RSI: 0000000000000001 RDI: ffffea0014c79680
RBP: ffff88014e433d50 R08: ffff88fe7dab4400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 0000000070f0ee17 R14: ffff88014e433c68 R15: ffff88014e433c68
FS:  00007fbd8bc2a940(0000) GS:ffff887e7f700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000564c6886714a CR3: 0000007e57b92000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff88014e433d50 ffffffff811959d2 ffffffff8119151e ffffea0014c79680
ffff887e7c49b440 ffff88807ffda540 ffff8801fb39e5a0 ffffea0014c796a0
ffff88014e433dd0 ffffffff811f3368 0000000200000000 ffff88fe7dab5d00
Call Trace:
[<ffffffff811959d2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff8119151e>] ? lru_cache_add+0xe/0x10
[<ffffffff811f3368>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f36e4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d767>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da12>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f5a6>] vfs_rmdir+0xd6/0x150
[<ffffffff81212241>] do_rmdir+0x1f1/0x220
[<ffffffff810ad25c>] ? task_work_run+0xac/0xf0
[<ffffffff81213446>] SyS_rmdir+0x16/0x20
[<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [slurmstepd:186222]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver squashfs loop overlay(T) mmfs26(OE) mmfslinux(OE) tracedev(OE) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu dm_mirror dm_region_hash dm_log skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm iTCO_wdt dell_smbios iTCO_vendor_support sparse_keymap irqbypass dcdbas crc32_pclmul crc32c_intel mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ghash_clmulni_intel drm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr mei_me mei shpchp i2c_i801 lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_pad acpi_power_meter binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data
dm_bufio dm_mod udf crc_itu_t vfat fat br_netfilter bridge stp llc xfs isci libsas qla3xxx hpsa e1000e ixgbe(OE) igb(OE) i2c_algo_bit i2c_core dca aacraid aic79xx ata_piix mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun sd_mod crc_t10dif crct10dif_generic sg crct10dif_pclmul crct10dif_common ahci libahci libata i40e(OE) megaraid_sas tg3(OE) ptp pps_core [last unloaded: ipmi_msghandler]
CPU: 4 PID: 186222 Comm: slurmstepd Tainted: G        W  OEL ------------ T 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R440/04JN2K, BIOS 2.21.1 03/07/2024
task: ffff88501a4bbf40 ti: ffff8831c07ac000 task.ti: ffff8831c07ac000
RIP: 0010:[<ffffffff811f1c8c>]  [<ffffffff811f1c8c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff8831c07afd10  EFLAGS: 00000246
RAX: ffff8803451f4440 RBX: ffff882fb8a91ae0 RCX: ffff8803451f4000
RDX: ffff8830ed9138c0 RSI: 0000000000000001 RDI: ffffea010b24e300
RBP: ffff8831c07afd50 R08: ffff8803451f4400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000bb286859 R14: ffff8831c07afc68 R15: ffff8831c07afc68
FS:  00002aaaaaafd180(0000) GS:ffff882fb8a80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffff7ff8000 CR3: 0000002f89284000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8831c07afd50 ffffffff811959a2 ffffffff811914ee ffffea010b24e300
ffff883222936440 ffff88607bfd7540 ffff8830ed9138c0 ffffea010b24e320
ffff8831c07afdd0 ffffffff811f33f8 0000000200000001 ffff8803451f4100
Call Trace:
[<ffffffff811959a2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff811914ee>] ? lru_cache_add+0xe/0x10
[<ffffffff811f33f8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff816b0325>] ? do_page_fault+0x35/0x90
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [systemd:1]
Modules linked in: xt_nat veth xt_addrtype br_netfilter cmac arc4 md4 nls_utf8 cifs ccm xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter fuse rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_uverbs(OE) ib_umad(OE) mlx5_fpga_tools(OE) mlx5_ib(OE) mlx5_core(OE) mlxfw(OE) mlx4_en(OE) overlay(T) sunrpc raid0 iTCO_wdt iTCO_vendor_support skx_edac coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd
pcspkr joydev nvme sg nvme_core mei_me mei lpc_ich shpchp i2c_i801 ipmi_ssif wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter knem(OE) ip_tables xfs libcrc32c mlx4_ib(OE) ib_core(OE) sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt mlx4_core(OE) fb_sys_fops ttm i40e drm ahci devlink megaraid_sas libahci mlx_compat(OE) ptp libata i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 13 PID: 1 Comm: systemd Kdump: loaded Tainted: G        W  OE  ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: Sugon I840-G30/80P48-US, BIOS 0THSY016 04/16/2018
task: ffff984b13998000 ti: ffff982b935dc000 task.ti: ffff982b935dc000
RIP: 0010:[<ffffffffbc00b954>]  [<ffffffffbc00b954>] mem_cgroup_move_account+0x94/0x160
RSP: 0018:ffff982b935dfce0  EFLAGS: 00000246
RAX: 0000000000000003 RBX: ffff984a3e955b80 RCX: ffff9832b7e46400
RDX: ffff9832b7e46400 RSI: 0000000000000001 RDI: fffff88e11644cc0
RBP: ffff982b935dfd20 R08: ffff982bbfdae400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000e8ff2fae R14: ffff982b935dfc38 R15: ffff982b935dfc38
FS:  00007f862f10a940(0000) GS:ffff984a3e940000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c02f64cfa0 CR3: 0000005ff70e0000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffbbfa7892>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffbbfa335e>] ? lru_cache_add+0xe/0x10
[<ffffffffbc00d058>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffbc00d3d4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffbbf1c1da>] cgroup_destroy_locked+0xea/0x370
[<ffffffffbbf1c482>] cgroup_rmdir+0x22/0x40
[<ffffffffbc0295ac>] vfs_rmdir+0xdc/0x150
[<ffffffffbc02cb31>] do_rmdir+0x1f1/0x220
[<ffffffffbc01d1ee>] ? ____fput+0xe/0x10
[<ffffffffbbeb7850>] ? task_work_run+0xc0/0xe0
[<ffffffffbc02dd66>] SyS_rmdir+0x16/0x20
[<ffffffffbc51f7d5>] system_call_fastpath+0x1c/0x21
Code: 83 fd ff ff 48 83 c4 18 44 89 f8 5b 41 5c 41 5d 41 5e 41 5f 5d c3 90 48 8b 53 08 4c 39 ea 75 ca 4d 8d bd e4 01 00 00 41 83 e6 01 <4c> 89 45 c0 44 89 4d cc 48 89 55 d0 4c 89 ff e8 68 9d 50 00 4d 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [runc:77708]
Modules linked in: xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipport ip_set_hash_ipportip ip_set dummy nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack cambricon_gdrdrv(OE) joydev iTCO_wdt iTCO_vendor_support intel_powerclamp coretemp kvm
irqbypass crc32_pclmul vfat fat ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr cp210x sg mei_me i2c_i801 mei wmi ipmi_si ipmi_devintf ipmi_msghandler pinctrl_lewisburg pinctrl_intel acpi_cpufreq acpi_power_meter cambricon_drv(OE) nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ast i2c_algo_bit drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect serio_raw sysimgblt fb_sys_fops ttm i40e drm ahci libahci megaraid_sas ptp libata drm_panel_orientation_quirks pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 77708 Comm: runc Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: Inspur NF5468M6/NF5468M6, BIOS 06.01.02 06/28/2022
task: ffff9c2ace9030c0 ti: ffff9c8725f94000 task.ti: ffff9c8725f94000
RIP: 0010:[<ffffffffa7c31860>]  [<ffffffffa7c31860>] mem_cgroup_move_account+0x80/0x160
RSP: 0018:ffff9c8725f97ce0  EFLAGS: 00000202
RAX: 0000000000000003 RBX: ffff9b287fa97c80 RCX: ffff9b9e2a243c00
RDX: ffff9c2b6f185720 RSI: 0000000000000001 RDI: ffffeb2405415c80
RBP: ffff9c8725f97d20 R08: ffff9ba5f8eb9400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 000000009094e96a R14: ffff9c8725f97c38 R15: ffff9c8725f97c38
FS:  00007f2e29502740(0000) GS:ffff9b287fa80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb899b67000 CR3: 0000007c32faa000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa7bca0c2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffa7bc588e>] ? lru_cache_add+0xe/0x10
[<ffffffffa7c32f78>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffa7c332f4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffa7b2573a>] cgroup_destroy_locked+0xea/0x370
[<ffffffffa7b259e2>] cgroup_rmdir+0x22/0x40
[<ffffffffa7c4dc2c>] vfs_rmdir+0xdc/0x150
[<ffffffffa7c530d1>] do_rmdir+0x1f1/0x220
[<ffffffffa7c47034>] ? SYSC_newfstatat+0x24/0x60
[<ffffffffa7c54335>] SyS_unlinkat+0x25/0x40
[<ffffffffa8174ddb>] system_call_fastpath+0x22/0x27
Code: 23 fe 4c 89 e6 4c 89 c7 e8 8e fd ff ff 4c 89 e6 4c 89 ef e8 83 fd ff ff 48 83 c4 18 44 89 f8 5b 41 5c 41 5d 41 5e 41 5f 5d c3 90 <48> 8b 53 08 4c 39 ea 75 ca 4d 8d bd ec 01 00 00 41 83 e6 01 4c 

NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [slurmstepd:297124]
Modules linked in: 8021q garp mrp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs fscache ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm rpcrdma rdma_ucm ib_uverbs ib_iser dm_mirror rdma_cm opa_vnic dm_region_hash iw_cm dm_log ib_umad libiscsi dm_mod ib_ipoib scsi_transport_iscsi ib_cm skx_edac intel_powerclamp iTCO_wdt coretemp iTCO_vendor_support intel_rapl iosf_mbi kvm hfi1 irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel rdmavt lrw gf128mul glue_helper ablk_helper cryptd ib_core pcspkr mei_me sg joydev
mei ipmi_si i2c_i801 shpchp lpc_ich wmi ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic i2c_algo_bit drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul sysimgblt crct10dif_common fb_sys_fops ixgbe crc32c_intel ttm ahci drm mdio libahci ptp libata pps_core i2c_core dca nfit libnvdimm fuse
CPU: 3 PID: 297124 Comm: slurmstepd Kdump: loaded Tainted: G             L ------------   3.10.0-862.9.1.el7.x86_64 #1
Hardware name: Advanced Clustering S2600BPB/S2600BPB, BIOS SE5C620.86B.00.01.0013.C1.030920180427 03/09/2018
task: ffffa0fba633af70 ti: ffffa1152af48000 task.ti: ffffa1152af48000
RIP: 0010:[<ffffffffa660c0bc>]  [<ffffffffa660c0bc>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffffa1152af4bce0  EFLAGS: 00000246
RAX: ffffa0d19ee6b840 RBX: ffffa0f03dad5bc0 RCX: ffffa0f027a00000
RDX: ffffa0f175433620 RSI: 0000000000000001 RDI: fffffdd3eaecd880
RBP: ffffa1152af4bd20 R08: ffffa0d19ee6b800 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000a3b59b84 R14: ffffa1152af4bc38 R15: ffffa1152af4bc38
FS:  00002af5520d1d40(0000) GS:ffffa0f03dac0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002af55c002378 CR3: 0000002dfe3c0000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa65a8052>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffa65a3b1e>] ? lru_cache_add+0xe/0x10
[<ffffffffa660d818>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffa660db94>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffa651c9ca>] cgroup_destroy_locked+0xea/0x370
[<ffffffffa651cc72>] cgroup_rmdir+0x22/0x40
[<ffffffffa6629d6c>] vfs_rmdir+0xdc/0x150
[<ffffffffa662d2f1>] do_rmdir+0x1f1/0x220
[<ffffffffa65f6ec2>] ? kmem_cache_free+0x1e2/0x200
[<ffffffffa662e526>] SyS_rmdir+0x16/0x20
[<ffffffffa6b20795>] system_call_fastpath+0x1c/0x21
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [slurmstepd:352798]
Modules linked in: binfmt_misc nvidia_uvm(OE) nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi ext4 kvm_intel kvm mbcache jbd2 irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq ipmi_ssif joydev snd_seq_device pcspkr snd_pcm sg snd_timer snd mei_me mei lpc_ich i2c_i801 soundcore shpchp ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_cpufreq acpi_pad acpi_power_meter ip_tables xfs libcrc32c nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) sd_mod crc_t10dif crct10dif_generic ast i2c_algo_bit drm_kms_helper crct10dif_pclmul ttm crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ahci megaraid_sas drm ixgbe libahci libata mdio ptp i2c_core pps_core dca wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 352798 Comm: slurmstepd Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: Supermicro SYS-7049GP-TRT/X11DPG-QT, BIOS 3.1 05/23/2019
task: ffff8e3ab054af70 ti: ffff8e3941244000 task.ti: ffff8e3941244000
RIP: 0010:[<ffffffffa2e0b8fc>]  [<ffffffffa2e0b8fc>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff8e3941247ce0  EFLAGS: 00000246
RAX: ffff8e6b26273040 RBX: ffff8e383ca95b80 RCX: ffff8e6b26271800
RDX: ffff8e39bcd08f40 RSI: 0000000000000001 RDI: ffffd80c0e223d00
RBP: ffff8e3941247d20 R08: ffff8e6b26273000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000b717f2b7 R14: ffff8e3941247c38 R15: ffff8e3941247c38
FS:  00002b6f91875580(0000) GS:ffff8e383ca80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fccdfeba810 CR3: 0000003eb8f70000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa2da7892>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffa2da335e>] ? lru_cache_add+0xe/0x10
[<ffffffffa2e0d058>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffa2e0d3d4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffa2d1c1da>] cgroup_destroy_locked+0xea/0x370
[<ffffffffa2d1c482>] cgroup_rmdir+0x22/0x40
[<ffffffffa2e295ac>] vfs_rmdir+0xdc/0x150
[<ffffffffa2e2cb31>] do_rmdir+0x1f1/0x220
[<ffffffffa2e2dd66>] SyS_rmdir+0x16/0x20
[<ffffffffa331f7d5>] system_call_fastpath+0x1c/0x21
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#98 stuck for 22s! [runc:50756]
Modules linked in: xt_nat veth nf_conntrack_ftp udp_diag tcp_diag inet_diag nvidia_uvm(OE) binfmt_misc nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support vfat fat intel_powerclamp coretemp kvm_intel kvm irqbypass crc32_pclmul
ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd nvidia_drm(POE) nvidia_modeset(POE) dm_service_time nvidia(POE) snd_hda_codec_hdmi cdc_ether usbnet snd_hda_intel joydev pcspkr mii snd_hda_codec sg snd_hda_core snd_hwdep dm_multipath snd_seq snd_seq_device snd_pcm ipmi_ssif snd_timer snd mei_me mei i2c_i801 soundcore ipmi_si ipmi_devintf ipmi_msghandler pinctrl_lewisburg pinctrl_intel acpi_cpufreq acpi_pad acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod lpfc ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crc32c_intel nvmet_fc drm nvmet igb crc_t10dif crct10dif_generic crct10dif_pclmul nvme_fc ahci nvme_fabrics nvme_core megaraid_sas libahci ptp drm_panel_orientation_quirks pps_core scsi_transport_fc
dca libata i2c_algo_bit scsi_tgt crct10dif_common wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 98 PID: 50756 Comm: runc Tainted: P        W  OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: Nettrix X640 G40/60WB32, BIOS NKMH041018 10/08/2021
task: ffff8c88990430c0 ti: ffff8c80ae6b8000 task.ti: ffff8c80ae6b8000
RIP: 0010:[<ffffffffa363181c>]  [<ffffffffa363181c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff8c80ae6bbce0  EFLAGS: 00000246
RAX: ffff8cc13324e448 RBX: ffff8cc1bee97c80 RCX: ffff8cc13324e800
RDX: ffff8c830255a3f0 RSI: 0000000000000001 RDI: ffffdc2338368fc0
RBP: ffff8c80ae6bbd20 R08: ffff8cc13324e400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 0000000001204316 R14: ffff8c80ae6bbc38 R15: ffff8c80ae6bbc38
FS:  00007fa565f53740(0000) GS:ffff8cc1bee80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00028b021 CR3: 0000004259276000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa35ca0c2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffa35c588e>] ? lru_cache_add+0xe/0x10
[<ffffffffa3632f78>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffa36332f4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffa352573a>] cgroup_destroy_locked+0xea/0x370
[<ffffffffa35259e2>] cgroup_rmdir+0x22/0x40
[<ffffffffa364dc2c>] vfs_rmdir+0xdc/0x150
[<ffffffffa36530d1>] do_rmdir+0x1f1/0x220
[<ffffffffa3647034>] ? SYSC_newfstatat+0x24/0x60
[<ffffffffa3654335>] SyS_unlinkat+0x25/0x40
[<ffffffffa3b74ddb>] system_call_fastpath+0x22/0x27
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#29 stuck for 23s! [slurmstepd:307451]
Modules linked in: squashfs loop overlay(T) mmfs26(OE) mmfslinux(OE) tracedev(OE) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu dm_mirror dm_region_hash dm_log iTCO_wdt dell_smbios iTCO_vendor_support sparse_keymap dcdbas skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul crc32c_intel ghash_clmulni_intel mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr mei_me mei shpchp i2c_i801 lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data dm_bufio dm_mod udf crc_itu_t vfat fat br_netfilter
bridge stp llc xfs isci libsas qla3xxx hpsa e1000e ixgbe(OE) igb(OE) i2c_algo_bit i2c_core dca aacraid aic79xx ata_piix mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun sd_mod crc_t10dif crct10dif_generic sg crct10dif_pclmul crct10dif_common i40e(OE) ahci tg3(OE) libahci ptp megaraid_sas pps_core libata [last unloaded: ipmi_msghandler]
CPU: 29 PID: 307451 Comm: slurmstepd Tainted: G           OE  ------------ T 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R440/0WKGTH, BIOS 2.5.4 01/14/2020
task: ffff8830e0e40fd0 ti: ffff88383fdd8000 task.ti: ffff88383fdd8000
RIP: 0010:[<ffffffff811f1c8c>]  [<ffffffff811f1c8c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff88383fddbd10  EFLAGS: 00000246
RAX: ffff882f58a95840 RBX: ffff885f7c191ae0 RCX: ffff880181bf0400
RDX: ffff8830aca6cbb0 RSI: 0000000000000001 RDI: ffffea010e7b2ec0
RBP: ffff88383fddbd50 R08: ffff882f58a95800 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 0000000013ab72bc R14: ffff88383fddbc68 R15: ffff88383fddbc68
FS:  00002aaaaaafd180(0000) GS:ffff885f7c180000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002aac9a201078 CR3: 00000030d70e2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff88383fddbd50 ffffffff811959a2 ffffffff811914ee ffffea010e7b2ec0
ffff8831ef151440 ffff88603ffd7540 ffff8830aca6cbb0 ffffea010e7b2ee0
ffff88383fddbdd0 ffffffff811f33f8 0000000200000001 ffff880181bf0500
Call Trace:
[<ffffffff811959a2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff811914ee>] ? lru_cache_add+0xe/0x10
[<ffffffff811f33f8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#19 stuck for 22s! [slurmstepd:52248]
Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) mgc(OE) lustre(OE) lmv(OE) mdc(OE) fid(OE) osc(OE) lov(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c br_netfilter bridge 8021q garp mrp stp llc overlay(T) ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ses enclosure cdc_ether joydev scsi_transport_sas usbnet mii sg mei_me mei shpchp i2c_i801 wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter nfsd binfmt_misc auth_rpcgss nfs_acl
lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul drm crct10dif_common crc32c_intel ahci serio_raw libahci mlx5_core(OE) mlxfw(OE) ptp libata megaraid_sas pps_core auxiliary(OE) mlx_compat(OE) i2c_core devlink [last unloaded: tracedev]
CPU: 19 PID: 52248 Comm: slurmstepd Tainted: G        W  OE  ------------ T 3.10.0-693.5.2.el7.x86_64 #1
Hardware name: Lenovo ThinkSystem SR650 V2/7Z73CTO1WW, BIOS AFE112A-1.02 05/10/2021
task: ffff89f7e0acdee0 ti: ffff886c81b5c000 task.ti: ffff886c81b5c000
RIP: 0010:[<ffffffff811f1c0f>]  [<ffffffff811f1c0f>] mem_cgroup_move_account+0x4f/0x160
RSP: 0018:ffff886c81b5fd10  EFLAGS: 00000246
RAX: 0000000000000003 RBX: ffff89f83ded1ae0 RCX: ffff89f836ba1000
RDX: ffff8803bb8921c0 RSI: 0000000000000001 RDI: ffffea0511248700
RBP: ffff886c81b5fd50 R08: ffff89ba15ff4800 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 0000000042ffe6a5 R14: ffff886c81b5fc68 R15: ffff886c81b5fc68
FS:  00002b65ea80cfc0(0000) GS:ffff89f83dec0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7bb9a98000 CR3: 00000020e513d000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff886c81b5fd50 ffffffff811959d2 ffffffff8119151e ffffea0511248700
ffff89d6f4b91430 ffff8a003ffd8540 ffff8803bb8921c0 ffffea0511248720
ffff886c81b5fdd0 ffffffff811f3368 0000000200000000 ffff89f836ba1100
Call Trace:
[<ffffffff811959d2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff8119151e>] ? lru_cache_add+0xe/0x10
[<ffffffff811f3368>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f36e4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d767>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da12>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f5a6>] vfs_rmdir+0xd6/0x150
[<ffffffff81212241>] do_rmdir+0x1f1/0x220
[<ffffffff81213446>] SyS_rmdir+0x16/0x20
[<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
Code: 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 19 c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff <a8> 02 75 2d f0 80 23 fe 4c 89 e6 4c 89 c7 e8 8e fd ff ff 4c 89 

NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [systemd:1]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM tun devlink overlay(T) ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event coretemp ppdev pcspkr i2c_piix4 sg vmw_balloon joydev snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq vmw_vmci snd_seq_device snd_pcm snd_timer iosf_mbi
snd crc32_pclmul ghash_clmulni_intel aesni_intel soundcore lrw gf128mul glue_helper ablk_helper cryptd parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt nfit fb_sys_fops ttm libata libnvdimm drm crct10dif_pclmul crct10dif_common crc32c_intel mptspi scsi_transport_spi mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 1 Comm: systemd Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff926478770000 ti: ffff926478778000 task.ti: ffff926478778000
RIP: 0010:[<ffffffff8123cfa7>]  [<ffffffff8123cfa7>] mem_cgroup_move_account+0x37/0x160
RSP: 0018:ffff92647877bcd8  EFLAGS: 00010246
RAX: ffff92646bf04c48 RBX: fffffb1b4125af80 RCX: ffff9263aa750400
RDX: ffff926476db68c0 RSI: 0000000000000001 RDI: fffffb1b40f0de00
RBP: ffff92647877bd18 R08: ffff92646bf04c00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000001a2b1849
R13: ffff92647877bc30 R14: ffff92647877bc30 R15: ffff92647877bc20
FS:  00007f3a7aa05940(0000) GS:ffff926479600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f97c2ffce78 CR3: 0000000035dca000 CR4: 00000000001607f0
Call Trace:
[<ffffffff811d1e42>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff811cd4ee>] ? lru_cache_add+0xe/0x10
[<ffffffff8123e8e8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff8123ec64>] mem_cgroup_css_offline+0x84/0x170
[<ffffffff8112ae8a>] cgroup_destroy_locked+0xea/0x370
[<ffffffff8112b132>] cgroup_rmdir+0x22/0x40
[<ffffffff8125b4ec>] vfs_rmdir+0xdc/0x150
[<ffffffff81260171>] do_rmdir+0x1f1/0x220
[<ffffffff8125007e>] ? ____fput+0xe/0x10
[<ffffffff810c2760>] ? task_work_run+0xc0/0xe0
[<ffffffff812613a6>] SyS_rmdir+0x16/0x20
[<ffffffff81793f92>] system_call_fastpath+0x25/0x2a
Code: 41 57 41 56 41 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 <f0> 0f ba 2b 00 19 c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea 

NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [systemd:1]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) nvidia_uvm(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nfsv3 nfs_acl nfs lockd grace fscache sunrpc nvidia_drm(POE) nvidia_modeset(POE) iTCO_wdt iTCO_vendor_support nvidia(POE) vfat fat coretemp kvm_intel ext4 kvm irqbypass mbcache jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg mei_me mei i2c_i801 shpchp ipmi_si ipmi_devintf ipmi_msghandler pinctrl_lewisburg acpi_power_meter pinctrl_intel acpi_cpufreq
sch_fq_codel ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops igb ttm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ahci drm ptp pps_core dca i2c_algo_bit megaraid_sas libahci libata i2c_core wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 10 PID: 1 Comm: systemd Kdump: loaded Tainted: P        W  OE  ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: Inspur NF5468M6/NF5468M6, BIOS 05.00.03 09/13/2021
task: ffff91a927668000 ti: ffff9169a7300000 task.ti: ffff9169a7300000
RIP: 0010:[<ffffffff9080b8fc>]  [<ffffffff9080b8fc>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff9169a7303ce0  EFLAGS: 00000246
RAX: ffff91e7f37fd840 RBX: ffff91a7ffe95b80 RCX: ffff916aae65c000
RDX: ffff9169c6f5cbb0 RSI: 0000000000000001 RDI: ffffdf2365b72ec0
RBP: ffff9169a7303d20 R08: ffff91e7f37fd800 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 000000006b3bd225 R14: ffff9169a7303c38 R15: ffff9169a7303c38
FS:  00007f2175d28940(0000) GS:ffff91a7ffe80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000007e2560 CR3: 0000007f7e25e000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff907a7892>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff907a335e>] ? lru_cache_add+0xe/0x10
[<ffffffff9080d058>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff9080d3d4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff9071c1da>] cgroup_destroy_locked+0xea/0x370
[<ffffffff9071c482>] cgroup_rmdir+0x22/0x40
[<ffffffff908295ac>] vfs_rmdir+0xdc/0x150
[<ffffffff9082cb31>] do_rmdir+0x1f1/0x220
[<ffffffff9081d1ee>] ? ____fput+0xe/0x10
[<ffffffff906b7850>] ? task_work_run+0xc0/0xe0
[<ffffffff9082dd66>] SyS_rmdir+0x16/0x20
[<ffffffff90d1f7d5>] system_call_fastpath+0x1c/0x21
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [systemd:1]
Modules linked in: sctp_diag sctp dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag xt_nat veth xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rpcsec_gss_krb5 nfsv4 beegfs(OE) rdma_cm iw_cm dns_resolver ib_cm nfs ib_core fscache sb_edac edac_core intel_powerclamp ext4 coretemp mbcache jbd2 intel_rapl iosf_mbi kvm_intel kvm iTCO_wdt iTCO_vendor_support irqbypass mxm_wmi
dcdbas crc32_pclmul ipmi_ssif ghash_clmulni_intel aesni_intel ixgbe lrw gf128mul glue_helper ablk_helper cryptd mdio dca pcspkr ipmi_si mei_me sg mei lpc_ich ipmi_devintf ipmi_msghandler wmi shpchp acpi_power_meter nfsd nfs_acl lockd grace auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom mgag200 i2c_algo_bit drm_kms_helper syscopyarea qla2xxx sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ahci libahci libata tg3 megaraid_sas i2c_core scsi_transport_fc ptp pps_core scsi_tgt dm_mirror dm_region_hash dm_log dm_mod
CPU: 24 PID: 1 Comm: systemd Tainted: G           OE  ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R730/0WCJNT, BIOS 2.4.3 01/17/2017
task: ffff8801273d8000 ti: ffff8801273cc000 task.ti: ffff8801273cc000
RIP: 0010:[<ffffffff811f1c7c>]  [<ffffffff811f1c7c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff8801273cfd10  EFLAGS: 00000246
RAX: ffff883f7258c040 RBX: ffff883f7cc11ae0 RCX: ffff880275dd1c00
RDX: ffff8801610453c0 RSI: 0000000000000001 RDI: ffffea007bd14f00
RBP: ffff8801273cfd50 R08: ffff883f7258c000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000753aa414 R14: ffff8801273cfc68 R15: ffff8801273cfc68
FS:  00007f4964a1d940(0000) GS:ffff883f7cc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5bcd925e04 CR3: 0000003f75e7c000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8801273cfd50 ffffffff811959a2 ffffffff811914ee ffffea007bd14f00
ffff880102a1d440 ffff88407ffd8540 ffff8801610453c0 ffffea007bd14f20
ffff8801273cfdd0 ffffffff811f33e8 0000000200000000 ffff880275dd1d00
Call Trace:
[<ffffffff811959a2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff811914ee>] ? lru_cache_add+0xe/0x10
[<ffffffff811f33e8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f3764>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f626>] vfs_rmdir+0xd6/0x150
[<ffffffff81212251>] do_rmdir+0x1f1/0x220
[<ffffffff810ad24c>] ? task_work_run+0xac/0xf0
[<ffffffff81213456>] SyS_rmdir+0x16/0x20
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#31 stuck for 22s! [slurmstepd:427006]
Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu dm_mirror dm_region_hash dm_log iTCO_wdt iTCO_vendor_support skx_edac edac_core intel_powerclamp mgag200 coretemp ttm intel_rapl iosf_mbi kvm_intel drm_kms_helper kvm syscopyarea sysfillrect dell_smbios sysimgblt dcdbas irqbypass fb_sys_fops sparse_keymap crc32_pclmul drm crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr mei_me mei shpchp i2c_i801 lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_pad acpi_power_meter binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data dm_bufio dm_mod udf crc_itu_t vfat fat br_netfilter bridge stp llc xfs
isci libsas qla3xxx hpsa e1000e ixgbe(OE) igb(OE) i2c_algo_bit i2c_core dca aacraid aic79xx ata_piix mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun sd_mod crc_t10dif crct10dif_generic sg crct10dif_pclmul crct10dif_common i40e(OE) ahci libahci megaraid_sas libata tg3(OE) ptp pps_core [last unloaded: ipmi_msghandler]
CPU: 31 PID: 427006 Comm: slurmstepd Tainted: G           OEL ------------   3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R440/04JN2K, BIOS 2.9.3 09/23/2020
task: ffff8831854b4f10 ti: ffff8831d11c8000 task.ti: ffff8831d11c8000
RIP: 0010:[<ffffffff811f1c8c>]  [<ffffffff811f1c8c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff8831d11cbd10  EFLAGS: 00000246
RAX: ffff885f7bc1c040 RBX: ffff885fb81d1ae0 RCX: ffff885fb7cd4c00
RDX: ffff8830f9a00150 RSI: 0000000000000001 RDI: ffffea00db600540
RBP: ffff8831d11cbd50 R08: ffff885f7bc1c000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000723601b8 R14: ffff8831d11cbc68 R15: ffff8831d11cbc68
FS:  00002aaaaaafd180(0000) GS:ffff885fb81c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffff7ff5000 CR3: 0000003082caa000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8831d11cbd50 ffffffff811959a2 ffffffff811914ee ffffea00db600540
ffff88319d227440 ffff88607bfd7540 ffff8830f9a00150 ffffea00db600560
ffff8831d11cbdd0 ffffffff811f33f8 0000000200000001 ffff885fb7cd4d00
Call Trace:
[<ffffffff811959a2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff811914ee>] ? lru_cache_add+0xe/0x10
[<ffffffff811f33f8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff816b0325>] ? do_page_fault+0x35/0x90
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [slurmstepd:391734]
Modules linked in: mmfs26(OE) mmfslinux(OE) squashfs loop overlay(T) tracedev(OE) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu dm_mirror dm_region_hash dm_log skx_edac edac_core iTCO_wdt intel_powerclamp iTCO_vendor_support coretemp dcdbas intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul crc32c_intel mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ghash_clmulni_intel drm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr mei_me mei shpchp i2c_i801 lpc_ich ipmi_si ipmi_devintf nfit ipmi_msghandler libnvdimm acpi_power_meter acpi_pad binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data dm_bufio dm_mod udf crc_itu_t vfat fat br_netfilter bridge stp llc xfs isci
libsas qla3xxx hpsa e1000e ixgbe(OE) igb(OE) i2c_algo_bit i2c_core dca aacraid aic79xx ata_piix mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun sd_mod crc_t10dif crct10dif_generic sg crct10dif_pclmul crct10dif_common i40e(OE) ahci megaraid_sas tg3(OE) libahci ptp pps_core libata [last unloaded: mmfslinux]
CPU: 14 PID: 391734 Comm: slurmstepd Tainted: G    B      OEL ------------ T 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R440/0WKGTH, BIOS 1.3.7 02/09/2018
task: ffff882f77863f40 ti: ffff88020afe0000 task.ti: ffff88020afe0000
RIP: 0010:[<ffffffff811f1c8c>]  [<ffffffff811f1c8c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff88020afe3d10  EFLAGS: 00000246
RAX: ffff885f445fbc40 RBX: ffff882f7dbd1ae0 RCX: ffff88311930dc00
RDX: ffff8830a9767d90 RSI: 0000000000000001 RDI: ffffea011bb9f640
RBP: ffff88020afe3d50 R08: ffff885f445fbc00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 000000006825271c R14: ffff88020afe3c68 R15: ffff88020afe3c68
FS:  00002aaaaaafd180(0000) GS:ffff882f7dbc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002aaaab599318 CR3: 0000003207210000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff88020afe3d50 ffffffff811959a2 ffffffff811914ee ffffea011bb9f640
ffff8831a3cc3430 ffff88603ffd7540 ffff8830a9767d90 ffffea011bb9f660
ffff88020afe3dd0 ffffffff811f33f8 0000000200000001 ffff88311930dd00
Call Trace:
[<ffffffff811959a2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff811914ee>] ? lru_cache_add+0xe/0x10
[<ffffffff811f33f8>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff810c13e6>] ? __cond_resched+0x26/0x30
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff816b0325>] ? do_page_fault+0x35/0x90
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#74 stuck for 23s! [runc:111221]
Modules linked in: xt_nat veth vxlan ip6_udp_tunnel udp_tunnel xt_mark nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc nfsd auth_rpcgss nfs_acl lockd grace nls_utf8 isofs loop ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod sunrpc ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm iTCO_wdt iTCO_vendor_support vfat fat i40iw ib_core skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel
kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper joydev ablk_helper cryptd mei_me ses mei ipmi_ssif enclosure scsi_transport_sas pcspkr sg i2c_i801 lpc_ich shpchp ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad sch_fq_codel ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm i40e drm ahci libahci megaraid_sas libata ptp pps_core i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 74 PID: 111221 Comm: runc Kdump: loaded Tainted: G               ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: Inspur NF8480M5/IIMS, BIOS 4.1.11 12/26/2019
task: ffff89d4007c3f40 ti: ffff89d3ffb9c000 task.ti: ffff89d3ffb9c000
RIP: 0010:[<ffffffffb340b8fc>]  [<ffffffffb340b8fc>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff89d3ffb9fce0  EFLAGS: 00000246
RAX: ffff897bd46ef840 RBX: ffff89dbfee95b80 RCX: ffff897bf0bf1800
RDX: ffff897e2c5079c0 RSI: 0000000000000001 RDI: ffffc8b48f21e700
RBP: ffff89d3ffb9fd20 R08: ffff897bd46ef800 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 0000000075e0014e R14: ffff89d3ffb9fc38 R15: ffff89d3ffb9fc38
FS:  00007fab09113740(0000) GS:ffff89dbfee80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005612eeb272f7 CR3: 00000172a43a4000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb33a7892>] ? isolate_lru_page+0x122/0x1c0
[<ffffffffb33a335e>] ? lru_cache_add+0xe/0x10
[<ffffffffb340d058>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffffb340d3d4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffb331c1da>] cgroup_destroy_locked+0xea/0x370
[<ffffffffb331c482>] cgroup_rmdir+0x22/0x40
[<ffffffffb34295ac>] vfs_rmdir+0xdc/0x150
[<ffffffffb342cb31>] do_rmdir+0x1f1/0x220
[<ffffffffb3420d84>] ? SYSC_newfstatat+0x24/0x60
[<ffffffffb342dd95>] SyS_unlinkat+0x25/0x40
[<ffffffffb391f7d5>] system_call_fastpath+0x1c/0x21
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [slurmstepd:85542]
Modules linked in: 8021q garp mrp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nfsv3 nfs fscache ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm rpcrdma rdma_ucm ib_uverbs dm_mirror ib_iser dm_region_hash dm_log rdma_cm opa_vnic iw_cm ib_umad dm_mod libiscsi ib_ipoib scsi_transport_iscsi ib_cm skx_edac intel_powerclamp coretemp intel_rapl iTCO_wdt iosf_mbi iTCO_vendor_support kvm irqbypass crc32_pclmul ghash_clmulni_intel hfi1 aesni_intel lrw gf128mul glue_helper ablk_helper cryptd rdmavt joydev ib_core sg pcspkr mei_me
mei ipmi_si shpchp i2c_i801 lpc_ich ipmi_devintf wmi ipmi_msghandler nfit libnvdimm acpi_cpufreq acpi_power_meter acpi_pad nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ixgbe ttm ahci crct10dif_pclmul crct10dif_common crc32c_intel mdio drm libahci ptp libata pps_core i2c_core dca
CPU: 4 PID: 85542 Comm: slurmstepd Kdump: loaded Not tainted 3.10.0-862.3.2.el7.x86_64 #1
Hardware name: Advanced Clustering S2600BPB/S2600BPB, BIOS SE5C620.86B.00.01.0013.C1.030920180427 03/09/2018
task: ffff94cc32e60fd0 ti: ffff94c7acbb0000 task.ti: ffff94c7acbb0000
RIP: 0010:[<ffffffff92c0bc3c>]  [<ffffffff92c0bc3c>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff94c7acbb3ce0  EFLAGS: 00000246
RAX: ffff949b83fa4c40 RBX: ffff949c3db15b80 RCX: ffff949b83fa0000
RDX: ffff946e1fb6ba20 RSI: 0000000000000001 RDI: ffffdec8829ae880
RBP: ffff94c7acbb3d20 R08: ffff949b83fa4c00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 00000000f077a507 R14: ffff94c7acbb3c38 R15: ffff94c7acbb3c38
FS:  00002adb525cbd40(0000) GS:ffff949c3db00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002afc0fdd3338 CR3: 000000247d7e2000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff92ba7bd2>] ? isolate_lru_page+0x122/0x1c0
[<ffffffff92ba369e>] ? lru_cache_add+0xe/0x10
[<ffffffff92c0d398>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff92c0d714>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff92b1c50a>] cgroup_destroy_locked+0xea/0x370
[<ffffffff92b1c7b2>] cgroup_rmdir+0x22/0x40
[<ffffffff92c298ec>] vfs_rmdir+0xdc/0x150
[<ffffffff92c2ce71>] do_rmdir+0x1f1/0x220
[<ffffffff92c2e0a6>] SyS_rmdir+0x16/0x20
[<ffffffff9312082f>] system_call_fastpath+0x1c/0x21
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02 

NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [docker-runc:162328]
Modules linked in: veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rpcsec_gss_krb5 beegfs(OE) rdma_cm nfsv4 iw_cm ib_cm dns_resolver ib_core nfs fscache iTCO_wdt ipmi_devintf iTCO_vendor_support dcdbas dm_service_time intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass
crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif ixgbe qla2xxx pcspkr scsi_transport_fc mdio ipmi_si sb_edac scsi_tgt dca edac_core mei_me sg ipmi_msghandler mei lpc_ich i2c_i801 shpchp wmi acpi_power_meter dm_multipath nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci ttm libahci drm libata tg3 i2c_core ptp megaraid_sas pps_core fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 10 PID: 162328 Comm: docker-runc Tainted: G    B   W  OEL ------------   3.10.0-514.26.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R930/0Y0V4F, BIOS 2.3.1 01/09/2017
task: ffff8801250f2f10 ti: ffff880be6bac000 task.ti: ffff880be6bac000
RIP: 0010:[<ffffffff811f01ac>]  [<ffffffff811f01ac>] mem_cgroup_move_account+0x3c/0x160
RSP: 0018:ffff880be6bafd10  EFLAGS: 00000246
RAX: ffff88014b71d440 RBX: ffffea002dadf300 RCX: ffff880232681800
RDX: ffff8801f4faa690 RSI: 0000000000000001 RDI: ffffea002daa9a40
RBP: ffff880be6bafd50 R08: ffff88014b71d400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000e
R13: 000000000df3c6d1 R14: ffff880be6bafc68 R15: ffff880be6bafc68
FS:  00007f9f8b95a740(0000) GS:ffff887e7c880000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000d8dff0 CR3: 0000007e5c25f000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880be6bafd50 ffffffff81194d73 ffffffffffffffff ffffea002daa9a40
ffff88274934f440 ffff88807ffd7540 ffff8801f4faa690 ffffea002daa9a60
ffff880be6bafdd0 ffffffff811f1998 0000000200000000 ffff880232681900
Call Trace:
[<ffffffff81194d73>] ? isolate_lru_page+0x153/0x1f0
[<ffffffff811f1998>] mem_cgroup_reparent_charges+0x1f8/0x3c0
[<ffffffff811f1d14>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d6f7>] cgroup_destroy_locked+0xe7/0x360
[<ffffffff8110d992>] cgroup_rmdir+0x22/0x40
[<ffffffff8120a948>] vfs_rmdir+0xa8/0x100
[<ffffffff8120f9e5>] do_rmdir+0x1a5/0x200
[<ffffffff81203ffe>] ? SYSC_newstat+0x2e/0x60
[<ffffffff81692ff5>] ? do_page_fault+0x35/0x90
[<ffffffff81210bc5>] SyS_unlinkat+0x25/0x40
[<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Code: 55 49 89 cd 41 54 49 89 fc 53 48 89 d3 48 83 ec 18 83 fe 01 4c 8b 77 08 76 0c 48 8b 07 f6 c4 40 0f 84 09 01 00 00 f0 0f ba 2b 00 <19> c0 85 c0 0f 85 05 01 00 00 48 8b 03 41 bf ea ff ff ff a8 02