Info

Permalink
Function
put_page
First occurrence
2021-02-20
Last occurrence
2025-08-30
State
NEW
Components
kernel
Quality
-19

Statistics

Operating system Count
CentOS 7 22
Architecture Count
x86_64 22
Related packages Count
kernel 22
 0:3.10.0-693.2.2.el7 9
 0:3.10.0-862.el7 6
 0:3.10.0-693.5.2.el7 4
 0:3.10.0-862.6.3.el7 2
 0:3.10.0-957.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1415770
Frame # Function Binary Source or offset Line
1
put_page
vmlinux 0xffffffff81190359
Build id: 3.10.0-693.2.2.el7.x86_64
0x29
2
mem_cgroup_reparent_charges
vmlinux 0xffffffff811f3432
Build id: 3.10.0-693.2.2.el7.x86_64
0x232
3
mem_cgroup_css_offline
vmlinux 0xffffffff811f3774
Build id: 3.10.0-693.2.2.el7.x86_64
0x84
4
cgroup_destroy_locked
vmlinux 0xffffffff8110d757
Build id: 3.10.0-693.2.2.el7.x86_64
0xe7
5
cgroup_rmdir
vmlinux 0xffffffff8110da02
Build id: 3.10.0-693.2.2.el7.x86_64
0x22
6
vfs_rmdir
vmlinux 0xffffffff8120f636
Build id: 3.10.0-693.2.2.el7.x86_64
0xd6
7
do_rmdir
vmlinux 0xffffffff812122d1
Build id: 3.10.0-693.2.2.el7.x86_64
0x1f1
8
? do_page_fault
vmlinux 0xffffffff816b0325
Build id: 3.10.0-693.2.2.el7.x86_64
0x35
9
SyS_rmdir
vmlinux 0xffffffff812134d6
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
10
system_call_fastpath
vmlinux 0xffffffff816b5009
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#48 stuck for 22s! [slurmstepd:84268]
Modules linked in: squashfs loop overlay(T) mmfs26(OE) mmfslinux(OE) tracedev(OE) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver dm_mirror dm_region_hash dm_log joydev iTCO_wdt iTCO_vendor_support skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul crc32c_intel mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ghash_clmulni_intel drm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sr_mod cdrom mei_me mei shpchp i2c_i801 lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data dm_bufio
dm_mod udf crc_itu_t vfat fat br_netfilter bridge stp llc xfs isci libsas qla3xxx hpsa e1000e ixgbe(OE) megaraid_sas aacraid aic79xx ata_piix mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil tg3(OE) e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun sd_mod crc_t10dif crct10dif_generic sg crct10dif_pclmul crct10dif_common ahci libahci igb(OE) i2c_algo_bit libata i2c_core dca i40e(OE) ptp pps_core [last unloaded: ipmi_msghandler]
CPU: 48 PID: 84268 Comm: slurmstepd Tainted: G        W  OE  ------------ T 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: FUJITSU PRIMERGY RX2540 M5/D3384-B1, BIOS V5.0.0.14 R1.15.0 for D3384-B1x                    10/17/2019
task: ffff88307508eeb0 ti: ffff883a18694000 task.ti: ffff883a18694000
RIP: 0010:[<ffffffff81190359>]  [<ffffffff81190359>] put_page+0x29/0x50
RSP: 0018:ffff883a18697d50  EFLAGS: 00000202
RAX: 00efffff0000004c RBX: ffffea015b2ac000 RCX: 0000000000000001
RDX: 000000000000000b RSI: 0000000000000001 RDI: ffffea013b720500
RBP: ffff883a18697d50 R08: ffff885fdc297c00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffff81190590
R13: 0000000000000000 R14: ffffffff81190eac R15: ffff883a18697cf8
FS:  00002aaaaaafd180(0000) GS:ffff882fdf380000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002ab03ccef000 CR3: 0000001847b2e000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff883a18697dd0 ffffffff811f3432 0000000200000003 ffff885c77765d00
0000000000000003 0000000000001000 ffff885c77765c40 0000000000000001
0000000100000003 ffff885c77765c00 0000000000000000 0000000000000000
Call Trace:
[<ffffffff811f3432>] mem_cgroup_reparent_charges+0x232/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff816b0325>] ? do_page_fault+0x35/0x90
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: 00 00 0f 1f 44 00 00 48 8b 07 55 48 89 e5 a9 00 00 00 80 74 0d e8 18 0c ff ff 5d c3 66 0f 1f 44 00 00 f6 c4 c0 75 14 f0 ff 4f 1c <0f> 94 c0 84 c0 66 90 74 e6 e8 99 ff ff ff 5d c3 0f 1f 80 00 00 
Complete report #1148785
Frame # Function Binary Source or offset Line
1
put_page
vmlinux 0xffffffff81190359
Build id: 3.10.0-693.2.2.el7.x86_64
0x29
2
mem_cgroup_reparent_charges
vmlinux 0xffffffff811f3432
Build id: 3.10.0-693.2.2.el7.x86_64
0x232
3
mem_cgroup_css_offline
vmlinux 0xffffffff811f3774
Build id: 3.10.0-693.2.2.el7.x86_64
0x84
4
cgroup_destroy_locked
vmlinux 0xffffffff8110d757
Build id: 3.10.0-693.2.2.el7.x86_64
0xe7
5
cgroup_rmdir
vmlinux 0xffffffff8110da02
Build id: 3.10.0-693.2.2.el7.x86_64
0x22
6
vfs_rmdir
vmlinux 0xffffffff8120f636
Build id: 3.10.0-693.2.2.el7.x86_64
0xd6
7
do_rmdir
vmlinux 0xffffffff812122d1
Build id: 3.10.0-693.2.2.el7.x86_64
0x1f1
8
SyS_rmdir
vmlinux 0xffffffff812134d6
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
9
system_call_fastpath
vmlinux 0xffffffff816b5009
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#63 stuck for 23s! [slurmstepd:392445]
Modules linked in: squashfs loop overlay(T) mmfs26(OE) mmfslinux(OE) tracedev(OE) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver dell_rbu dm_mirror dm_region_hash dm_log skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi iTCO_wdt iTCO_vendor_support kvm_intel kvm irqbypass crc32_pclmul dell_smbios crc32c_intel dcdbas sparse_keymap ghash_clmulni_intel mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops aesni_intel lrw gf128mul drm glue_helper ablk_helper cryptd pcspkr mei_me mei shpchp i2c_i801 lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data
dm_bufio dm_mod udf crc_itu_t vfat fat br_netfilter bridge stp llc xfs isci libsas qla3xxx hpsa sg e1000e ixgbe(OE) igb(OE) i2c_algo_bit i2c_core dca aacraid aic79xx ata_piix sd_mod crc_t10dif mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun crct10dif_pclmul crct10dif_common ahci libahci libata megaraid_sas i40e(OE) tg3(OE) ptp pps_core [last unloaded: ipmi_msghandler]
CPU: 63 PID: 392445 Comm: slurmstepd Tainted: G           OEL ------------ T 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R440/0WKGTH, BIOS 2.21.1 03/07/2024
task: ffff884b0d060fd0 ti: ffff8801b1a00000 task.ti: ffff8801b1a00000
RIP: 0010:[<ffffffff81190359>]  [<ffffffff81190359>] put_page+0x29/0x50
RSP: 0018:ffff8801b1a03d50  EFLAGS: 00000202
RAX: 006fffff0000006c RBX: ffffea00dd311300 RCX: ffff8801b1a03c58
RDX: 0000000000000000 RSI: 000000000000000e RDI: ffffea00dd311300
RBP: ffff8801b1a03d50 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffff81190590
R13: 0000000000000000 R14: ffffffff81190eac R15: ffff8801b1a03cf8
FS:  00002aaaaaafd180(0000) GS:ffff885f7c5c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000069c3260 CR3: 0000000bc1d8e000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8801b1a03dd0 ffffffff811f3432 0000000200000001 ffff880f1f945900
0000000000000001 0000000000001000 ffff880f1f945840 0000000000000001
0000000100000003 ffff880f1f945800 0000000000000000 0000000000000000
Call Trace:
[<ffffffff811f3432>] mem_cgroup_reparent_charges+0x232/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: 00 00 0f 1f 44 00 00 48 8b 07 55 48 89 e5 a9 00 00 00 80 74 0d e8 18 0c ff ff 5d c3 66 0f 1f 44 00 00 f6 c4 c0 75 14 f0 ff 4f 1c <0f> 94 c0 84 c0 66 90 74 e6 e8 99 ff ff ff 5d c3 0f 1f 80 00 00 
Complete report #1063040
Frame # Function Binary Source or offset Line
1
put_page
vmlinux 0xffffffff81190389
Build id: 3.10.0-693.5.2.el7.x86_64
0x29
2
mem_cgroup_reparent_charges
vmlinux 0xffffffff811f33a2
Build id: 3.10.0-693.5.2.el7.x86_64
0x232
3
mem_cgroup_css_offline
vmlinux 0xffffffff811f36e4
Build id: 3.10.0-693.5.2.el7.x86_64
0x84
4
cgroup_destroy_locked
vmlinux 0xffffffff8110d767
Build id: 3.10.0-693.5.2.el7.x86_64
0xe7
5
cgroup_rmdir
vmlinux 0xffffffff8110da12
Build id: 3.10.0-693.5.2.el7.x86_64
0x22
6
vfs_rmdir
vmlinux 0xffffffff8120f5a6
Build id: 3.10.0-693.5.2.el7.x86_64
0xd6
7
do_rmdir
vmlinux 0xffffffff81212241
Build id: 3.10.0-693.5.2.el7.x86_64
0x1f1
8
SyS_rmdir
vmlinux 0xffffffff81213446
Build id: 3.10.0-693.5.2.el7.x86_64
0x16
9
system_call_fastpath
vmlinux 0xffffffff816b5089
Build id: 3.10.0-693.5.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [slurmstepd:119171]
Modules linked in: squashfs loop overlay(T) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache panfs(POE) 8021q garp mrp stp llc nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) nvidia(POE) mlx5_fpga_tools(OE) ip6_tables iptable_filter dell_smbios sparse_keymap dcdbas kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr shpchp i2c_piix4 ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter tpm_crb mlx5_ib(OE) mlx4_ib(OE) ib_uverbs(OE) mlx4_en(OE) mlx5_core(OE) nfsd mlxfw(OE) vfio_mdev(OE) binfmt_misc vfio_iommu_type1 vfio auth_rpcgss mdev(OE) nfs_acl ptp lockd pps_core grace mlx4_core(OE) sunrpc devlink knem(OE) ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200
i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci crct10dif_pclmul drm crct10dif_common libahci crc32c_intel libata bnxt_en i2c_core megaraid_sas wmi ib_ipoib(OE) ib_cm(OE) ib_umad(OE) ib_core(OE) mlx_compat(OE)
CPU: 0 PID: 119171 Comm: slurmstepd Tainted: P        W  OE  ------------ T 3.10.0-693.5.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R7525/0H3K7P, BIOS 2.8.4 06/23/2022
task: ffff893590ad8000 ti: ffff897679e24000 task.ti: ffff897679e24000
RIP: 0010:[<ffffffff81190389>]  [<ffffffff81190389>] put_page+0x29/0x50
RSP: 0018:ffff897679e27d50  EFLAGS: 00000202
RAX: 006fffff0000000c RBX: ffffea051fc8dd80 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffea051fc8de00
RBP: ffff897679e27d50 R08: ffff88763ff98c00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffff811905c0
R13: 0000000000000000 R14: ffffffff81190edc R15: ffff897679e27cf8
FS:  00002af80d423300(0000) GS:ffff887e7e200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000425bd0 CR3: 0000017dbae5c000 CR4: 00000000003406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff897679e27dd0 ffffffff811f33a2 0000000200000001 ffff887dff7bc900
0000000000000001 0000000000001000 ffff887dff7bc840 0000000000000001
0000000100000002 ffff887dff7bc800 0000000000000000 0000000000000000
Call Trace:
[<ffffffff811f33a2>] mem_cgroup_reparent_charges+0x232/0x3c0
[<ffffffff811f36e4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d767>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da12>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f5a6>] vfs_rmdir+0xd6/0x150
[<ffffffff81212241>] do_rmdir+0x1f1/0x220
[<ffffffff81213446>] SyS_rmdir+0x16/0x20
[<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
Code: 00 00 66 66 66 66 90 48 8b 07 55 48 89 e5 a9 00 00 00 80 74 0d e8 f8 0b ff ff 5d c3 66 0f 1f 44 00 00 f6 c4 c0 75 14 f0 ff 4f 1c <0f> 94 c0 84 c0 66 90 74 e6 e8 99 ff ff ff 5d c3 0f 1f 80 00 00 
Complete report #961754
Frame # Function Binary Source or offset Line
1
put_page
vmlinux 0xffffffffa1ba2989
Build id: 3.10.0-862.6.3.el7.x86_64
0x29
2
mem_cgroup_reparent_charges
vmlinux 0xffffffffa1c0d852
Build id: 3.10.0-862.6.3.el7.x86_64
0x232
3
mem_cgroup_css_offline
vmlinux 0xffffffffa1c0db94
Build id: 3.10.0-862.6.3.el7.x86_64
0x84
4
cgroup_destroy_locked
vmlinux 0xffffffffa1b1c9ca
Build id: 3.10.0-862.6.3.el7.x86_64
0xea
5
cgroup_rmdir
vmlinux 0xffffffffa1b1cc72
Build id: 3.10.0-862.6.3.el7.x86_64
0x22
6
vfs_rmdir
vmlinux 0xffffffffa1c29d6c
Build id: 3.10.0-862.6.3.el7.x86_64
0xdc
7
do_rmdir
vmlinux 0xffffffffa1c2d2f1
Build id: 3.10.0-862.6.3.el7.x86_64
0x1f1
8
? ____fput
vmlinux 0xffffffffa1c1d9ae
Build id: 3.10.0-862.6.3.el7.x86_64
0xe
9
? task_work_run
vmlinux 0xffffffffa1ab8040
Build id: 3.10.0-862.6.3.el7.x86_64
0xc0
10
SyS_rmdir
vmlinux 0xffffffffa1c2e526
Build id: 3.10.0-862.6.3.el7.x86_64
0x16
11
system_call_fastpath
vmlinux 0xffffffffa2120795
Build id: 3.10.0-862.6.3.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [systemd:1]
Modules linked in: fuse binfmt_misc uas usb_storage 8021q garp mrp vhost_net vhost macvtap macvlan xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bridge stp llc bonding ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ucm i40iw rpcrdma sunrpc rdma_ucm ib_uverbs ib_iser rdma_cm iw_cm libiscsi ib_umad ib_ipoib scsi_transport_iscsi ib_cm iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd mlx4_ib mlx5_ib ib_core
pcspkr sg mei_me joydev i2c_i801 mei lpc_ich shpchp ipmi_si ipmi_devintf ipmi_msghandler wmi nfit acpi_cpufreq libnvdimm acpi_power_meter acpi_pad ip_tables xfs libcrc32c mlx4_en sd_mod crc_t10dif crct10dif_generic ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt mlx5_core fb_sys_fops ttm mlx4_core crct10dif_pclmul i40e crct10dif_common ahci drm crc32c_intel libahci mlxfw megaraid_sas libata devlink ptp pps_core i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 1 Comm: systemd Kdump: loaded Not tainted 3.10.0-862.6.3.el7.x86_64 #1
Hardware name: Supermicro X11DPi-N(T)/X11DPi-N, BIOS 2.0b 02/28/2018
task: ffff975027668000 ti: ffff9710e76dc000 task.ti: ffff9710e76dc000
RIP: 0010:[<ffffffffa1ba2989>]  [<ffffffffa1ba2989>] put_page+0x29/0x50
RSP: 0018:ffff9710e76dfd20  EFLAGS: 00000206
RAX: 006fffff00000848 RBX: fffffe3cc11aee80 RCX: 0000000000000001
RDX: 0000000000000008 RSI: 0000000000000001 RDI: fffffe3cc11a90c0
RBP: ffff9710e76dfd20 R08: ffff978e5c76d400 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffffa1ba2bc0
R13: 0000000000000000 R14: ffffffffa1ba34dc R15: ffff9710e76dfcc8
FS:  00007fc75db69940(0000) GS:ffff974eff6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055a8fd1071d8 CR3: 0000003f35ca8000 CR4: 00000000007627e0
PKRU: 55555554
Call Trace:
[<ffffffffa1c0d852>] mem_cgroup_reparent_charges+0x232/0x3c0
[<ffffffffa1c0db94>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffa1b1c9ca>] cgroup_destroy_locked+0xea/0x370
[<ffffffffa1b1cc72>] cgroup_rmdir+0x22/0x40
[<ffffffffa1c29d6c>] vfs_rmdir+0xdc/0x150
[<ffffffffa1c2d2f1>] do_rmdir+0x1f1/0x220
[<ffffffffa1c1d9ae>] ? ____fput+0xe/0x10
[<ffffffffa1ab8040>] ? task_work_run+0xc0/0xe0
[<ffffffffa1c2e526>] SyS_rmdir+0x16/0x20
[<ffffffffa2120795>] system_call_fastpath+0x1c/0x21
Code: 00 00 0f 1f 44 00 00 48 8b 07 55 48 89 e5 a9 00 00 00 80 74 0d e8 58 ff fe ff 5d c3 66 0f 1f 44 00 00 f6 c4 c0 75 14 f0 ff 4f 1c <0f> 94 c0 84 c0 66 90 74 e6 e8 99 ff ff ff 5d c3 0f 1f 80 00 00 
Complete report #686057
Frame # Function Binary Source or offset Line
1
put_page
vmlinux 0xffffffffa2da21c9
Build id: 3.10.0-862.el7.x86_64
0x29
2
mem_cgroup_reparent_charges
vmlinux 0xffffffffa2e0d092
Build id: 3.10.0-862.el7.x86_64
0x232
3
mem_cgroup_css_offline
vmlinux 0xffffffffa2e0d3d4
Build id: 3.10.0-862.el7.x86_64
0x84
4
cgroup_destroy_locked
vmlinux 0xffffffffa2d1c1da
Build id: 3.10.0-862.el7.x86_64
0xea
5
cgroup_rmdir
vmlinux 0xffffffffa2d1c482
Build id: 3.10.0-862.el7.x86_64
0x22
6
vfs_rmdir
vmlinux 0xffffffffa2e295ac
Build id: 3.10.0-862.el7.x86_64
0xdc
7
do_rmdir
vmlinux 0xffffffffa2e2cb31
Build id: 3.10.0-862.el7.x86_64
0x1f1
8
SyS_rmdir
vmlinux 0xffffffffa2e2dd66
Build id: 3.10.0-862.el7.x86_64
0x16
9
system_call_fastpath
vmlinux 0xffffffffa331f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#39 stuck for 23s! [slurmstepd:273709]
Modules linked in: binfmt_misc nvidia_uvm(OE) nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi ext4 kvm_intel kvm mbcache jbd2 irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq ipmi_ssif joydev snd_seq_device pcspkr snd_pcm sg snd_timer snd mei_me mei lpc_ich i2c_i801 soundcore shpchp ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_cpufreq acpi_pad acpi_power_meter ip_tables xfs libcrc32c nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) sd_mod crc_t10dif crct10dif_generic ast i2c_algo_bit drm_kms_helper crct10dif_pclmul ttm crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ahci megaraid_sas drm ixgbe libahci libata mdio ptp i2c_core pps_core dca wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 39 PID: 273709 Comm: slurmstepd Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: Supermicro SYS-7049GP-TRT/X11DPG-QT, BIOS 3.1 05/23/2019
task: ffff8e3276a5eeb0 ti: ffff8e2951388000 task.ti: ffff8e2951388000
RIP: 0010:[<ffffffffa2da21c9>]  [<ffffffffa2da21c9>] put_page+0x29/0x50
RSP: 0018:ffff8e295138bd20  EFLAGS: 00000202
RAX: 006fffff0002000c RBX: ffffd80ca555c040 RCX: 0000000000000001
RDX: 000000000000000a RSI: 0000000000000001 RDI: ffffd80cc7c8c480
RBP: ffff8e295138bd20 R08: ffff8e3abe3e6c00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffffa2da2400
R13: 0000000000000000 R14: ffffffffa2da2d1c R15: ffff8e295138bcc8
FS:  00002b3c18355580(0000) GS:ffff8e783c2c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6a51465000 CR3: 00000070045c4000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa2e0d092>] mem_cgroup_reparent_charges+0x232/0x3c0
[<ffffffffa2e0d3d4>] mem_cgroup_css_offline+0x84/0x140
[<ffffffffa2d1c1da>] cgroup_destroy_locked+0xea/0x370
[<ffffffffa2d1c482>] cgroup_rmdir+0x22/0x40
[<ffffffffa2e295ac>] vfs_rmdir+0xdc/0x150
[<ffffffffa2e2cb31>] do_rmdir+0x1f1/0x220
[<ffffffffa2e2dd66>] SyS_rmdir+0x16/0x20
[<ffffffffa331f7d5>] system_call_fastpath+0x1c/0x21
Code: 00 00 0f 1f 44 00 00 48 8b 07 55 48 89 e5 a9 00 00 00 80 74 0d e8 58 ff fe ff 5d c3 66 0f 1f 44 00 00 f6 c4 c0 75 14 f0 ff 4f 1c <0f> 94 c0 84 c0 66 90 74 e6 e8 99 ff ff ff 5d c3 0f 1f 80 00 00 
Complete report #162385
Frame # Function Binary Source or offset Line
1
put_page
vmlinux 0xffffffff81190359
Build id: 3.10.0-693.2.2.el7.x86_64
0x29
2
mem_cgroup_reparent_charges
vmlinux 0xffffffff811f3432
Build id: 3.10.0-693.2.2.el7.x86_64
0x232
3
mem_cgroup_css_offline
vmlinux 0xffffffff811f3774
Build id: 3.10.0-693.2.2.el7.x86_64
0x84
4
cgroup_destroy_locked
vmlinux 0xffffffff8110d757
Build id: 3.10.0-693.2.2.el7.x86_64
0xe7
5
cgroup_rmdir
vmlinux 0xffffffff8110da02
Build id: 3.10.0-693.2.2.el7.x86_64
0x22
6
vfs_rmdir
vmlinux 0xffffffff8120f636
Build id: 3.10.0-693.2.2.el7.x86_64
0xd6
7
do_rmdir
vmlinux 0xffffffff812122d1
Build id: 3.10.0-693.2.2.el7.x86_64
0x1f1
8
? do_page_fault
vmlinux 0xffffffff816b0325
Build id: 3.10.0-693.2.2.el7.x86_64
0x35
9
SyS_rmdir
vmlinux 0xffffffff812134d6
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
10
system_call_fastpath
vmlinux 0xffffffff816b5009
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#17 stuck for 22s! [slurmstepd:204817]
Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) squashfs loop overlay(T) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu dm_mirror dm_region_hash dm_log joydev dcdbas skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul crc32c_intel iTCO_wdt iTCO_vendor_support ghash_clmulni_intel mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr mei_me mei i2c_i801 shpchp lpc_ich ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data dm_bufio dm_mod udf crc_itu_t vfat fat br_netfilter bridge stp llc
xfs isci libsas qla3xxx hpsa e1000e ixgbe(OE) igb(OE) i2c_algo_bit i2c_core dca aacraid aic79xx ata_piix mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun sd_mod crc_t10dif crct10dif_generic sg crct10dif_pclmul crct10dif_common i40e(OE) megaraid_sas ahci libahci tg3(OE) libata ptp pps_core [last unloaded: tracedev]
CPU: 17 PID: 204817 Comm: slurmstepd Tainted: G    B      OE  ------------ T 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R440/0WKGTH, BIOS 1.3.7 02/09/2018
task: ffff884d85208fd0 ti: ffff8801ef720000 task.ti: ffff8801ef720000
RIP: 0010:[<ffffffff81190359>]  [<ffffffff81190359>] put_page+0x29/0x50
RSP: 0000:ffff8801ef723d50  EFLAGS: 00000202
RAX: 006fffff0000000c RBX: ffffea014f41bf80 RCX: 0000000000000001
RDX: 0000000000000009 RSI: 0000000000000001 RDI: ffffea00c9f5e780
RBP: ffff8801ef723d50 R08: ffff885ed173dc00 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffff81190590
R13: 0000000000000000 R14: ffffffff81190eac R15: ffff8801ef723cf8
FS:  00002aaaaaafd180(0000) GS:ffff885f7c000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002aaabe75a000 CR3: 00000001dddb7000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8801ef723dd0 ffffffff811f3432 0000000200000001 ffff8857f6fcd900
0000000000000001 0000000000001000 ffff8857f6fcd840 0000000000000001
0000000100000002 ffff8857f6fcd800 0000000000000000 0000000000000000
Call Trace:
[<ffffffff811f3432>] mem_cgroup_reparent_charges+0x232/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff816b0325>] ? do_page_fault+0x35/0x90
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: 00 00 0f 1f 44 00 00 48 8b 07 55 48 89 e5 a9 00 00 00 80 74 0d e8 18 0c ff ff 5d c3 66 0f 1f 44 00 00 f6 c4 c0 75 14 f0 ff 4f 1c <0f> 94 c0 84 c0 66 90 74 e6 e8 99 ff ff ff 5d c3 0f 1f 80 00 00 
Complete report #73712
Frame # Function Binary Source or offset Line
1
put_page
vmlinux 0xffffffff81190350
Build id: 3.10.0-693.2.2.el7.x86_64
0x20
2
mem_cgroup_reparent_charges
vmlinux 0xffffffff811f3432
Build id: 3.10.0-693.2.2.el7.x86_64
0x232
3
mem_cgroup_css_offline
vmlinux 0xffffffff811f3774
Build id: 3.10.0-693.2.2.el7.x86_64
0x84
4
cgroup_destroy_locked
vmlinux 0xffffffff8110d757
Build id: 3.10.0-693.2.2.el7.x86_64
0xe7
5
cgroup_rmdir
vmlinux 0xffffffff8110da02
Build id: 3.10.0-693.2.2.el7.x86_64
0x22
6
vfs_rmdir
vmlinux 0xffffffff8120f636
Build id: 3.10.0-693.2.2.el7.x86_64
0xd6
7
do_rmdir
vmlinux 0xffffffff812122d1
Build id: 3.10.0-693.2.2.el7.x86_64
0x1f1
8
? do_page_fault
vmlinux 0xffffffff816b0325
Build id: 3.10.0-693.2.2.el7.x86_64
0x35
9
SyS_rmdir
vmlinux 0xffffffff812134d6
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
10
system_call_fastpath
vmlinux 0xffffffff816b5009
Build id: 3.10.0-693.2.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#39 stuck for 22s! [slurmstepd:60168]
Modules linked in: squashfs loop overlay(T) mmfs26(OE) mmfslinux(OE) tracedev(OE) devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dell_rbu dm_mirror dm_region_hash dm_log dcdbas skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi iTCO_wdt iTCO_vendor_support kvm_intel kvm irqbypass crc32_pclmul crc32c_intel ghash_clmulni_intel mgag200 ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr mei_me mei shpchp i2c_i801 lpc_ich ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad binfmt_misc ip_tables bnxt_en nvme nvme_core qed hpilo dm_thin_pool dm_bio_prison dm_persistent_data dm_bufio dm_mod udf crc_itu_t vfat fat br_netfilter bridge stp llc xfs isci
libsas qla3xxx hpsa e1000e ixgbe(OE) igb(OE) i2c_algo_bit i2c_core dca aacraid aic79xx ata_piix mpt2sas raid_class mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas arcmsr sata_nv sata_svw bnx2x(OE) mdio libcrc32c bnx2(OE) ext4 mbcache jbd2 sata_sil e1000 nfsv3 nfs_acl nfs lockd grace sunrpc fscache tun sd_mod crc_t10dif crct10dif_generic sg crct10dif_pclmul crct10dif_common i40e(OE) ahci libahci megaraid_sas libata tg3(OE) ptp pps_core [last unloaded: ipmi_msghandler]
CPU: 39 PID: 60168 Comm: slurmstepd Tainted: G           OE  ------------ T 3.10.0-693.2.2.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R440/0WKGTH, BIOS 1.3.7 02/09/2018
task: ffff88316a61bf40 ti: ffff88310f804000 task.ti: ffff88310f804000
RIP: 0010:[<ffffffff81190350>]  [<ffffffff81190350>] put_page+0x20/0x50
RSP: 0018:ffff88310f807d50  EFLAGS: 00000246
RAX: 006fffff00000068 RBX: ffffea0161ef8440 RCX: ffff88310f807c58
RDX: 0000000000000000 RSI: 000000000000000e RDI: ffffea0161ef8440
RBP: ffff88310f807d50 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffff81190590
R13: 0000000000000000 R14: ffffffff81190eac R15: ffff88310f807cf8
FS:  00002aaaaaafd180(0000) GS:ffff885f7c2c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffff7ff5000 CR3: 000000012fb12000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff88310f807dd0 ffffffff811f3432 0000000200000001 ffff884a01437500
0000000000000001 0000000000001000 ffff884a01437440 0000000000000001
0000000100000003 ffff884a01437400 0000000000000000 0000000000000000
Call Trace:
[<ffffffff811f3432>] mem_cgroup_reparent_charges+0x232/0x3c0
[<ffffffff811f3774>] mem_cgroup_css_offline+0x84/0x140
[<ffffffff8110d757>] cgroup_destroy_locked+0xe7/0x370
[<ffffffff8110da02>] cgroup_rmdir+0x22/0x40
[<ffffffff8120f636>] vfs_rmdir+0xd6/0x150
[<ffffffff812122d1>] do_rmdir+0x1f1/0x220
[<ffffffff816b0325>] ? do_page_fault+0x35/0x90
[<ffffffff812134d6>] SyS_rmdir+0x16/0x20
[<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
Code: c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 07 55 48 89 e5 a9 00 00 00 80 74 0d e8 18 0c ff ff 5d c3 66 0f 1f 44 00 00 <f6> c4 c0 75 14 f0 ff 4f 1c 0f 94 c0 84 c0 66 90 74 e6 e8 99 ff