Complete report #1421285
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a787c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
2
|
|
vmlinux |
0xffffffff810c0548
Build id: 3.10.0-693.el7.x86_64
|
0xa8
|
|
3
|
compaction_alloc
|
vmlinux |
0xffffffff811a7ccd
Build id: 3.10.0-693.el7.x86_64
|
0x22d
|
|
4
|
migrate_pages
|
vmlinux |
0xffffffff811e6c51
Build id: 3.10.0-693.el7.x86_64
|
0x221
|
|
5
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a7aa0
Build id: 3.10.0-693.el7.x86_64
|
0x2c0
|
|
6
|
compact_zone
|
vmlinux |
0xffffffff811a8b39
Build id: 3.10.0-693.el7.x86_64
|
0x2d9
|
|
7
|
compact_zone_order
|
vmlinux |
0xffffffff811a8d2c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
8
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a90e1
Build id: 3.10.0-693.el7.x86_64
|
0x121
|
|
9
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8169f5cd
Build id: 3.10.0-693.el7.x86_64
|
0xac
|
|
10
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff8169fa9d
Build id: 3.10.0-693.el7.x86_64
|
0x3e9
|
|
11
|
|
vmlinux |
0xffffffff8121ad00
Build id: 3.10.0-693.el7.x86_64
|
0x100
|
|
12
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118cd85
Build id: 3.10.0-693.el7.x86_64
|
0x405
|
|
13
|
copy_process
|
vmlinux |
0xffffffff8108510d
Build id: 3.10.0-693.el7.x86_64
|
0x1dd
|
|
14
|
do_fork
|
vmlinux |
0xffffffff81086a51
Build id: 3.10.0-693.el7.x86_64
|
0x91
|
|
15
|
|
vmlinux |
0xffffffff81220195
Build id: 3.10.0-693.el7.x86_64
|
0x25
|
|
16
|
SyS_clone
|
vmlinux |
0xffffffff81086d66
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
|
17
|
stub_clone
|
vmlinux |
0xffffffff816b5319
Build id: 3.10.0-693.el7.x86_64
|
0x69
|
|
18
|
|
vmlinux |
0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#23 stuck for 23s! [systemd:1]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter arc4 md4 nls_utf8 cifs dns_resolver vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptsas drm scsi_transport_sas
ata_piix mptscsih crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 23 PID: 1 Comm: systemd Tainted: G L ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88016db70000 ti: ffff88016db78000 task.ti: ffff88016db78000
RIP: 0010:[<ffffffff811a787c>] [<ffffffff811a787c>] isolate_freepages_block+0x9c/0x2c0
RSP: 0018:ffff88016db7b948 EFLAGS: 00010206
RAX: 00000000ffffffff RBX: 0000000000000000 RCX: ffff88016db7bb40
RDX: 0000000000996000 RSI: ffff88016db7bb98 RDI: ffff88016db7bb40
RBP: ffff88016db7b9d8 R08: 0000000000000000 R09: ffff880c3ffd8000
R10: 0000000000100c00 R11: 0000000000c40000 R12: 0000000000000000
R13: ffff880c3ffd8000 R14: 0000000000100c00 R15: 0000000000c40000
FS: 00007faa83848940(0000) GS:ffff880c0f5c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcb96812000 CR3: 0000000c0937e000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff810c0548 ffff880c0f456cc0 ffff88016db7ba30 ffff88016db7b978
ffff88016db7bb40 ffff88016db7bb80 ffff88016db7bb98 ffff88016db7bb40
ffff880c3ffd8000 0000000000000000 0000000000996000 0000000000000001
Call Trace:
[<ffffffff810c0548>] ? resched_curr+0xa8/0xc0
[<ffffffff811a7ccd>] compaction_alloc+0x22d/0x2a0
[<ffffffff811e6c51>] migrate_pages+0x221/0x7a0
[<ffffffff811a7aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a8b39>] compact_zone+0x2d9/0x430
[<ffffffff811a8d2c>] compact_zone_order+0x9c/0xf0
[<ffffffff811a90e1>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8169f5cd>] __alloc_pages_direct_compact+0xac/0x193
[<ffffffff8169fa9d>] __alloc_pages_slowpath+0x3e9/0x724
[<ffffffff8121ad00>] ? __d_alloc+0x100/0x180
[<ffffffff8118cd85>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff8108510d>] copy_process+0x1dd/0x1970
[<ffffffff81086a51>] do_fork+0x91/0x320
[<ffffffff81220195>] ? __fd_install+0x25/0x60
[<ffffffff81086d66>] SyS_clone+0x16/0x20
[<ffffffff816b5319>] stub_clone+0x69/0x90
[<ffffffff816b4fc9>] ? system_call_fastpath+0x16/0x1b
Code: 45 89 c5 48 89 75 a0 48 89 4d 90 83 f8 80 74 2e 45 84 ed 0f 85 16 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 04 01 00 00 <8b> 43 18 41 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b
Complete report #838340
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a787f
Build id: 3.10.0-693.el7.x86_64
|
0x9f
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a7ccd
Build id: 3.10.0-693.el7.x86_64
|
0x22d
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e6c51
Build id: 3.10.0-693.el7.x86_64
|
0x221
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a7aa0
Build id: 3.10.0-693.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a8b39
Build id: 3.10.0-693.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a8d2c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a90e1
Build id: 3.10.0-693.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8169f5cd
Build id: 3.10.0-693.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff8169fa9d
Build id: 3.10.0-693.el7.x86_64
|
0x3e9
|
|
10
|
|
vmlinux |
0xffffffff8121ad00
Build id: 3.10.0-693.el7.x86_64
|
0x100
|
|
11
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118cd85
Build id: 3.10.0-693.el7.x86_64
|
0x405
|
|
12
|
copy_process
|
vmlinux |
0xffffffff8108510d
Build id: 3.10.0-693.el7.x86_64
|
0x1dd
|
|
13
|
do_fork
|
vmlinux |
0xffffffff81086a51
Build id: 3.10.0-693.el7.x86_64
|
0x91
|
|
14
|
|
vmlinux |
0xffffffff81220195
Build id: 3.10.0-693.el7.x86_64
|
0x25
|
|
15
|
SyS_clone
|
vmlinux |
0xffffffff81086d66
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
|
16
|
stub_clone
|
vmlinux |
0xffffffff816b5319
Build id: 3.10.0-693.el7.x86_64
|
0x69
|
|
17
|
|
vmlinux |
0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#73 stuck for 23s! [systemd:1]
Modules linked in: binfmt_misc xt_nat veth sctp_diag sctp dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag vfat fat fuse ses enclosure scsi_transport_sas uas usb_storage nfsv3 nfs fscache xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink tun xt_addrtype br_netfilter overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support dell_smbios dcdbas sparse_keymap
skx_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif pcspkr sg mei_me mei shpchp i2c_i801 lpc_ich wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_pad acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci tg3 megaraid_sas ixgbe(OE) libata dca ptp i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 73 PID: 1 Comm: systemd Tainted: G OEL ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: Dell Inc. PowerEdge R840/0TF0V7, BIOS 2.4.7 10/27/2019
task: ffff89206efa8000 ti: ffff88012eee0000 task.ti: ffff88012eee0000
RIP: 0010:[<ffffffff811a787f>] [<ffffffff811a787f>] isolate_freepages_block+0x9f/0x2c0
RSP: 0018:ffff88012eee3948 EFLAGS: 00000216
RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff88012eee3b40
RDX: 000000000d411e00 RSI: ffff88012eee3b98 RDI: ffff88012eee3b40
RBP: ffff88012eee39d8 R08: 0000000000000000 R09: ffff89203ffda000
R10: 000000001122a000 R11: 0000000012040000 R12: 0000000000000003
R13: 0000000000000001 R14: ffff89203ffae120 R15: ffff89203ffda000
FS: 00007f44ff5dc940(0000) GS:ffff88bebe680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005640f8a16ff0 CR3: 0000005eb76f6000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000012040000 ffff89203ffae020 ffff88012eee3a30 ffff89203ffae020
ffff88012eee3b40 ffff88012eee3b80 ffff88012eee3b98 ffff88012eee3b40
ffff89203ffda000 0000000000000000 000000000d411e00 0000000000000001
Call Trace:
[<ffffffff811a7ccd>] compaction_alloc+0x22d/0x2a0
[<ffffffff811e6c51>] migrate_pages+0x221/0x7a0
[<ffffffff811a7aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a8b39>] compact_zone+0x2d9/0x430
[<ffffffff811a8d2c>] compact_zone_order+0x9c/0xf0
[<ffffffff811a90e1>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8169f5cd>] __alloc_pages_direct_compact+0xac/0x193
[<ffffffff8169fa9d>] __alloc_pages_slowpath+0x3e9/0x724
[<ffffffff8121ad00>] ? __d_alloc+0x100/0x180
[<ffffffff8118cd85>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff8108510d>] copy_process+0x1dd/0x1970
[<ffffffff81086a51>] do_fork+0x91/0x320
[<ffffffff81220195>] ? __fd_install+0x25/0x60
[<ffffffff81086d66>] SyS_clone+0x16/0x20
[<ffffffff816b5319>] stub_clone+0x69/0x90
[<ffffffff816b4fc9>] ? system_call_fastpath+0x16/0x1b
Code: 48 89 75 a0 48 89 4d 90 83 f8 80 74 2e 45 84 ed 0f 85 16 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 04 01 00 00 8b 43 18 <41> 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b 45 a8 0f
Complete report #104598
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a77ec
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x9c
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a7c3d
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x22d
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e6be1
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x221
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a7a10
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a8aa9
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a8c9c
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a9051
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8169f69d
Build id: 3.10.0-693.11.1.el7.x86_64
|
0xac
|
|
9
|
__alloc_pages_slowpath
|
vmlinux |
0xffffffff8169fb6d
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x3e9
|
|
10
|
|
vmlinux |
0xffffffff8121ad00
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x170
|
|
11
|
__alloc_pages_nodemask
|
vmlinux |
0xffffffff8118cdb5
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x405
|
|
12
|
copy_process
|
vmlinux |
0xffffffff8108511d
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x1dd
|
|
13
|
do_fork
|
vmlinux |
0xffffffff81086a61
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x91
|
|
14
|
|
vmlinux |
0xffffffff81220125
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x25
|
|
15
|
SyS_clone
|
vmlinux |
0xffffffff81086d76
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x16
|
|
16
|
stub_clone
|
vmlinux |
0xffffffff816b53d9
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x69
|
|
17
|
|
vmlinux |
0xffffffff816b5089
Build id: 3.10.0-693.11.1.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [systemd:1]
Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 nfit shpchp vmw_vmci libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci crct10dif_pclmul libahci crct10dif_common
ata_piix crc32c_intel libata serio_raw vmxnet3 vmw_pvscsi i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 1 Comm: systemd Tainted: G L ------------ 3.10.0-693.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff88017a1b0000 ti: ffff88017a1b8000 task.ti: ffff88017a1b8000
RIP: 0010:[<ffffffff811a77ec>] [<ffffffff811a77ec>] isolate_freepages_block+0x9c/0x2c0
RSP: 0018:ffff88017a1bb948 EFLAGS: 00010202
RAX: 00000000ffffffff RBX: ffff88043ffd7800 RCX: ffff88017a1bbb40
RDX: 0000000000079400 RSI: ffff88017a1bbb98 RDI: ffff88017a1bbb40
RBP: ffff88017a1bb9d8 R08: 0000000000000000 R09: ffff88043ffd7800
R10: 0000000000002000 R11: 0000000000100000 R12: ffff88017a1bb918
R13: ffff88017a1bb988 R14: ffffea0000071ec0 R15: ffffea0002ffca80
FS: 00007fa53d524900(0000) GS:ffff88043fd40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f41623038f0 CR3: 0000000428e4b000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffea0002ffc180 ffff880424ad9740 ffff88017a1bba30 ffffea0000070b80
ffff88017a1bbb40 ffff88017a1bbb80 ffff88017a1bbb98 ffff88017a1bbb40
ffff88043ffd7800 0000000000000000 0000000000079400 0000000000000001
Call Trace:
[<ffffffff811a7c3d>] compaction_alloc+0x22d/0x2a0
[<ffffffff811e6be1>] migrate_pages+0x221/0x7a0
[<ffffffff811a7a10>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a8aa9>] compact_zone+0x2d9/0x430
[<ffffffff811a8c9c>] compact_zone_order+0x9c/0xf0
[<ffffffff811a9051>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8169f69d>] __alloc_pages_direct_compact+0xac/0x193
[<ffffffff8169fb6d>] __alloc_pages_slowpath+0x3e9/0x724
[<ffffffff8121ad00>] ? __d_alloc+0x170/0x180
[<ffffffff8118cdb5>] __alloc_pages_nodemask+0x405/0x420
[<ffffffff8108511d>] copy_process+0x1dd/0x1970
[<ffffffff81086a61>] do_fork+0x91/0x320
[<ffffffff81220125>] ? __fd_install+0x25/0x60
[<ffffffff81086d76>] SyS_clone+0x16/0x20
[<ffffffff816b53d9>] stub_clone+0x69/0x90
[<ffffffff816b5089>] ? system_call_fastpath+0x16/0x1b
Code: 45 89 c5 48 89 75 a0 48 89 4d 90 83 f8 80 74 2e 45 84 ed 0f 85 16 01 00 00 49 83 c4 01 48 83 c3 40 4c 39 65 c0 0f 86 04 01 00 00 <8b> 43 18 41 83 c7 01 4d 85 f6 4c 0f 44 f3 83 f8 80 75 d2 48 8b