Complete report #1421208
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a787c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a7ccd
Build id: 3.10.0-693.el7.x86_64
|
0x22d
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e6c51
Build id: 3.10.0-693.el7.x86_64
|
0x221
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a7aa0
Build id: 3.10.0-693.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a8b39
Build id: 3.10.0-693.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a8d2c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a90e1
Build id: 3.10.0-693.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8169f5cd
Build id: 3.10.0-693.el7.x86_64
|
0xac
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [antivirus.monit:13155]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter arc4 md4 nls_utf8 cifs dns_resolver vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptsas drm scsi_transport_sas
ata_piix mptscsih crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 13155 Comm: antivirus.monit Tainted: G L ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880bf5e00fd0 ti: ffff880bf2a44000 task.ti: ffff880bf2a44000
RIP: 0010:[<ffffffff811a787c>] [<ffffffff811a787c>] isolate_freepages_block+0x9c/0x2c0
RSP: 0018:ffff880bf2a47948 EFLAGS: 00010202
RAX: 00000000ffffffff RBX: 0000ffff810cd794 RCX: ffff880bf2a47b40
RDX: 000000000062ee00 RSI: ffff880bf2a47b98 RDI: ffff880bf2a47b40
RBP: ffff880bf2a479d8 R08: 0000000000000000 R09: ffff880c3ffd8000
R10: 0000000000100c00 R11: 0000000000c40000 R12: ffff880b15052be0
R13: 0000000000000000 R14: 0000000000000000 R15: 00ff880b00000000
FS: 00007f5054196740(0000) GS:ffff880c0f100000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001545d08 CR3: 0000000bf2b53000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880bf2a479d8 0000000000100c00 0000000000c40000 0000000000100c00
ffff880bf2a47b40 ffff880bf2a47b80 ffff880bf2a47b98 ffff880bf2a47b40
ffff880c3ffd8000 0000000000000000 000000000062ee00 ffffffffffffff10
Call Trace:
[<ffffffff811a7ccd>] compaction_alloc+0x22d/0x2a0
[<ffffffff811e6c51>] migrate_pages+0x221/0x7a0
[<ffffffff811a7aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a8b39>] compact_zone+0x2d9/0x430
[<ffffffff811a8d2c>] compact_zone_order+0x9c/0xf0
[<ffffffff811a90e1>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8169f5cd>] __alloc_pages_direct_compact+0xac/0x193
Complete report #919936
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a787c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a7ccd
Build id: 3.10.0-693.el7.x86_64
|
0x22d
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e6c51
Build id: 3.10.0-693.el7.x86_64
|
0x221
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a7aa0
Build id: 3.10.0-693.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a8b39
Build id: 3.10.0-693.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a8d2c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a90e1
Build id: 3.10.0-693.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff8169f5cd
Build id: 3.10.0-693.el7.x86_64
|
0xac
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [snmpd:62923]
Modules linked in: sysmon_edr(OE) netfilter_edr(OE) sctp_diag sctp dccp_diag dccp udp_diag unix_diag af_packet_diag netlink_diag tcp_lp bnep fuse binfmt_misc tcp_diag inet_diag nfnetlink_queue nfnetlink_log nfnetlink bluetooth rfkill xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod ext4 mbcache jbd2 ppdev virtio_balloon joydev pcspkr parport_pc i2c_piix4 parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi virtio_net virtio_console virtio_scsi virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci serio_raw virtio_ring i2c_core floppy virtio [last unloaded: netfilter_edr]
CPU: 6 PID: 62923 Comm: snmpd Tainted: G OEL ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff8800227e1fa0 ti: ffff88001d028000 task.ti: ffff88001d028000
RIP: 0010:[<ffffffff811a787c>] [<ffffffff811a787c>] isolate_freepages_block+0x9c/0x2c0
RSP: 0018:ffff88001d02b608 EFLAGS: 00010206
RAX: 0000000000000000 RBX: ffffea0020293000 RCX: ffff88001d02b800
RDX: 000000000080a600 RSI: ffff88001d02b858 RDI: ffff88001d02b800
RBP: ffff88001d02b698 R08: 0000000000000000 R09: ffff88103ffa3000
R10: 0000000000d8f800 R11: 0000000001040000 R12: 000000000080a4c0
R13: 0000000000000000 R14: ffffea0020290000 R15: 00000000000000c0
FS: 00007fce8900e880(0000) GS:ffff880ffcf80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000043557006 CR3: 0000000fe7681000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000020290000 000000000080a400 ffff88001d02b6f0 ffff88001d02b800
ffff88001d02b800 ffff88001d02b840 ffff88001d02b858 ffff88001d02b800
ffff88103ff4f000 0000000000000000 000000000080a600 0000000000000004
Call Trace:
[<ffffffff811a7ccd>] compaction_alloc+0x22d/0x2a0
[<ffffffff811e6c51>] migrate_pages+0x221/0x7a0
[<ffffffff811a7aa0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a8b39>] compact_zone+0x2d9/0x430
[<ffffffff811a8d2c>] compact_zone_order+0x9c/0xf0
[<ffffffff811a90e1>] try_to_compact_pages+0x121/0x1a0
[<ffffffff8169f5cd>] __alloc_pages_direct_compact+0xac/0x193
Complete report #925488
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
compaction_alloc
|
vmlinux |
0xffffffff87dba17e
Build id: 3.10.0-862.el7.x86_64
|
0x12e
|
|
2
|
migrate_pages
|
vmlinux |
0xffffffff87dfef44
Build id: 3.10.0-862.el7.x86_64
|
0x224
|
|
3
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff87dba050
Build id: 3.10.0-862.el7.x86_64
|
0x2c0
|
|
4
|
compact_zone
|
vmlinux |
0xffffffff87dbb139
Build id: 3.10.0-862.el7.x86_64
|
0x2d9
|
|
5
|
compact_zone_order
|
vmlinux |
0xffffffff87dbb32c
Build id: 3.10.0-862.el7.x86_64
|
0x9c
|
|
6
|
try_to_compact_pages
|
vmlinux |
0xffffffff87dbb6e1
Build id: 3.10.0-862.el7.x86_64
|
0x121
|
|
7
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff88309531
Build id: 3.10.0-862.el7.x86_64
|
0xac
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [ksmtuned:129890]
Modules linked in: binfmt_misc fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event pcspkr crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec aesni_intel lrw gf128mul glue_helper ablk_helper ac97_bus snd_seq snd_seq_device cryptd snd_pcm ppdev snd_timer snd soundcore
sg nfit vmw_balloon joydev libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi crct10dif_pclmul libata crct10dif_common mptscsih crc32c_intel mptbase e1000 serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 129890 Comm: ksmtuned Tainted: G L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8b68986b2f70 ti: ffff8b6338738000 task.ti: ffff8b6338738000
RIP: 0010:[<ffffffff87dba17e>] [<ffffffff87dba17e>] compaction_alloc+0x12e/0x2a0
RSP: 0000:ffff8b633873b7e8 EFLAGS: 00010286
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [php-fpm:15750]
RAX: ffff8b69eebd72a0 RBX: ffff8b6958f9e400 RCX: ffffe83e52a58000
RDX: ffff8b69eebd7000 RSI: 0000000000000004 RDI: 0000000000000030
RBP: ffff8b633873b830 R08: 0000000000000001 R09: ffff8b69eebda000
R10: 0000000000100200 R11: 000000000072ec00 R12: ffff8b69d0718b00
R13: 0000000000000000 R14: ffffffff87dba014 R15: ffff8b633873b7d8
FS: 00007f574758f740(0000) GS:ffff8b69d0700000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f89d415cff0 CR3: 0000000078782000 CR4: 00000000007607e0
DR0: 00007f0baffcf254 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
PKRU: 55555554
Call Trace:
[<ffffffff87dfef44>] migrate_pages+0x224/0x7a0
[<ffffffff87dba050>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff87dbb139>] compact_zone+0x2d9/0x430
[<ffffffff87dbb32c>] compact_zone_order+0x9c/0xf0
[<ffffffff87dbb6e1>] try_to_compact_pages+0x121/0x1a0
[<ffffffff88309531>] __alloc_pages_direct_compact+0xac/0x193
Complete report #47138
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a6f82
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xa2
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a73b5
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x215
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e4059
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x219
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a71a0
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x2c0
|
|
5
|
compact_zone
|
vmlinux |
0xffffffff811a82b9
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x2d9
|
|
6
|
compact_zone_order
|
vmlinux |
0xffffffff811a84ac
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x9c
|
|
7
|
try_to_compact_pages
|
vmlinux |
0xffffffff811a8861
Build id: 3.10.0-514.21.1.el7.x86_64
|
0x121
|
|
8
|
__alloc_pages_direct_compact
|
vmlinux |
0xffffffff81682306
Build id: 3.10.0-514.21.1.el7.x86_64
|
0xac
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [redis-server:855]
Modules linked in: fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cfg80211 rfkill vmw_vsock_vmci_transport vsock falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_lsm_pinned_10807(E) intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit vmw_balloon parport_pc pcspkr libnvdimm parport sg vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common vmwgfx
crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm drm mptspi ata_piix scsi_transport_spi mptscsih libata vmxnet3 mptbase i2c_core fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 855 Comm: redis-server Tainted: P E ------------ 3.10.0-514.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff880895140fb0 ti: ffff880891658000 task.ti: ffff880891658000
RIP: 0010:[<ffffffff811a6f82>] [<ffffffff811a6f82>] isolate_freepages_block+0xa2/0x2c0
RSP: 0000:ffff88089165b858 EFLAGS: 00010202
RAX: 00000000ffffffff RBX: 0000000000000003 RCX: ffff88089165ba40
RDX: 0000000000805200 RSI: ffff88089165ba98 RDI: ffff88089165ba40
RBP: ffff88089165b8e8 R08: 0000000000000000 R09: ffff8808bffd8000
R10: 00000000002b4600 R11: 00000000008c0000 R12: ffff8808bfbd3d40
R13: ffff8808bffd8000 R14: 00000000002b4600 R15: 00000000008c0000
FS: 00007f88d964b780(0000) GS:ffff88089b740000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f88d8495318 CR3: 000000089a51e000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 ffff8800355b3e80 ffff88089165b938 ffff8800355b3e80
ffff88089165ba40 ffff88089165ba80 ffff88089165ba98 ffff88089165ba40
ffff8808bffd4000 0000000000000000 0000000000805200 0000000000000003
Call Trace:
[<ffffffff811a73b5>] compaction_alloc+0x215/0x290
[<ffffffff811e4059>] migrate_pages+0x219/0x790
[<ffffffff811a71a0>] ? isolate_freepages_block+0x2c0/0x2c0
[<ffffffff811a82b9>] compact_zone+0x2d9/0x430
[<ffffffff811a84ac>] compact_zone_order+0x9c/0xf0
[<ffffffff811a8861>] try_to_compact_pages+0x121/0x1a0
[<ffffffff81682306>] __alloc_pages_direct_compact+0xac/0x196