Complete report #635325
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff9cbba5ec
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x9c
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff9cbbaa3d
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x22d
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff9cbff704
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x224
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff9cbba810
Build id: 3.10.0-862.9.1.el7.x86_64
|
0x2c0
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [gdb:125735]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi aesni_intel ppdev lrw snd_ac97_codec gf128mul vmw_balloon glue_helper ablk_helper ac97_bus cryptd snd_seq snd_seq_device snd_pcm joydev snd_timer snd pcspkr soundcore nfit libnvdimm sg vmw_vmci shpchp i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod
crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata e1000 mptspi scsi_transport_spi serio_raw mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 125735 Comm: gdb Kdump: loaded Tainted: G W L ------------ 3.10.0-862.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff9efaef048000 ti: ffff9ef9b2aec000 task.ti: ffff9ef9b2aec000
RIP: 0010:[<ffffffff9cbba5ec>] [<ffffffff9cbba5ec>] isolate_freepages_block+0x9c/0x2c0
RSP: 0000:ffff9ef9b2aef8d8 EFLAGS: 00010202
RAX: 00000000ffffffff RBX: 0000000000000002 RCX: ffff9ef9b2aefad0
RDX: 0000000000533200 RSI: ffff9ef9b2aefb28 RDI: ffff9ef9b2aefad0
RBP: ffff9ef9b2aef968 R08: 0000000000000000 R09: ffff9efb12fda000
R10: 0000000000100200 R11: 0000000000793000 R12: ffffffff9cb9e917
R13: ffff9ef9b2aef970 R14: 0000000000000002 R15: 0000000000000000
FS: 00007f45091a0840(0000) GS:ffff9efaf3000000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000004258510 CR3: 0000000648ee4000 CR4: 00000000001607f0
Call Trace:
[<ffffffff9cbbaa3d>] compaction_alloc+0x22d/0x2a0
[<ffffffff9cbff704>] migrate_pages+0x224/0x7a0
[<ffffffff9cbba810>] ? isolate_freepages_block+0x2c0/0x2c0
Complete report #207335
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff811a787c
Build id: 3.10.0-693.el7.x86_64
|
0x9c
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff811a7ccd
Build id: 3.10.0-693.el7.x86_64
|
0x22d
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811e6c51
Build id: 3.10.0-693.el7.x86_64
|
0x221
|
|
4
|
?
isolate_freepages_block
|
vmlinux |
0xffffffff811a7aa0
Build id: 3.10.0-693.el7.x86_64
|
0x2c0
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [gitlab-unicorn-:1618]
Modules linked in: tcp_diag inet_diag cfg80211 rfkill tun ip_set nfnetlink bridge stp llc fuse coretemp ppdev vmw_balloon iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel sg aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev parport_pc parport vmw_vmci shpchp i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix mptspi serio_raw scsi_transport_spi libata mptscsih e1000 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 1618 Comm: gitlab-unicorn- Tainted: G L ------------ 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff88061e8ecf10 ti: ffff88061ab3c000 task.ti: ffff88061ab3c000
RIP: 0010:[<ffffffff811a787c>] [<ffffffff811a787c>] isolate_freepages_block+0x9c/0x2c0
RSP: 0018:ffff88061ab3f948 EFLAGS: 00010202
RAX: 00000000ffffffff RBX: ffff8806266d6d40 RCX: ffff88061ab3fb40
RDX: 0000000000250a00 RSI: ffff88061ab3fb98 RDI: ffff88061ab3fb40
RBP: ffff88061ab3f9d8 R08: 0000000000000000 R09: ffff88063ffd8000
R10: 0000000000105a00 R11: 0000000000640000 R12: ffffffff811a31b8
R13: ffff88061ab3f8d0 R14: 0000000000000082 R15: ffff88061ab3f8e0
FS: 00007f2916fec740(0000) GS:ffff880626640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f29169c8490 CR3: 0000000618f43000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88063ffd8090 ffff88061ab3f9c0 ffff88061ab3fa30 ffffffff81911047
ffff88061ab3fb40 ffff88061ab3fb80 ffff88061ab3fb98 ffff88061ab3fb40
ffff88063ffd8000 0000000000000000 0000000000250a00 0000000000000001
Call Trace:
[<ffffffff811a7ccd>] compaction_alloc+0x22d/0x2a0
[<ffffffff811e6c51>] migrate_pages+0x221/0x7a0
[<ffffffff811a7aa0>] ? isolate_freepages_block+0x2c0/0x2c0
Complete report #752019
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffffb67b9e2c
Build id: 3.10.0-862.el7.x86_64
|
0x9c
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffffb67ba27d
Build id: 3.10.0-862.el7.x86_64
|
0x22d
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffffb67fef44
Build id: 3.10.0-862.el7.x86_64
|
0x224
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [java:61678]
Modules linked in: ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi ac97_bus crc32_pclmul ghash_clmulni_intel snd_seq aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_seq_device snd_pcm snd_timer snd ppdev soundcore vmw_balloon vmw_vmci i2c_piix4 joydev pcspkr sg shpchp nfit libnvdimm parport_pc parport binfmt_misc xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix crct10dif_pclmul scsi_transport_spi crct10dif_common drm libata crc32c_intel e1000 mptscsih mptbase serio_raw i2c_core [last unloaded: ip_tables]
CPU: 2 PID: 61678 Comm: java Tainted: G W L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8beab426cf10 ti: ffff8be5353b8000 task.ti: ffff8be5353b8000
RIP: 0010:[<ffffffffb67b9e2c>] [<ffffffffb67b9e2c>] isolate_freepages_block+0x9c/0x2c0
RSP: 0018:ffff8be5353bb438 EFLAGS: 00010202
RAX: 00000000ffffffff RBX: ffffffffb6d12b08 RCX: ffff8be5353bb630
RDX: 0000000000f4a800 RSI: ffff8be5353bb688 RDI: ffff8be5353bb630
RBP: ffff8be5353bb4c8 R08: 0000000000000000 R09: ffff8beefffda000
R10: 0000000000101c00 R11: 0000000001040000 R12: ffffffffb6d12b08
R13: ffffffffb6d12b14 R14: ffffffffb6d12b08 R15: ffffffffb6d12b14
FS: 00002b74c940f700(0000) GS:ffff8beebd680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002ba7b2f5f100 CR3: 000000097cb72000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffb67ba27d>] compaction_alloc+0x22d/0x2a0
[<ffffffffb67fef44>] migrate_pages+0x224/0x7a0
Complete report #165449
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
isolate_freepages_block
|
vmlinux |
0xffffffff8118e37c
Build id: 3.10.0-327.36.3.el7.x86_64
|
0xac
|
|
2
|
compaction_alloc
|
vmlinux |
0xffffffff8118e7cf
Build id: 3.10.0-327.36.3.el7.x86_64
|
0x1cf
|
|
3
|
migrate_pages
|
vmlinux |
0xffffffff811c7909
Build id: 3.10.0-327.36.3.el7.x86_64
|
0x1e9
|
BUG: soft lockup - CPU#6 stuck for 26s! [tc:8890]
Modules linked in: ppp_async crc_ccitt ppp_generic slhc iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack ip6_tables iptable_mangle iptable_raw ipt_REJECT iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr sg parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper ttm crct10dif_pclmul crct10dif_common ata_piix drm mptspi scsi_transport_spi crc32c_intel serio_raw mptscsih libata mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log
dm_mod
CPU: 6 PID: 8890 Comm: tc Not tainted 3.10.0-327.36.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880f25f69700 ti: ffff88019d5f4000 task.ti: ffff88019d5f4000
RIP: 0010:[<ffffffff8118e37c>] [<ffffffff8118e37c>] isolate_freepages_block+0xac/0x330
BUG: soft lockup - CPU#5 stuck for 27s! [httpd:32051]
Modules linked in: ppp_async crc_ccitt ppp_generic slhc iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack ip6_tables iptable_mangle iptable_raw ipt_REJECT iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr sg parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper ttm crct10dif_pclmul crct10dif_common ata_piix drm mptspi scsi_transport_spi crc32c_intel serio_raw mptscsih libata mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log
dm_mod
CPU: 5 PID: 32051 Comm: httpd Not tainted 3.10.0-327.36.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880f24cb8000 ti: ffff880f252ac000 task.ti: ffff880f252ac000
RIP: 0010:[<ffffffff8118e37c>] [<ffffffff8118e37c>] isolate_freepages_block+0xac/0x330
RSP: 0018:ffff880f252af818 EFLAGS: 00010286
RAX: 00000000ffffffff RBX: 0000000000000001 RCX: ffff880f252afa68
RDX: 00000000002f7600 RSI: 00000000002f7400 RDI: ffff880f252afa10
RBP: ffff880f252af8b0 R08: 0000000000000000 R09: ffff8810143d8000
R10: 000000000bdd0000 R11: 0000000001014400 R12: 0000000000000003
R13: 0000000000000001 R14: ffff881013fcef20 R15: ffff8810143d8000
FS: 00007f6a30fe1700(0000) GS:ffff881014140000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000df2008 CR3: 0000000fbc8e5000 CR4: 00000000003407e0
RSP: 0018:ffff88019d5f75a8 EFLAGS: 00010286
RAX: 00000000ffffffff RBX: ffff88019d5f77e0 RCX: ffff88019d5f77f8
RDX: 00000000002fda00 RSI: 00000000002fd800 RDI: ffff88019d5f77a0
RBP: ffff88019d5f7640 R08: 0000000000000000 R09: ffff8810143d8000
R10: 000000000bf60000 R11: 0000000001014400 R12: ffff88019d5f77f8
R13: ffff8810143d8000 R14: 000000000dbb0000 R15: 0000000001014400
FS: 00007f346f02e740(0000) GS:ffff881014180000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd0603bec0 CR3: 000000019bbd2000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000001014400 0000000000000034 ffff880f252af900 0000000000000034
00000000002f7400 ffff880f252afa10 0000000001014400 0000000000000000
ffff880f252afa50 ffff880f252afa68 ffff880f252afa10 ffff8810143d8000
Call Trace:
[<ffffffff8118e7cf>] compaction_alloc+0x1cf/0x240
[<ffffffff811c7909>] migrate_pages+0x1e9/0x790