| Operating system | Count |
|---|---|
| CentOS 7 | 58 |
| Architecture | Count |
|---|---|
| x86_64 | 58 |
| Related packages | Count |
|---|---|
| kernel | 57 |
| kernel-lt | 1 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__free_pages_ok.part.60
|
vmlinux |
0xffffffff811713e6
Build id: 3.10.0-327.el7.x86_64 |
0x16 |
BUG: soft lockup - CPU#0 stuck for 22s! [watchdog/0:907] Modules linked in: 8021q garp stp mrp llc mpt2sas raid_class scsi_transport_sas mptctl mptbase beegfs(OE) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt dell_rbu ib_ucm ext4 mbcache jbd2 dcdbas coretemp kvm_intel svcrdma xprtrdma kvm rdma_ucm crc32_pclmul ib_uverbs(OE) ghash_clmulni_intel ib_iser ib_umad rdma_cm ipmi_devintf ib_ipoib iw_cm aesni_intel lrw ib_cm gf128mul libiscsi glue_helper ablk_helper ib_sa cryptd scsi_transport_iscsi pcspkr raid0 sg shpchp wmi ipmi_si ipmi_msghandler acpi_pad acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace ip_tables sd_mod crc_t10dif crct10dif_generic hfi1(OE) crct10dif_pclmul crct10dif_common rdmavt(OE) crc32c_intel ib_mad i40e ib_core igb ib_addr vxlan i2c_algo_bit ahci ip6_udp_tunnel i2c_core libahci dca udp_tunnel ptp pps_core libata usb_storage sunrpc CPU: 0 PID: 907 Comm: watchdog/0 Tainted: G OEL ------------ 3.10.0-327.el7.x86_64 #1 Hardware name: Dell Inc. PowerEdge C6420/0XG00K, BIOS 2.19.1 06/12/2023 task: ffff8817b905ae00 ti: ffff8817b9094000 task.ti: ffff8817b9094000 RIP: 0010:[<ffffffff811713e6>] [<ffffffff811713e6>] __free_pages_ok.part.60+0x16/0x90 RSP: 0000:ffff8817df603a50 EFLAGS: 00000202 RAX: 0000000000000202 RBX: fff00000fe000000 RCX: fff00000ffffffff RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffffea005e28f400 RBP: ffff8817df603a68 R08: fff00000fe000000 R09: 0000000000000000 R10: 0000025c00000000 R11: 0000800000000040 R12: ffff8817df6039c8 R13: ffffffff8164655d R14: ffff8817df603a68 R15: ffffea005e28f400 FS: 0000000000000000(0000) GS:ffff8817df600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000b8 CR3: 000000000194a000 CR4: 00000000003407f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffffea005e28f400 0000000000000003 ffff882fb67ed200 ffff8817df603a88 ffffffff81171498 ffffea005e28f400 ffff882fb67ed200 ffff8817df603aa0 ffffffff81176cdf ffffea005e28f400 ffff8817df603ac8 ffffffff81176d51 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__free_pages_ok
|
vmlinux |
0xffffffff957c7109
Build id: 3.10.0-1160.24.1.el7.x86_64 |
0x99 |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [kworker/1:1:29874] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_15402(E) nf_conntrack_netlink falcon_lsm_pinned_15309(E) falcon_lsm_pinned_14713(E) falcon_lsm_pinned_14604(E) falcon_lsm_pinned_14306(E) falcon_lsm_pinned_14504(E) xt_CHECKSUM xt_addrtype ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables br_netfilter tun devlink nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ipmi_devintf ipmi_msghandler overlay(T) bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock ppdev sb_edac vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi nfit scsi_transport_spi mptscsih crct10dif_pclmul libnvdimm crct10dif_common crc32c_intel mptbase serio_raw vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal] CPU: 1 PID: 29874 Comm: kworker/1:1 Kdump: loaded Tainted: P W E ------------ T 3.10.0-1160.24.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018 Workqueue: events fb_flashcursor task: ffff905fccbde300 ti: ffff905a14394000 task.ti: ffff905a14394000 RIP: 0010:[<ffffffff957c7109>] [<ffffffff957c7109>] __free_pages_ok+0x99/0xc0 RSP: 0018:ffff905fdf243d48 EFLAGS: 00000202 RAX: 000000000001fd48 RBX: 00000000000000ec RCX: ffffffffffffff83 RDX: 0000000000000008 RSI: 0000000000000000 RDI: 0000000000000202 RBP: ffff905fdf243d60 R08: 0000000000000006 R09: 0000000000000000 R10: ffff905fffbd4e40 R11: fff00000f0000000 R12: ffff905fdf243cb8 R13: ffffffff95d96fba R14: ffff905fdf243d60 R15: ffffeabf8f4c3b80 FS: 0000000000000000(0000) GS:ffff905fdf240000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f2953fa3000 CR3: 0000000586410000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__free_pages_ok
|
vmlinux |
0xffffffff8118af69
Build id: 3.10.0-693.2.2.el7.x86_64 |
0x99 |
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [khungtaskd:29] Modules linked in: gsch(OE) redirfs(OE) nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dsa_filter(POE) vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd ppdev sg vmw_vmci joydev pcspkr parport_pc shpchp parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common mptspi scsi_transport_spi mptscsih crc32c_intel serio_raw ata_piix libata mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod CPU: 2 PID: 29 Comm: khungtaskd Tainted: P OEL ------------ 3.10.0-693.2.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014 task: ffff88017cf21fa0 ti: ffff88023f7ec000 task.ti: ffff88023f7ec000 RIP: 0010:[<ffffffff8118af69>] [<ffffffff8118af69>] __free_pages_ok+0x99/0xc0 RSP: 0018:ffff88023fd03330 EFLAGS: 00000202 RAX: 000000000001a1b0 RBX: 0000000000000002 RCX: ffffffffffffffdc RDX: ffffffffffffffe8 RSI: 0000000000000000 RDI: 0000000000000202 RBP: ffff88023fd03348 R08: ffffffffffffffe4 R09: 0000000000000000 R10: ffff88023fbd5920 R11: ffffea000126ce00 R12: ffff88023fd032a8 R13: ffffffff816b5c5d R14: ffff88023fd03348 R15: ffffea000126ce00 FS: 0000000000000000(0000) GS:ffff88023fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000006eb0f8 CR3: 00000000bb4de000 CR4: 00000000001407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Stack: ffffea000126ce00 0000000000000002 ffffffffc0554e98 ffff88023fd03358 ffffffff8118b7bd ffff88023fd03378 ffffffff8118b9d2 ffff880049b38000 ffff880049b38000 ffff88023fd033a0 ffffffff811deee4 ffff880049b38010 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__free_pages_ok
|
vmlinux |
0xffffffffa75bf319
Build id: 3.10.0-957.10.1.el7.x86_64 |
0x99 |
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [kworker/u20:0:21989] Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ixgbe_zc(OE) ptp pps_core vxlan ip6_udp_tunnel udp_tunnel uio pf_ring(OE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon joydev pcspkr parport_pc parport sg vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmwgfx drm_kms_helper syscopyarea floppy sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: pps_core] CPU: 9 PID: 21989 Comm: kworker/u20:0 Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.10.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 Workqueue: netns cleanup_net task: ffff934874d4d140 ti: ffff934cc1ba8000 task.ti: ffff934cc1ba8000 RIP: 0010:[<ffffffffa75bf319>] [<ffffffffa75bf319>] __free_pages_ok+0x99/0xc0 RSP: 0018:ffff934cf6403c50 EFLAGS: 00000202 RAX: 000000000001f7b0 RBX: 0000000000000003 RCX: ffffffffffffffc0 RDX: fffffffffffffffa RSI: 0000000000000000 RDI: 0000000000000202 RBP: ffff934cf6403c68 R08: fffffffffffffff2 R09: 0000000000000000 R10: ffff934cffbc9920 R11: 0000000000000001 R12: ffff934cf6403bc8 R13: ffffffffa7b76df2 R14: ffff934cf6403c68 R15: fffff12a8c2ae000 FS: 0000000000000000(0000) GS:ffff934cf6400000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ff451ba3b10 CR3: 00000000347aa000 CR4: 00000000000607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__free_pages_ok
|
vmlinux |
0xffffffffa3dc0619
Build id: 3.10.0-957.27.2.el7.x86_64 |
0x99 |
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 375s! [abrtd:1255] Modules linked in: xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 tun ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_conntrack ebtable_filter ebtables ip6table_filter ip6_tables devlink xt_addrtype xt_set xt_mark ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set nfnetlink dummy xt_comment iptable_nat nf_nat_ipv4 nf_nat overlay(T) nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 parport_pc parport vmw_vmci br_netfilter bridge stp llc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel mptscsih libata serio_raw nfit mptbase libnvdimm vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 7 PID: 1255 Comm: abrtd Tainted: G W L ------------ T 3.10.0-957.27.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 task: ffff9647eac00000 ti: ffff9643f50f0000 task.ti: ffff9643f50f0000 RIP: 0010:[<ffffffffa3dc0619>] [<ffffffffa3dc0619>] __free_pages_ok+0x99/0xc0 RSP: 0018:ffff9647ffdc39c8 EFLAGS: 00000202 RAX: 000000000001f748 RBX: 0000000000000270 RCX: ffffffffffffffc0 RDX: 0000000000000008 RSI: 0000000000000000 RDI: 0000000000000202 RBP: ffff9647ffdc39e0 R08: 0000000000000000 R09: 0000000000000000 R10: ffff9647ffbd6ea0 R11: ffff9647d67c0650 R12: ffff9647ffdc3938 R13: ffffffffa4377df2 R14: ffff9647ffdc39e0 R15: ffffee0b8fac9c00 FS: 00007feb3be01900(0000) GS:ffff9647ffdc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000073aa1c0 CR3: 0000000035fa2000 CR4: 00000000003607e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__free_pages_ok
|
vmlinux |
0xffffffffa1fc7299
Build id: 3.10.0-1127.el7.x86_64 |
0x99 |
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 21s! [vmtoolsd:808] Modules linked in: vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod ppdev sb_edac coretemp vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptsas crct10dif_pclmul crct10dif_common libata crc32c_intel scsi_transport_sas mptscsih e1000 mptbase serio_raw drm_panel_orientation_quirks CPU: 8 PID: 808 Comm: vmtoolsd Kdump: loaded Tainted: G L ------------ 3.10.0-1127.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff970ed8d29070 ti: ffff970f99fe4000 task.ti: ffff970f99fe4000 RIP: 0010:[<ffffffffa1fc7299>] [<ffffffffa1fc7299>] __free_pages_ok+0x99/0xc0 RSP: 0018:ffff970fa1803e08 EFLAGS: 00000202 RAX: 000000000001fd48 RBX: 00000000000003e0 RCX: ffffffffffffff83 RDX: ffffffffffffffe5 RSI: 0000000000000000 RDI: 0000000000000202 RBP: ffff970fa1803e20 R08: ffffffffffffffe1 R09: 0000000000000000 R10: ffff970fbfbd60a0 R11: ffffffffffffff83 R12: ffff970fa1803d78 R13: ffffffffa2593efa R14: ffff970fa1803e20 R15: ffffda4d8db4f800 FS: 00007ff0ba55d880(0000) GS:ffff970fa1800000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ff0ba573000 CR3: 000000071e664000 CR4: 00000000001607e0 Call Trace: