| Operating system | Count |
|---|---|
| CentOS 7 | 124 |
| Architecture | Count |
|---|---|
| x86_64 | 124 |
| Related packages | Count |
|---|---|
| kernel | 124 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
page_frag_free
|
vmlinux |
0xffffffff9f1c712a
Build id: 3.10.0-1160.15.2.el7.x86_64 |
0x3a |
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 25s! [com.alibaba.nac:11502] Modules linked in: binfmt_misc resguard_linux(OE) secmodel_linux(OE) syshook_linux(OE) secmodel3(OE) xt_set ip_set_hash_net xt_NFLOG nfnetlink_log ip_set xt_CHECKSUM iptable_raw ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock nf_conntrack_netlink nfnetlink nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev joydev vmw_balloon pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel mptspi libata scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 11 PID: 11502 Comm: com.alibaba.nac Kdump: loaded Tainted: G W OEL ------------ 3.10.0-1160.15.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff8d30a572d280 ti: ffff8d25d9bdc000 task.ti: ffff8d25d9bdc000 RIP: 0010:[<ffffffff9f1c712a>] [<ffffffff9f1c712a>] page_frag_free+0x3a/0x80 RSP: 0018:ffff8d347d8c3970 EFLAGS: 00010286 RAX: 000072db00000000 RBX: 0000000300000000 RCX: ffff8d249a4d7100 RDX: 0000000080000000 RSI: ffff8d2595fff800 RDI: ffffe7c1006935c0 RBP: ffff8d347d8c3970 R08: 0000000000000000 R09: 0000000000000000 R10: 00000000ad9480f2 R11: ffffe7c1065d3600 R12: ffff8d347d8c38e8 R13: ffffffff9f795fba R14: ffff8d347d8c3970 R15: ffff8d2595fff800 FS: 00007f0d9e4fa700(0000) GS:ffff8d347d8c0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f0ddc0b8000 CR3: 000000016dbae000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
page_frag_free
|
vmlinux |
0xffffffffa31d2406
Build id: 3.10.0-1160.114.2.el7.x86_64 |
0x46 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:1H:567] Modules linked in: esvacc(OE) esvnic(OE) 8021q garp mrp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc vfat fat iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm ipmi_ssif irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd mei_me pcspkr mei i2c_i801 lpc_ich sg wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_pad acpi_power_meter ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm i40e ahci drm libahci crct10dif_pclmul crct10dif_common libata crc32c_intel ptp pps_core drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: esvnic] CPU: 0 PID: 567 Comm: kworker/0:1H Kdump: loaded Tainted: G B OEL ------------ 3.10.0-1160.114.2.el7.x86_64 #1 Hardware name: Lenovo ThinkSystem ST558 -[7Y16CTO1WW]-/-[7Y16CTO1WW]-, BIOS -[O0E176H-3.20]- 11/04/2021 Workqueue: kblockd blk_timeout_work task: ffff9f919878c200 ti: ffff9f919856c000 task.ti: ffff9f919856c000 RIP: 0010:[<ffffffffa31d2406>] [<ffffffffa31d2406>] page_frag_free+0x46/0x80 RSP: 0018:ffff9f953be03da0 EFLAGS: 00000202 RAX: 002fffff00004000 RBX: ffffffffa37c9ef6 RCX: 0000000000000000 RDX: 0000000080000000 RSI: 00000000000000be RDI: ffffee722241f200 RBP: ffff9f953be03da0 R08: ffff9f8e73400000 R09: 0000000000000600 R10: 0000000000000000 R11: 0000000000000001 R12: ffff9f953be03d18 R13: ffffffffa37c63f2 R14: ffff9f953be03da0 R15: ffff9f919b3fc800 FS: 0000000000000000(0000) GS:ffff9f953be00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000b8 CR3: 0000000c72a10000 CR4: 00000000007607f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 00000000 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
page_frag_free
|
vmlinux |
0xffffffff8f3c741a
Build id: 3.10.0-1127.19.1.el7.x86_64 |
0x3a |
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [zabbix_agentd:1309] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel sg lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon ppdev pcspkr vmw_vmci joydev parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic ahci drm nfit libahci ata_piix crct10dif_pclmul crct10dif_common libnvdimm libata vmxnet3 crc32c_intel serio_raw vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 7 PID: 1309 Comm: zabbix_agentd Kdump: loaded Tainted: G L ------------ 3.10.0-1127.19.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 task: ffff923e6588a0e0 ti: ffff923e63174000 task.ti: ffff923e63174000 RIP: 0010:[<ffffffff8f3c741a>] [<ffffffff8f3c741a>] page_frag_free+0x3a/0x80 RSP: 0018:ffff923e7fdc3ad0 EFLAGS: 00010286 RAX: 00006dc740000000 RBX: ffff923e5ec8d180 RCX: 0000000000000089 RDX: 0000000080000000 RSI: 0000000000000002 RDI: fffff516c5d813c0 RBP: ffff923e7fdc3ad0 R08: 000000000000004e R09: 000000001e0010ac R10: ffff9239b604f8e2 R11: ffffffff8ff15c80 R12: ffff923e7fdc3a48 R13: ffffffff8f993efa R14: ffff923e7fdc3ad0 R15: ffff923e629d1700 FS: 00007f17a3627d80(0000) GS:ffff923e7fdc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f17a364a000 CR3: 00000006240d4000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
page_frag_free
|
vmlinux |
0xffffffffa899fa81
Build id: 3.10.0-862.14.4.el7.x86_64 |
0x41 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [vmtoolsd:633] Modules linked in: snapapi26(OE) tcp_lp nls_utf8 isofs binfmt_misc fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr vmxnet3 sg joydev nfit libnvdimm parport_pc parport vmw_vmci i2c_piix4 shpchp ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm serio_raw mptspi scsi_transport_spi mptscsih ata_piix mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: edd] CPU: 0 PID: 633 Comm: vmtoolsd Kdump: loaded Tainted: P OEL ------------ 3.10.0-862.14.4.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 task: ffff88c6b6b25ee0 ti: ffff88c737d90000 task.ti: ffff88c737d90000 RIP: 0010:[<ffffffffa899fa81>] [<ffffffffa899fa81>] page_frag_free+0x41/0x80 RSP: 0000:ffff88c73fc03c48 EFLAGS: 00010282 RAX: fffffce884122600 RBX: ffff88c70489f8ce RCX: ffff88c6b927c000 RDX: 002fffff00008000 RSI: 000000000000000c RDI: fffffce8841227c0 RBP: ffff88c73fc03c68 R08: ffffffffa8e0b3d0 R09: 0000000000000000 R10: ffff88c6a990cae0 R11: ffff88c6a9ad9050 R12: ffff88c73fc03bb8 R13: ffffffffa8f267b2 R14: ffff88c73fc03c68 R15: ffff88c67f500d00 FS: 00007f917b5dc880(0000) GS:ffff88c73fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ff26da3ed48 CR3: 00000000b6a96000 CR4: 00000000001607f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
page_frag_free
|
vmlinux |
0xffffffffa1dc4d3a
Build id: 3.10.0-1062.4.1.el7.x86_64 |
0x3a |
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [gnome-shell:3439] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14203(E) falcon_lsm_pinned_14108(E) falcon_lsm_pinned_14006(E) devlink binfmt_misc ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_hda_codec_generic lrw gf128mul glue_helper ablk_helper cryptd joydev snd_hda_intel virtio_balloon snd_hda_codec sg snd_hda_core snd_hwdep parport_pc snd_seq parport snd_seq_device snd_pcm snd_timer snd i2c_piix4 pcspkr soundcore auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal] CPU: 6 PID: 3439 Comm: gnome-shell Kdump: loaded Tainted: P EL ------------ 3.10.0-1062.4.1.el7.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 task: ffffa053d91920e0 ti: ffffa024b5d0c000 task.ti: ffffa024b5d0c000 RIP: 0010:[<ffffffffa1dc4d3a>] [<ffffffffa1dc4d3a>] page_frag_free+0x3a/0x80 RSP: 0000:ffffa053df183ba0 EFLAGS: 00010282 RAX: 00005fdb00000000 RBX: ffffa04bf47ec300 RCX: 0000000000000006 RDX: 0000000080000000 RSI: 0000000000000000 RDI: ffffddd01023ed40 RBP: ffffa053df183ba0 R08: 000000000a700289 R09: ffffa053c9a52500 R10: 0000000000023c9c R11: 0000000000000001 R12: ffffa053df183b18 R13: ffffffffa238cefa R14: ffffa053df183ba0 R15: ffffa04bf47ec300 FS: 00007f9bbc832a80(0000) GS:ffffa053df180000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000083b81006 CR3: 0000002f2fbbc000 CR4: 00000000001606e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
page_frag_free
|
vmlinux |
0xffffffffacdc70ca
Build id: 3.10.0-1160.21.1.el7.x86_64 |
0x3a |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [nginx:20144] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 29s! [watchdog/3:22] Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables btrfs raid6_pq xor vfat msdos fat ext4 mbcache jbd2 cdp(OE) binfmt_misc tun devlink bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock sunrpc coretemp crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev parport_pc sg parport vmw_vmci i2c_piix4 xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm serio_raw ata_piix mptspi scsi_transport_spi libata mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ip_tables] CPU: 3 PID: 22 Comm: watchdog/3 Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.21.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015 task: ffff8cef3db7b180 ti: ffff8cef3df48000 task.ti: ffff8cef3df48000 RIP: 0010:[<ffffffffacdc70ca>] [<ffffffffacdc70ca>] page_frag_free+0x3a/0x80 RSP: 0000:ffff8cefffd839c0 EFLAGS: 00010282 RAX: 00007311c0000000 RBX: ffff8cefa757a6c0 RCX: ffff8cefb19ee980 RDX: 0000000080000000 RSI: ffff8cef843b8200 RDI: ffffd48007c67b80 RBP: ffff8cefffd839c0 R08: 19c4ec88bb01e30c R09: 0000000000c80036 R10: 4bb8f48a585a19c4 R11: ffff8cedf45121f0 R12: ffff8cefffd83938 R13: ffffffffad396fba R14: ffff8cefffd839c0 R15: ffff8cef843b8200 FS: 0000000000000000(0000) GS:ffff8cefffd80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f020c8dc000 CR3: 00000001f6b14000 CR4: 00000000000007e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
page_frag_free
|
vmlinux |
0xffffffffa3dc06e6
Build id: 3.10.0-957.27.2.el7.x86_64 |
0x36 |
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 294s! [abrtd:1255] Modules linked in: xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 tun ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_conntrack ebtable_filter ebtables ip6table_filter ip6_tables devlink xt_addrtype xt_set xt_mark ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set nfnetlink dummy xt_comment iptable_nat nf_nat_ipv4 nf_nat overlay(T) nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 parport_pc parport vmw_vmci br_netfilter bridge stp llc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix crct10dif_pclmul crct10dif_common scsi_transport_spi crc32c_intel mptscsih libata serio_raw nfit mptbase libnvdimm vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 5 PID: 1255 Comm: abrtd Tainted: G W L ------------ T 3.10.0-957.27.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 task: ffff9647eac00000 ti: ffff9643f50f0000 task.ti: ffff9643f50f0000 RIP: 0010:[<ffffffffa3dc06e6>] [<ffffffffa3dc06e6>] page_frag_free+0x36/0x80 RSP: 0000:ffff9647ffd439f8 EFLAGS: 00010286 RAX: 000069bbc0000000 RBX: 0000000000d43980 RCX: 0000000000000008 RDX: 0000000080000000 RSI: 0000000000000000 RDI: ffffee0b8eaed5c0 RBP: ffff9647ffd43a18 R08: 000000000000ec4b R09: 000000000000192b R10: 00000000790eb9fe R11: 000000002208010a R12: ffff9647ffd43968 R13: ffffffffa4377df2 R14: ffff9647ffd43a18 R15: ffff9647ee1ea300 FS: 00007feb3be01900(0000) GS:ffff9647ffd40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f8f835b042c CR3: 0000000035fa2000 CR4: 00000000003607e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: