| Operating system | Count |
|---|---|
| CentOS 7 | 25 |
| Architecture | Count |
|---|---|
| x86_64 | 25 |
| Related packages | Count |
|---|---|
| kernel | 25 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_ct_remove_expectations
|
nf_conntrack |
0xffffffffc06e33c1
Build id: 3.10.0-1160.80.1.el7.x86_64 |
0x11 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [xfsaild/dm-0:401] Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vboxsf(OE) sunrpc snd_intel8x0 nfit snd_ac97_codec ac97_bus libnvdimm snd_seq joydev snd_seq_device ppdev snd_pcm iosf_mbi vboxguest(OE) sg pcspkr snd_timer parport_pc video parport snd soundcore i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix libahci e1000 libata crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 401 Comm: xfsaild/dm-0 Kdump: loaded Tainted: G W OEL ------------ 3.10.0-1160.80.1.el7.x86_64 #1 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 task: ffff8e343572c200 ti: ffff8e3435624000 task.ti: ffff8e3435624000 RIP: 0010:[<ffffffffc06e33c1>] [<ffffffffc06e33c1>] nf_ct_remove_expectations+0x11/0xa0 [nf_conntrack] RSP: 0018:ffff8e381fc03da8 EFLAGS: 00010282 RAX: ffff8e3435650e40 RBX: ffff8e381fc03d28 RCX: 00000000319dc013 RDX: ffff8e380ae10708 RSI: 00000000000020e1 RDI: ffff8e380e370dc0 RBP: ffff8e381fc03df0 R08: 00000000cfbf0cbb R09: 000000001efcd14f R10: ffff8e380e370e30 R11: 0000000024584d5e R12: ffff8e381fc03d18 R13: ffffffff94fc63f2 R14: ffff8e381fc03df0 R15: ffffffff95516180 FS: 0000000000000000(0000) GS:ffff8e381fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f7ca38e3a18 CR3: 000000040ce3e000 CR4: 00000000000206f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_ct_remove_expectations
|
nf_conntrack |
0xffffffffc13442f8
Build id: 3.10.0-1160.21.1.el7.x86_64 |
0x88 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [feconf:8500] Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt rpcrdma rdma_ucm ib_umad ib_iser ib_ipoib rdma_cm iw_cm libiscsi scsi_transport_iscsi ib_cm iTCO_wdt iTCO_vendor_support mxm_wmi sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm snd_hda_codec_realtek snd_hda_codec_generic irqbypass crc32_pclmul ghash_clmulni_intel ftdi_sio aesni_intel snd_hda_intel lrw gf128mul glue_helper ablk_helper snd_hda_codec cryptd pcspkr mlx5_ib snd_hda_core sg snd_hwdep snd_seq ib_uverbs snd_seq_device snd_pcm lpc_ich joydev ib_core i2c_i801 snd_timer snd soundcore ipmi_si ipmi_devintf ipmi_msghandler wmi acpi_power_meter flx(OE) io_rcc(POE) openafs(POE) auth_rpcgss binfmt_misc cmem_rcc(OE) sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mlx5_core drm ahci libahci libata crct10dif_pclmul crct10dif_common crc32c_intel igb serio_raw mlxfw devlink ptp drm_panel_orientation_quirks dca i2c_algo_bit pps_core dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 8500 Comm: feconf Kdump: loaded Tainted: P OEL ------------ 3.10.0-1160.21.1.el7.x86_64 #1 Hardware name: Supermicro X10SRA-F/X10SRA-F, BIOS 2.0 01/28/2016 task: ffff8b49cca54200 ti: ffff8b51bd24c000 task.ti: ffff8b51bd24c000 RIP: 0010:[<ffffffffc13442f8>] [<ffffffffc13442f8>] nf_ct_remove_expectations+0x88/0x90 [nf_conntrack] RSP: 0000:ffff8b51df203da8 EFLAGS: 00000246 RAX: ffff8b516c16e300 RBX: 0000000000000001 RCX: 000000005c51c9ed RDX: 0000000000000000 RSI: 000000000000a4f1 RDI: ffff8b5170446dc0 RBP: ffff8b51df203df0 R08: 00000000508a6cba R09: 00000000885ad69b R10: ffff8b5170446e30 R11: ffff8b5170447a40 R12: ffff8b51df203d18 R13: ffffffffbd796fba R14: ffff8b51df203df0 R15: ffffffffbdd15e40 FS: 00007fd47f1f6740(0000) GS:ffff8b51df200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fd47f28e4c0 CR3: 000000015fbb0000 CR4: 00000000001607f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_ct_remove_expectations
|
nf_conntrack |
0xffffffffc0935281
Build id: 3.10.0-1160.71.1.el7.x86_64 |
0x11 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 67s! [abrt-hook-ccpp:15504] Modules linked in: ppp_deflate bsd_comp ppp_async crc_ccitt ppp_generic slhc tcp_lp nls_utf8 isofs vboxsf(OE) nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter binfmt_misc overlay(T) snd_intel8x0 joydev snd_ac97_codec ac97_bus ppdev snd_seq snd_seq_device vboxguest(OE) snd_pcm sg iosf_mbi pcspkr snd_timer i2c_piix4 snd parport_pc parport video soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm drm_panel_orientation_quirks crc32c_intel serio_raw ahci libahci ata_piix libata e1000 dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 15504 Comm: abrt-hook-ccpp Kdump: loaded Tainted: G OEL ------------ T 3.10.0-1160.71.1.el7.x86_64 #1 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 task: ffff9a3c49186300 ti: ffff9a3d1202c000 task.ti: ffff9a3d1202c000 RIP: 0010:[<ffffffffc0935281>] [<ffffffffc0935281>] nf_ct_remove_expectations+0x11/0x90 [nf_conntrack] RSP: 0018:ffff9a3d56c03d40 EFLAGS: 00010286 RAX: ffff9a3cf7ee2900 RBX: ffff9a3d56c15a58 RCX: ffffffff9f95e300 RDX: ffffffff9f87a380 RSI: 0000000000000011 RDI: ffff9a3d4dc8a000 RBP: ffff9a3d56c03d58 R08: 0000000000000000 R09: 000000018010000f R10: 0000000000000001 R11: ffff9a3c25bdb000 R12: ffff9a3d56c03cb8 R13: ffffffff9f39afba R14: ffff9a3d56c03d58 R15: ffff9a3d4dc8a000 FS: 0000000000000000(0000) GS:ffff9a3d56c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007eff05859be0 CR3: 0000000138126000 CR4: 00000000000206f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_ct_remove_expectations
|
nf_conntrack |
0xffffffffc06ea281
Build id: 3.10.0-1160.15.2.el7.x86_64 |
0x11 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [watchdog/0:11] Modules linked in: xt_multiport xt_owner tcp_diag inet_diag xt_time ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ext4 mbcache jbd2 loop vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev parport_pc joydev pcspkr sg vmw_balloon parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm ahci crc32c_intel ata_piix libahci nfit serio_raw libata libnvdimm vmxnet3 drm_panel_orientation_quirks vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 11 Comm: watchdog/0 Kdump: loaded Tainted: G L ------------ 3.10.0-1160.15.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018 task: ffff8e277a743180 ti: ffff8e277a750000 task.ti: ffff8e277a750000 RIP: 0010:[<ffffffffc06ea281>] [<ffffffffc06ea281>] nf_ct_remove_expectations+0x11/0x90 [nf_conntrack] RSP: 0000:ffff8e277fc03d88 EFLAGS: 00010282 RAX: ffff8e2658f03a80 RBX: ffff8e277fc03da8 RCX: 00000000000005c1 RDX: 0000000000000200 RSI: 0000000000000006 RDI: ffff8e26759523c0 RBP: ffff8e277fc03da0 R08: ffff8e27681de100 R09: 000000018010000e R10: 00000000681de601 R11: ffff8e27681de100 R12: ffff8e277fc03cf8 R13: ffffffff82b95fba R14: ffff8e277fc03da0 R15: ffff8e26759523c0 FS: 0000000000000000(0000) GS:ffff8e277fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000001cfe060 CR3: 00000000b73ac000 CR4: 00000000003607f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_ct_remove_expectations
|
nf_conntrack |
0xffffffffc051de45
Build id: 3.10.0-693.el7.x86_64 |
0x15 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:0:85898] Modules linked in: tcp_lp fuse xt_CHECKSUM ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter xt_conntrack iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables devlink veth br_netfilter overlay(T) tun ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit vmw_balloon pcspkr joydev sg libnvdimm parport_pc parport vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx mptspi scsi_transport_spi mptscsih drm_kms_helper mptbase crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect ata_piix sysimgblt fb_sys_fops ttm libata drm serio_raw e1000 i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables] CPU: 0 PID: 85898 Comm: kworker/0:0 Tainted: G W L ------------ T 3.10.0-693.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017 Workqueue: events vmstat_shepherd task: ffff8800162aeeb0 ti: ffff8800063fc000 task.ti: ffff8800063fc000 RIP: 0010:[<ffffffffc051de45>] [<ffffffffc051de45>] nf_ct_remove_expectations+0x15/0x90 [nf_conntrack] RSP: 0018:ffff880139603d48 EFLAGS: 00000286 RAX: ffff880102a04900 RBX: ffff880139603cc8 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000002408 RDI: ffff88001ccf70c0 RBP: ffff880139603d90 R08: 00000000dae7ccdd R09: 000000005acf70d7 R10: ffff88001ccf7130 R11: ffffea0000733d00 R12: ffff880139603cb8 R13: ffffffff816b5c1d R14: ffff880139603d90 R15: ffffffff81ad9d40 FS: 0000000000000000(0000) GS:ffff880139600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000b8 CR3: 00000000019f2000 CR4: 00000000003407f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffffffffc0519b55 ffff88001ccf7118 ffff88001ccf70e0 0000240800000002 ffff88001ccf70c0 ffff880102a04930 0000000000000000 0000000000000000 0000000000000000 ffff880139603df0 ffffffffc051a046 ffff880139603eb0 Call Trace: