| Operating system | Count |
|---|---|
| CentOS 7 | 205 |
| Architecture | Count |
|---|---|
| x86_64 | 205 |
| Related packages | Count |
|---|---|
| kernel | 205 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_nat_cleanup_conntrack
|
nf_nat |
0xffffffffc084d27b
Build id: 3.10.0-862.9.1.el7.x86_64 |
0x5b |
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [swapper/7:0] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_15508(E) falcon_lsm_pinned_15402(E) falcon_lsm_pinned_15309(E) falcon_lsm_pinned_15110(E) falcon_lsm_pinned_15003(E) falcon_lsm_pinned_14812(E) falcon_lsm_pinned_14810(E) falcon_lsm_pinned_14713(E) falcon_lsm_pinned_14712(E) falcon_lsm_pinned_14604(E) falcon_lsm_pinned_14504(E) falcon_lsm_pinned_14306(E) falcon_lsm_pinned_14203(E) falcon_lsm_pinned_14006(E) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink falcon_lsm_pinned_13804(E) vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev i2c_piix4 vmw_vmci sg nfit shpchp libnvdimm parport_pc parport nfsd nfs_acl lockd auth_rpcgss grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal] CPU: 7 PID: 0 Comm: swapper/7 Tainted: P W EL ------------ 3.10.0-862.9.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018 task: ffff9eb03a250fd0 ti: ffff9eb03a25c000 task.ti: ffff9eb03a25c000 RIP: 0010:[<ffffffffc084d27b>] [<ffffffffc084d27b>] nf_nat_cleanup_conntrack+0x5b/0x70 [nf_nat] RSP: 0018:ffff9eb2e1dc3d70 EFLAGS: 00010246 RAX: dead000000000200 RBX: 000000010001fdf8 RCX: ffffd2913fdc1f50 RDX: ffff9eb2cae74888 RSI: 0000000000000200 RDI: ffffffffc0851478 RBP: ffff9eb2e1dc3d78 R08: 000000005e54a005 R09: 000000007d6e5236 R10: ffff9eb2a17af0b0 R11: ffffd5b74aa9ff00 R12: ffff9eb2e1dc3ce8 R13: ffffffff9fd217b2 R14: ffff9eb2e1dc3d78 R15: ffff9eb15ec742d8 FS: 0000000000000000(0000) GS:ffff9eb2e1dc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f8c52aad278 CR3: 00000000b9fa4000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_nat_cleanup_conntrack
|
nf_nat |
0xffffffffc076b27b
Build id: 3.10.0-957.1.3.el7.x86_64 |
0x5b |
NMI watchdog: BUG: soft lockup - CPU#52 stuck for 21s! [swapper/52:0] Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter dgrp(OE) vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev pcspkr sg i2c_piix4 nfit parport_pc parport libnvdimm vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ahci fb_sys_fops ttm libahci mptspi drm ata_piix scsi_transport_spi e1000 crct10dif_pclmul mptscsih crct10dif_common libata serio_raw crc32c_intel mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 52 PID: 0 Comm: swapper/52 Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.1.3.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018 task: ffff8ff6d0c7b0c0 ti: ffff8ff6d0c8c000 task.ti: ffff8ff6d0c8c000 RIP: 0010:[<ffffffffc076b27b>] [<ffffffffc076b27b>] nf_nat_cleanup_conntrack+0x5b/0x70 [nf_nat] RSP: 0018:ffff9000d4a03d70 EFLAGS: 00010246 RAX: dead000000000200 RBX: 0000000100010001 RCX: 0000000000000001 RDX: ffff8fecdf848f98 RSI: 0000000000000001 RDI: ffffffffc076f478 RBP: ffff9000d4a03d78 R08: 00000000a81933d0 R09: 00000000de98f1b0 R10: ffff8fe1c6292af0 R11: ffffddc984da8d00 R12: ffff9000d4a03ce8 R13: ffffffffb3b75df2 R14: ffff9000d4a03d78 R15: ffff8fefd8da6458 FS: 0000000000000000(0000) GS:ffff9000d4a00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fec4cc6b140 CR3: 0000001dcfb24000 CR4: 00000000000607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_nat_cleanup_conntrack
|
nf_nat |
0xffffffffa05ad27b
Build id: 3.10.0-514.26.2.el7.x86_64 |
0x5b |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [kworker/0:0:11224] Modules linked in: binfmt_misc fuse nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay ext4 mbcache jbd2 sg hv_utils intel_powerclamp crc32_pclmul ghash_clmulni_intel i2c_piix4 aesni_intel lrw i2c_core gf128mul glue_helper ablk_helper cryptd pcspkr involflt(POE) nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic hv_netvsc hyperv_keyboard hid_hyperv hv_storvsc crct10dif_pclmul crct10dif_common ata_piix crc32c_intel hyperv_fb serio_raw libata hv_vmbus fjes floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 11224 Comm: kworker/0:0 Tainted: P OEL ------------ 3.10.0-514.26.2.el7.x86_64 #1 Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090006 05/23/2012 task: ffff8801ea275e20 ti: ffff8800ed2d0000 task.ti: ffff8800ed2d0000 RIP: 0010:[<ffffffffa05ad27b>] [<ffffffffa05ad27b>] nf_nat_cleanup_conntrack+0x5b/0x70 [nf_nat] RSP: 0018:ffff8801fe603d30 EFLAGS: 00010246 RAX: dead000000000200 RBX: 0000000100019bf8 RCX: ffffe8fffee07c00 RDX: ffff8801fd440b18 RSI: 0000000000000006 RDI: ffffffffa05b1478 RBP: ffff8801fe603d38 R08: 000000003de0c5f6 R09: 00000000459cd778 R10: ffff8800073f3db0 R11: ffffea0001c4dd00 R12: ffff8801fe603ca8 R13: ffffffff816994ad R14: ffff8801fe603d38 R15: ffff8800a446b358 FS: 0000000000000000(0000) GS:ffff8801fe600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000b8 CR3: 00000001e4267000 CR4: 00000000001406f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Stack: 0000000000000001 ffff8801fe603d60 ffffffffa058ae95 ffff8800073f3d40 ffffffff81aa0e80 0000000000000000 ffff8801fe603d80 ffffffffa0581315 ffff8800073f3d40 ffffffff81aa0e80 ffff8801fe603da0 ffffffffa05829b0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_nat_cleanup_conntrack
|
nf_nat |
0xffffffffc07e8261
Build id: 3.10.0-957.27.2.el7.x86_64 |
0x41 |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mysqld:9494] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [httpd:7091] Modules linked in: ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 ip6table_filter nf_log_ipv4 nf_log_common nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_conntrack nf_conntrack_ftp xt_LOG xt_limit xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 iptable_filter sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon pcspkr joydev parport_pc sg parport i2c_piix4 vmw_vmci tcp_htcp ip_tables xfs libcrc32c vmwgfx drm_kms_helper sd_mod sr_mod syscopyarea cdrom sysfillrect crc_t10dif sysimgblt fb_sys_fops crct10dif_generic ata_generic pata_acpi crct10dif_pclmul ttm ata_piix crct10dif_common drm crc32c_intel serio_raw e1000 vmw_pvscsi libata drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: talpa_syscallhook] CPU: 0 PID: 7091 Comm: httpd Kdump: loaded Tainted: G W OEL ------------ 3.10.0-957.27.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014 task: ffffa0fd57ec5140 ti: ffffa0fc87fe8000 task.ti: ffffa0fc87fe8000 RIP: 0010:[<ffffffffc07e8261>] [<ffffffffc07e8261>] nf_nat_cleanup_conntrack+0x41/0x70 [nf_nat] RSP: 0018:ffffa0fd7fc03dd8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffffa0fc81273ed8 RCX: ffffd828bfc024d8 RDX: ffffa0fd59a4e8a8 RSI: 0000000000000200 RDI: ffffffffc07ec478 RBP: ffffa0fd7fc03de0 R08: 0000000096eb10b1 R09: 000000007295dbcb R10: ffffa0fcd4fe66b0 R11: 0000000000000000 R12: ffffa0fd7fc03d48 R13: ffffffffa4377df2 R14: ffffa0fd7fc03de0 R15: ffffa0fd5ffd0a58 FS: 00007f6501732880(0000) GS:ffffa0fd7fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f64eb4be000 CR3: 00000002c3f78000 CR4: 00000000000407f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_nat_cleanup_conntrack
|
nf_nat |
0xffffffffc093d261
Build id: 3.10.0-862.6.3.el7.x86_64 |
0x41 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [mysqld:9955] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr joydev vmw_balloon sg parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common mptspi crc32c_intel scsi_transport_spi mptscsih e1000 serio_raw mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 9955 Comm: mysqld Kdump: loaded Tainted: G W L ------------ 3.10.0-862.6.3.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015 task: ffff8fcf378f1fa0 ti: ffff8fd17e868000 task.ti: ffff8fd17e868000 RIP: 0010:[<ffffffffc093d261>] [<ffffffffc093d261>] nf_nat_cleanup_conntrack+0x41/0x70 [nf_nat] RSP: 0018:ffff8fcf38c03dd8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000100018b40 RCX: ffffc1f1f8c057b8 RDX: ffff8fcfa2348140 RSI: 0000000000000200 RDI: ffffffffc0941478 RBP: ffff8fcf38c03de0 R08: 000000005e352fae R09: 0000000026d827a7 R10: ffff8fd0899e4070 R11: ffffd9a942e64100 R12: ffff8fcf38c03d48 R13: ffffffff9af217b2 R14: ffff8fcf38c03de0 R15: ffff8fcf0d874518 FS: 00007fd599232700(0000) GS:ffff8fcf38c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fd598a1a9c8 CR3: 000000000f260000 CR4: 00000000001607f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
nf_nat_cleanup_conntrack
|
nf_nat |
0xffffffffa0574261
Build id: 3.10.0-514.el7.x86_64 |
0x41 |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [rpm:10789] Modules linked in: fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic iosf_mbi crc32_pclmul snd_hda_intel ghash_clmulni_intel aesni_intel snd_hda_codec lrw gf128mul glue_helper ablk_helper snd_hda_core snd_hwdep snd_seq ppdev snd_seq_device cryptd snd_pcm snd_timer snd pcspkr parport_pc virtio_balloon parport sg i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect crc32c_intel sysimgblt ata_piix fb_sys_fops serio_raw ttm drm virtio_pci virtio_ring virtio libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 1 PID: 10789 Comm: rpm Tainted: G L ------------ 3.10.0-514.el7.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 task: ffff88019b228fb0 ti: ffff880002904000 task.ti: ffff880002904000 RIP: 0010:[<ffffffffa0574261>] [<ffffffffa0574261>] nf_nat_cleanup_conntrack+0x41/0x70 [nf_nat] RSP: 0018:ffff880239c83d30 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff8800b8e0b2d8 RCX: ffffe8ffffc82d50 RDX: ffff880220a43ff0 RSI: 00000000c126c124 RDI: ffffffffa0578478 RBP: ffff880239c83d38 R08: 00000000096ff4b8 R09: 0000000031ce759a R10: ffff8801c864e070 R11: ffffea0007219380 R12: ffff880239c83ca8 R13: ffffffff8169721d R14: ffff880239c83d38 R15: ffff8800b8e0b2d8 FS: 00007f31bafd4840(0000) GS:ffff880239c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000cfc94001 CR3: 00000000a77d9000 CR4: 00000000000406e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Stack: 0000000000000001 ffff880239c83d60 ffffffffa0549e95 ffff8801c864e000 ffffffff81a9cd80 0000000000000000 ffff880239c83d80 ffffffffa0540315 ffff8801c864e000 ffffffff81a9cd80 ffff880239c83da0 ffffffffa05419b0 Call Trace: