| Operating system | Count |
|---|---|
| CentOS 7 | 33 |
| Architecture | Count |
|---|---|
| x86_64 | 33 |
| Related packages | Count |
|---|---|
| kernel | 33 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__ip_route_output_key_hash
|
vmlinux |
0xffffffff94ec8c81
Build id: 3.10.0-1160.80.1.el7.x86_64 |
0x241 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [Wrapper-Control:2251] Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vboxsf(OE) sunrpc snd_intel8x0 nfit snd_ac97_codec ac97_bus libnvdimm snd_seq joydev snd_seq_device ppdev snd_pcm iosf_mbi vboxguest(OE) sg pcspkr snd_timer parport_pc video parport snd soundcore i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix libahci e1000 libata crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 2251 Comm: Wrapper-Control Kdump: loaded Tainted: G W OE ------------ 3.10.0-1160.80.1.el7.x86_64 #1 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 task: ffff8e37efe38000 ti: ffff8e37efe34000 task.ti: ffff8e37efe34000 RIP: 0010:[<ffffffff94ec8c81>] [<ffffffff94ec8c81>] __ip_route_output_key_hash+0x241/0x7f0 RSP: 0000:ffff8e381fc03ca0 EFLAGS: 00010282 RAX: ffffd360bfc000d0 RBX: 0000000000000010 RCX: ffff8e34d807fa00 RDX: 0000000000000000 RSI: 00000000e315940a RDI: ffff8e34d807fa68 RBP: ffff8e381fc03d10 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000001 R11: 0000000000000000 R12: ffff8e381fc03c18 R13: ffffffff94fc63f2 R14: ffff8e381fc03d10 R15: ffff8e380c602ca8 FS: 00007f7cc6d7c700(0000) GS:ffff8e381fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f7cb119ffe8 CR3: 000000040ce3e000 CR4: 00000000000206f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__ip_route_output_key_hash
|
vmlinux |
0xffffffffb8495132
Build id: 3.10.0-1062.4.1.el7.x86_64 |
0xe2 |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [gsd-color:3065] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13207(E) falcon_lsm_pinned_13109(E) falcon_lsm_pinned_13005(E) nls_utf8 isofs fuse rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache devlink binfmt_misc falcon_lsm_pinned_12905(E) ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_hda_codec_generic lrw gf128mul glue_helper ablk_helper cryptd snd_hda_intel joydev snd_hda_codec virtio_balloon snd_hda_core sg snd_hwdep snd_seq parport_pc snd_seq_device parport snd_pcm snd_timer snd pcspkr i2c_piix4 soundcore auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel floppy virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal] CPU: 1 PID: 3065 Comm: gsd-color Kdump: loaded Tainted: P W EL ------------ 3.10.0-1062.4.1.el7.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 task: ffffa0a8f6d120e0 ti: ffffa0d6593a8000 task.ti: ffffa0d6593a8000 RIP: 0010:[<ffffffffb8495132>] [<ffffffffb8495132>] __ip_route_output_key_hash+0xe2/0x7b0 RSP: 0000:ffffa0d71f043ca0 EFLAGS: 00010246 RAX: ffffa0d71965b000 RBX: ffffa0d7171eefe8 RCX: ffffa0d718b73780 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffffa0d71f043d10 R08: ffffffffb8b12940 R09: 000000008502700a R10: 0000000000000006 R11: 0000000000000000 R12: ffffa0d71f043c18 R13: ffffffffb858cefa R14: ffffa0d71f043d10 R15: ffffa0d7171eefe8 FS: 00007f7f8e1e1a40(0000) GS:ffffa0d71f040000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000fac9a001 CR3: 0000002f186b6000 CR4: 00000000001606e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__ip_route_output_key_hash
|
vmlinux |
0xffffffff9249bcff
Build id: 3.10.0-1160.6.1.el7.x86_64 |
0x24f |
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 25s! [vmtoolsd:4199] Modules linked in: binfmt_misc veth tcp_lp nls_utf8 isofs nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd pcspkr joydev sg vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ahci ata_piix mptspi crc32c_intel libahci scsi_transport_spi mptscsih serio_raw libata mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 4 PID: 4199 Comm: vmtoolsd Kdump: loaded Tainted: G L ------------ T 3.10.0-1160.6.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff998c1d3ca100 ti: ffff998c1dfc0000 task.ti: ffff998c1dfc0000 RIP: 0010:[<ffffffff9249bcff>] [<ffffffff9249bcff>] __ip_route_output_key_hash+0x24f/0x7f0 RSP: 0018:ffff99aa3d703728 EFLAGS: 00010286 RAX: ffff99a93a453800 RBX: 0000000000000283 RCX: ffff998c0abdff00 RDX: 0000000000000000 RSI: 000000002401a8c0 RDI: ffff998c0abdff68 RBP: ffff99aa3d703798 R08: 0000000000000000 R09: 0000000000000001 R10: 0000000000000001 R11: 0000000000000000 R12: ffff99aa3d703698 R13: ffffffff92595fba R14: ffff99aa3d703798 R15: ffff99aa3d703900 FS: 00007fb1b688b780(0000) GS:ffff99aa3d700000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f6c1376cbd0 CR3: 000000019dfa8000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__ip_route_output_key_hash
|
vmlinux |
0xffffffff815c6c15
Build id: 3.10.0-693.el7.x86_64 |
0x565 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:0:85898] Modules linked in: tcp_lp fuse xt_CHECKSUM ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter xt_conntrack iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables devlink veth br_netfilter overlay(T) tun ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit vmw_balloon pcspkr joydev sg libnvdimm parport_pc parport vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx mptspi scsi_transport_spi mptscsih drm_kms_helper mptbase crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect ata_piix sysimgblt fb_sys_fops ttm libata drm serio_raw e1000 i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables] CPU: 0 PID: 85898 Comm: kworker/0:0 Tainted: G W L ------------ T 3.10.0-693.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017 Workqueue: events e1000_watchdog [e1000] task: ffff8800162aeeb0 ti: ffff8800063fc000 task.ti: ffff8800063fc000 RIP: 0010:[<ffffffff815c6c15>] [<ffffffff815c6c15>] __ip_route_output_key_hash+0x565/0x9b0 RSP: 0018:ffff880139603888 EFLAGS: 00000286 RAX: 0000000000000000 RBX: ffffffffffffff5a RCX: 0000000000000010 RDX: 0000000000000000 RSI: ffff8800b2da3b40 RDI: ffff8801396038a8 RBP: ffff880139603938 R08: 000000000ac7ea7b R09: ffff8800ba6a9a00 R10: 000060fec480a410 R11: 0000000000000001 R12: ffff8801396037f8 R13: ffffffff816b5c1d R14: ffff880139603938 R15: ffff8801396039a0 FS: 0000000000000000(0000) GS:ffff880139600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000b8 CR3: 00000000019f2000 CR4: 00000000003407f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: 0000000000000000 ffff880006800000 ffff880139603a28 ffff880139603960 0000000000010010 ffff8800ba6a9a00 ffff8800b2da3b40 ffff880136b79e38 ffffffffc04515f8 ffff88000d8216f0 00000000a82790c4 ffffffff81ad9d40 Call Trace: