| Operating system | Count |
|---|---|
| CentOS 7 | 73 |
| Architecture | Count |
|---|---|
| x86_64 | 73 |
| Related packages | Count |
|---|---|
| kernel | 73 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
ip_finish_output
|
vmlinux |
0xffffffffbb4a5bc1
Build id: 3.10.0-1160.15.2.el7.x86_64 |
0x1c1 |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [tuned:1453] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:2:14142] Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13402(E) vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr vmw_vmci i2c_piix4 parport_pc parport auth_rpcgss sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic sg vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix serio_raw libata vmw_pvscsi vmxnet3 nfit libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 14142 Comm: kworker/0:2 Kdump: loaded Tainted: P EL ------------ 3.10.0-1160.15.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 Workqueue: events vmw_fb_dirty_flush [vmwgfx] task: ffff8caba1fa4200 ti: ffff8cabae340000 task.ti: ffff8cabae340000 RIP: 0010:[<ffffffffbb4a5bc1>] [<ffffffffbb4a5bc1>] ip_finish_output+0x1c1/0x8d0 RSP: 0018:ffff8cabbfc03c70 EFLAGS: 00010282 RAX: 00000000c0406468 RBX: 0000000000000000 RCX: 0000000000000020 RDX: 00000000c701a8c0 RSI: ffff8caba1f32e00 RDI: ffff8ca83b9aaa00 RBP: ffff8cabbfc03cb8 R08: 0000000000000000 R09: ffffffffbb110c90 R10: 00000000000163ec R11: 000000000002cd80 R12: ffff8cabbfc03be8 R13: ffffffffbb595fba R14: ffff8cabbfc03cb8 R15: ffff8ca8f47aef00 FS: 0000000000000000(0000) GS:ffff8cabbfc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fec19bf5000 CR3: 00000000b951a000 CR4: 00000000000607f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
ip_finish_output
|
vmlinux |
0xffffffffb0c380b9
Build id: 3.10.0-862.3.2.el7.x86_64 |
0x1e9 |
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 50s! [watchdog/3:22] Modules linked in: veeamsnap(OE) arc4 md4 nls_utf8 cifs ccm dns_resolver xt_CHECKSUM iptable_mangle nf_conntrack_netlink nfnetlink ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat tun nf_conntrack br_netfilter bridge stp llc ebtable_filter ebtables dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop ip6table_filter ip6_tables iptable_filter devlink overlay ext4 mbcache jbd2 sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg joydev vmw_balloon pcspkr i2c_piix4 shpchp parport_pc vmw_vmci parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi libata scsi_transport_spi mptscsih crct10dif_pclmul mptbase crct10dif_common i2c_core crc32c_intel serio_raw vmxnet3 floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 3 PID: 22 Comm: watchdog/3 Kdump: loaded Tainted: G OEL ------------ 3.10.0-862.3.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff8ec93cfc0000 ti: ffff8ec93cfc8000 task.ti: ffff8ec93cfc8000 RIP: 0010:[<ffffffffb0c380b9>] [<ffffffffb0c380b9>] ip_finish_output+0x1e9/0x7a0 RSP: 0000:ffff8ec9ffd837f0 EFLAGS: 00010286 RAX: 000000008b6d6609 RBX: 00000000c50b9ba8 RCX: 0000000000000020 RDX: 000000008764a8c0 RSI: ffff8ec878b9b080 RDI: ffff8ec878b03500 RBP: ffff8ec9ffd83830 R08: 0000000000000000 R09: ffffffffc07ca280 R10: ffff8ec7f567fbe0 R11: ffff8ec9f3f64000 R12: ffff8ec9ffd83768 R13: ffffffffb0d218dc R14: ffff8ec9ffd83830 R15: ffff8ec9df30fe00 FS: 0000000000000000(0000) GS:ffff8ec9ffd80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f27bceae000 CR3: 00000000b3f70000 CR4: 00000000000607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
ip_finish_output
|
vmlinux |
0xffffffff815cfaa7
Build id: 3.10.0-693.el7.x86_64 |
0x297 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/0:0:85898] Modules linked in: tcp_lp fuse xt_CHECKSUM ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter xt_conntrack iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables devlink veth br_netfilter overlay(T) tun ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit vmw_balloon pcspkr joydev sg libnvdimm parport_pc parport vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx mptspi scsi_transport_spi mptscsih drm_kms_helper mptbase crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect ata_piix sysimgblt fb_sys_fops ttm libata drm serio_raw e1000 i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables] CPU: 0 PID: 85898 Comm: kworker/0:0 Tainted: G W L ------------ T 3.10.0-693.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017 Workqueue: events e1000_watchdog [e1000] task: ffff8800162aeeb0 ti: ffff8800063fc000 task.ti: ffff8800063fc000 RIP: 0010:[<ffffffff815cfaa7>] [<ffffffff815cfaa7>] ip_finish_output+0x297/0x780 RSP: 0018:ffff880139603820 EFLAGS: 00000286 RAX: 0000000000000000 RBX: ffff88002a61de00 RCX: 0000000000000000 RDX: ffff8800063fffd8 RSI: 0000000000000200 RDI: ffffffff815895a5 RBP: ffff880139603860 R08: 0000000000000100 R09: 0000000002000000 R10: 0000000080000000 R11: 0000000000000000 R12: ffff880139603798 R13: ffffffff816b5c1d R14: ffff880139603860 R15: ffff880014f7a900 FS: 0000000000000000(0000) GS:ffff880139600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000000000b8 CR3: 00000000019f2000 CR4: 00000000003407f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffff880136a16000 01e0c70ac0552040 00000000c4ff6299 ffff880014f7a900 ffff88013690cc00 ffff88013690cc00 ffff880139603a28 ffff8800b8bc1624 ffff8801396038c0 ffffffff815d0293 0000000000000038 8000000000000004 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
ip_finish_output
|
vmlinux |
0xffffffffaf2a3df1
Build id: 3.10.0-1127.18.2.el7.x86_64 |
0x1d1 |
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [kswapd0:129] Modules linked in: xt_multiport xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper sg ppdev ablk_helper cryptd vmw_balloon vmw_vmci pcspkr joydev parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi ata_piix drm mptscsih crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 6 PID: 129 Comm: kswapd0 Kdump: loaded Tainted: G L ------------ 3.10.0-1127.18.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013 task: ffff891279f59070 ti: ffff89152d510000 task.ti: ffff89152d510000 RIP: 0010:[<ffffffffaf2a3df1>] [<ffffffffaf2a3df1>] ip_finish_output+0x1d1/0x8d0 RSP: 0018:ffff89152fb03c88 EFLAGS: 00010a03 RAX: ffff891911b013c0 RBX: ffff89152fb03ce0 RCX: ffff891911b01380 RDX: 000000006101c80a RSI: ffff891925708d80 RDI: ffff89192d25c900 RBP: ffff89152fb03cd0 R08: 0000000000000000 R09: ffffffffc058d280 R10: ffff891524c62808 R11: ffff89192c5d8000 R12: ffff89152fb03bf8 R13: ffffffffaf393efa R14: ffff89152fb03cd0 R15: ffff891598155300 FS: 0000000000000000(0000) GS:ffff89152fb00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f07cab2fbd0 CR3: 000000082b480000 CR4: 00000000000407e0 Call Trace: