| Operating system | Count |
|---|---|
| CentOS 7 | 127 |
| Architecture | Count |
|---|---|
| x86_64 | 127 |
| Related packages | Count |
|---|---|
| kernel | 126 |
| kernel-lt | 1 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__inet_lookup_established
|
vmlinux |
0xffffffffa428d676
Build id: 3.10.0-957.10.1.el7.x86_64 |
0x56 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [dbmsrv:26708] Modules linked in: btrfs raid6_pq xor xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ext4 aesni_intel ppdev lrw gf128mul glue_helper ablk_helper mbcache cryptd jbd2 vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw nfit libnvdimm vmw_pvscsi vmwgfx drm_kms_helper floppy ata_generic pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 26708 Comm: dbmsrv Kdump: loaded Not tainted 3.10.0-957.10.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018 task: ffff9288dde25140 ti: ffff928978fe0000 task.ti: ffff928978fe0000 RIP: 0010:[<ffffffffa428d676>] [<ffffffffa428d676>] __inet_lookup_established+0x56/0x130 RSP: 0000:ffff9289ffc03be0 EFLAGS: 00010206 RAX: ffffa66400648000 RBX: 0000000000000001 RCX: 00000000e8bb2f73 RDX: 00000000000009dc RSI: 0000000018bd0f4e RDI: 00000000e2d66456 RBP: ffff9289ffc03c10 R08: 000000000000684b R09: 00000000000009dc R10: ffffffffa4911680 R11: 0000000000000001 R12: ffff9289ffc03b58 R13: ffffffffa4376df2 R14: ffff9289ffc03c10 R15: 00000000713de84b FS: 00007f24bbe40740(0000) GS:ffff9289ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f885f46e000 CR3: 00000000013b6000 CR4: 00000000000607f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__inet_lookup_established
|
vmlinux |
0xffffffff836a4838
Build id: 3.10.0-1062.18.1.el7.x86_64 |
0x68 |
NMI watchdog: BUG: soft lockup - CPU#18 stuck for 22s! [java:16521] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13501(E) falcon_lsm_pinned_13109(E) falcon_lsm_pinned_13005(E) falcon_lsm_pinned_12905(E) binfmt_misc cmac arc4 md4 nls_utf8 cifs ccm dns_resolver ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment xt_mark iptable_filter iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_netlink nfnetlink iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi overlay(T) vmw_vsock_vmci_transport vsock falcon_lsm_pinned_12803(E) ext4 sb_edac mbcache iosf_mbi jbd2 crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper pcc_cpufreq cryptd ppdev vmw_balloon pcspkr joydev parport_pc parport vmw_vmci sg i2c_piix4 br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi nfit crct10dif_pclmul libata crct10dif_common crc32c_intel mptscsih libnvdimm serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal] CPU: 18 PID: 16521 Comm: java Kdump: loaded Tainted: P EL ------------ T 3.10.0-1062.18.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018 task: ffff89108148a0e0 ti: ffff8907fe68c000 task.ti: ffff8907fe68c000 RIP: 0010:[<ffffffff836a4838>] [<ffffffff836a4838>] __inet_lookup_established+0x68/0x130 RSP: 0018:ffff89133f283bf8 EFLAGS: 00010246 RAX: ffff96b00c50d000 RBX: 0000000000000000 RCX: 000000005c8e5c5d RDX: ffff88f580fd6cd8 RSI: 0000000074f0e21a RDI: 000000004ee44442 RBP: ffff89133f283c28 R08: 000000000001bcc1 R09: 000000000000094c R10: ffff89033e0410b8 R11: ffffffffc04b35f8 R12: ffff89133f283b68 R13: ffffffff8378eefa R14: ffff89133f283c28 R15: 000000006429bcc1 FS: 00007fbad611b700(0000) GS:ffff89133f280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f445c00ecc8 CR3: 0000000a20d7e000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__inet_lookup_established
|
vmlinux |
0xffffffffb84a27f9
Build id: 3.10.0-1062.4.1.el7.x86_64 |
0x49 |
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [dbus-daemon-lau:31875] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13207(E) falcon_lsm_pinned_13109(E) falcon_lsm_pinned_13005(E) nls_utf8 isofs fuse rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache devlink binfmt_misc falcon_lsm_pinned_12905(E) ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_hda_codec_generic lrw gf128mul glue_helper ablk_helper cryptd snd_hda_intel joydev snd_hda_codec virtio_balloon snd_hda_core sg snd_hwdep snd_seq parport_pc snd_seq_device parport snd_pcm snd_timer snd pcspkr i2c_piix4 soundcore auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel floppy virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal] CPU: 7 PID: 31875 Comm: dbus-daemon-lau Kdump: loaded Tainted: P W EL ------------ 3.10.0-1062.4.1.el7.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 task: ffffa0d508b11070 ti: ffffa0cceaf5c000 task.ti: ffffa0cceaf5c000 RIP: 0010:[<ffffffffb84a27f9>] [<ffffffffb84a27f9>] __inet_lookup_established+0x49/0x130 RSP: 0018:ffffa0d71f1c3c48 EFLAGS: 00010a02 RAX: 000000003eebb3b3 RBX: ffffa0d2586b3e00 RCX: 000000004743cc81 RDX: 0000000000000360 RSI: 00000000c230a7ae RDI: 00000000c5e30173 RBP: ffffa0d71f1c3c78 R08: 000000003eebb3b3 R09: 0000000000000360 R10: ffffa0d71de08280 R11: ffffffffc068e5f8 R12: ffffa0d71f1c3bb8 R13: ffffffffb858cefa R14: ffffa0d71f1c3c78 R15: ffffa0d2586b3e00 FS: 00007f568e0c3880(0000) GS:ffffa0d71f1c0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000f8816007 CR3: 000000252af64000 CR4: 00000000001606e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__inet_lookup_established
|
vmlinux |
0xffffffff83ca8136
Build id: 3.10.0-1127.13.1.el7.x86_64 |
0x56 |
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [mysqld:19543] Firewall: *ICMP_IN Blocked* IN=ens32 OUT= MAC=00:50:56:0c:ea:3d:00:ff:ff:ff:ff:fe:08:00 SRC=92.222.185.1 DST=192.95.49.172 LEN=32 TOS=0x08 PREC=0x00 TTL=9 ID=1 DF PROTO=ICMP TYPE=8 CODE=0 ID=45733 SEQ=1 NMI watchdog: BUG: soft lockup - CPU#4 stuck for 25s! [kworker/4:3:19528] Modules linked in: tcp_diag inet_diag talpa_vfshook(OE) talpa_pedconnector(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 ip6table_filter nf_log_ipv4 nf_log_common nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_conntrack nf_conntrack_ftp xt_LOG xt_limit xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 iptable_filter talpa_pedevice(OE) sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev sg pcspkr i2c_piix4 vmw_vmci parport_pc parport tcp_htcp ip_tables xfs libcrc32c vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sr_mod cdrom sd_mod ata_generic crc_t10dif pata_acpi crct10dif_generic drm ata_piix crct10dif_pclmul crct10dif_common e1000 serio_raw crc32c_intel libata drm_panel_orientation_quirks vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: talpa_syscallhook] CPU: 4 PID: 19528 Comm: kworker/4:3 Kdump: loaded Tainted: G W OEL ------------ 3.10.0-1127.13.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014 Workqueue: events vmw_fb_dirty_flush [vmwgfx] task: ffff9397eb950000 ti: ffff93955e708000 task.ti: ffff93955e708000 RIP: 0010:[<ffffffff83ca8136>] [<ffffffff83ca8136>] __inet_lookup_established+0x56/0x130 RSP: 0018:ffff9397ffd03b58 EFLAGS: 00010202 RAX: ffffaa98c18d6000 RBX: 0000000000000002 RCX: 0000000079e985b0 RDX: 00000000456e908f RSI: 0000000034d6eb86 RDI: ffffffff84315c40 RBP: ffff9397ffd03b88 R08: 000000000001d3c1 R09: 0000000000000050 R10: ffff9397ffd039a0 R11: ffffffffc04a55f8 R12: ffff9397ffd03ac8 R13: ffffffff83d93efa R14: ffff9397ffd03b88 R15: 00000000a3e9d3c1 FS: 0000000000000000(0000) GS:ffff9397ffd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f868bfdb470 CR3: 000000019f774000 CR4: 00000000000407e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__inet_lookup_established
|
vmlinux |
0xffffffff815d3c96
Build id: 3.10.0-693.el7.x86_64 |
0x56 |
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [ftdc:32020] NMI watchdog: BUG: soft lockup - CPU#11 stuck for 22s! [Network.IO-RS-0:32034] Modules linked in: vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nfit vmw_balloon ppdev joydev sg libnvdimm vmw_vmci shpchp i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif vmwgfx crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul libata crct10dif_common crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 11 PID: 32034 Comm: Network.IO-RS-0 Tainted: G OEL ------------ T 3.10.0-693.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 task: ffff8805bc336eb0 ti: ffff880be205c000 task.ti: ffff880be205c000 RIP: 0010:[<ffffffff815d3c96>] [<ffffffff815d3c96>] __inet_lookup_established+0x56/0x130 RSP: 0018:ffff88103fd43bd0 EFLAGS: 00010202 RAX: ffffc90006280000 RBX: ffff88103fd43ba0 RCX: 00000000166c261e RDX: 0000000000006989 RSI: 0000000012386649 RDI: 0000000061b16851 RBP: ffff88103fd43c00 R08: 0000000000035029 R09: 0000000000006989 R10: 0000000000000001 R11: 0000000000000001 R12: ffff88103fd43b48 R13: ffffffff816b5c1d R14: ffff88103fd43c00 R15: 00000000f72b5029 FS: 00007f433ed16700(0000) GS:ffff88103fd40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00005599b11c9c00 CR3: 00000007cf76e000 CR4: 00000000000407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Stack: ffffffffc068f0a8 ffff88101f251200 ffff880b5659444e ffff88081d094000 ffff88081d094000 ffffffff81b257c0 ffff88103fd43c28 ffffffff815f005e ffff880800000002 ffff88101f251200 ffff880b5659444e ffff88103fd43c50 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
__inet_lookup_established
|
vmlinux |
0xffffffffad83b0da
Build id: 3.10.0-862.el7.x86_64 |
0x9a |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 33s! [java:27030] Modules linked in: xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 ipt_MASQUERADE tun nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 ebtable_filter nf_defrag_ipv4 ebtables nf_nat_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntrack ip6table_filter ip6_tables br_netfilter iptable_filter bridge stp llc devlink vmw_vsock_vmci_transport vsock overlay(T) sunrpc coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev pcspkr vmw_vmci sg i2c_piix4 floppy parport_pc parport shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix scsi_transport_spi drm crct10dif_pclmul crct10dif_common libata mptscsih crc32c_intel e1000 serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod CPU: 1 PID: 27030 Comm: java Kdump: loaded Tainted: G W L ------------ T 3.10.0-862.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff8d7e8be43f40 ti: ffff8d7f6dfb0000 task.ti: ffff8d7f6dfb0000 RIP: 0010:[<ffffffffad83b0da>] [<ffffffffad83b0da>] __inet_lookup_established+0x9a/0x130 RSP: 0000:ffff8d861ee43be0 EFLAGS: 00010202 RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000001 RDX: ffff8d85fefe7758 RSI: 0000000000000002 RDI: ffff8d85fefe776c RBP: ffff8d861ee43c10 R08: 000000000001475f R09: 0000000000009bd0 R10: ffffffffadefc900 R11: 0000000000000001 R12: ffff8d861ee43b58 R13: ffffffffad9207f2 R14: ffff8d861ee43c10 R15: 00000000cf1d475f FS: 00007fc55a317700(0000) GS:ffff8d861ee40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000559c36d51948 CR3: 00000004aa324000 CR4: 00000000000007e0 Call Trace: