| Operating system | Count |
|---|---|
| CentOS 7 | 251 |
| Architecture | Count |
|---|---|
| x86_64 | 251 |
| Related packages | Count |
|---|---|
| kernel | 249 |
| kernel-lt | 2 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
vmxnet3_xmit_frame
|
vmxnet3 |
0xffffffffc01053dc
Build id: 3.10.0-1160.15.2.el7.x86_64 |
0xaec |
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 33s! [sentinel-system:125009] Modules linked in: binfmt_misc resguard_linux(OE) secmodel_linux(OE) syshook_linux(OE) secmodel3(OE) xt_set ip_set_hash_net xt_NFLOG nfnetlink_log ip_set xt_CHECKSUM iptable_raw ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock nf_conntrack_netlink nfnetlink nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev joydev vmw_balloon pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel mptspi libata scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 5 PID: 125009 Comm: sentinel-system Kdump: loaded Tainted: G W OEL ------------ 3.10.0-1160.15.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff8d24b57ec200 ti: ffff8d2e79f4c000 task.ti: ffff8d2e79f4c000 RIP: 0010:[<ffffffffc01053dc>] [<ffffffffc01053dc>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0018:ffff8d347d743a20 EFLAGS: 00000282 RAX: 0000000000000000 RBX: ffffffffc065ffad RCX: 00000000000000b0 RDX: ffff9d74c6288638 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff8d347d743ab0 R08: 0000000135bf6110 R09: 0000000000c84042 R10: 0000000000000001 R11: ffff8d25937b8af0 R12: ffff8d347d743998 R13: ffffffff9f795fba R14: ffff8d347d743ab0 R15: 0000000000000001 FS: 00007fddc1fe3700(0000) GS:ffff8d347d740000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fdf34003000 CR3: 00000009f9faa000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
vmxnet3_xmit_frame
|
vmxnet3 |
0xffffffffc027544c
Build id: 3.10.0-1160.88.1.el7.x86_64 |
0xaec |
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 46s! [TLSLogForwarder:28680] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_15003(E) falcon_lsm_pinned_14812(E) iptable_nat nf_nat_ipv4 nf_nat falcon_lsm_pinned_14713(E) binfmt_misc ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ppdev vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg i2c_piix4 joydev pcspkr parport_pc vmw_vmci parport auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic ata_piix drm nfit libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal] CPU: 5 PID: 28680 Comm: TLSLogForwarder Kdump: loaded Tainted: P EL ------------ 3.10.0-1160.88.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018 task: ffff9a6be8c62100 ti: ffff9a4c41270000 task.ti: ffff9a4c41270000 RIP: 0010:[<ffffffffc027544c>] [<ffffffffc027544c>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0018:ffff9a6c7fd43a20 EFLAGS: 00000282 RAX: 0000000000000000 RBX: ffffffffc07072fd RCX: 00000000000000ad RDX: ffffadea8c476638 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff9a6c7fd43ab0 R08: 0000000d071f7910 R09: 0000000000c84042 R10: 0000000000000001 R11: ffff9a6bfbc64ac0 R12: ffff9a6c7fd43998 R13: ffffffff895c63f2 R14: ffff9a6c7fd43ab0 R15: 0000000000000001 FS: 00007f5ca2b00700(0000) GS:ffff9a6c7fd40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055ef4beec000 CR3: 0000001fbc9f6000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
vmxnet3_xmit_frame
|
vmxnet3 |
0xffffffffc01e4bcc
Build id: 3.10.0-957.el7.x86_64 |
0xaec |
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [runc:12487] Modules linked in: tcp_lp nls_utf8 isofs fuse binfmt_misc nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_set xt_u32 xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ip_set vxlan ip6_udp_tunnel udp_tunnel veth ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_statistic xt_nat xt_recent xt_mark xt_comment nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) dsa_filter(POE) vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 joydev sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci ata_piix crct10dif_pclmul crct10dif_common crc32c_intel nfit serio_raw libata libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 5 PID: 12487 Comm: runc Kdump: loaded Tainted: P W OEL ------------ T 3.10.0-957.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 task: ffff885088ab5140 ti: ffff8859cd820000 task.ti: ffff8859cd820000 RIP: 0010:[<ffffffffc01e4bcc>] [<ffffffffc01e4bcc>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0018:ffff885b3fd434b0 EFLAGS: 00000282 RAX: 0000000000000000 RBX: 657461657243222c RCX: 000000000000015f RDX: ffff989d06256638 RSI: 0000000000000293 RDI: 0000000000000293 RBP: ffff885b3fd43540 R08: 0000c9d8e5001150 R09: 0000000000004068 R10: 5000388910032a0a R11: ffff885afca255e0 R12: ffff885b3fd43428 R13: ffffffffb4f75df2 R14: ffff885b3fd43540 R15: 0000000000000001 FS: 00007f723d43d740(0000) GS:ffff885b3fd40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000056018174a5e0 CR3: 0000000b950f4000 CR4: 00000000007607e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
vmxnet3_xmit_frame
|
vmxnet3 |
0xffffffffc03ce3dc
Build id: 3.10.0-1127.10.1.el7.x86_64 |
0xaec |
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 30s! [postgres:26989] Modules linked in: binfmt_misc nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev vmw_vmci sg pcspkr i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod sysimgblt fb_sys_fops ttm crc_t10dif crct10dif_generic ata_piix drm crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 7 PID: 26989 Comm: postgres Kdump: loaded Tainted: G W L ------------ 3.10.0-1127.10.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff97135b2641c0 ti: ffff9713582d0000 task.ti: ffff9713582d0000 RIP: 0010:[<ffffffffc03ce3dc>] [<ffffffffc03ce3dc>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0000:ffff972b1efc35e0 EFLAGS: 00000286 RAX: 0000000000000000 RBX: 000000221efd0008 RCX: 000000000000013e RDX: ffffaa328c478600 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff972b1efc3670 R08: 0000000000000007 R09: 0000000000c84042 R10: 040200007a201072 R11: ffff972b1b20f3d0 R12: ffff972b1efc3558 R13: ffffffff93993efa R14: ffff972b1efc3670 R15: 0000000000000001 FS: 00007f9615b74880(0000) GS:ffff972b1efc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000005a40015 CR3: 0000000017804000 CR4: 00000000000407e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
vmxnet3_xmit_frame
|
vmxnet3 |
0xffffffffc00554dd
Build id: 3.10.0-693.2.2.el7.x86_64 |
0xb5d |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [chronyd:831] Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun lp ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus ppdev snd_seq snd_seq_device pcspkr snd_pcm snd_timer vmw_balloon snd soundcore joydev nfit sg libnvdimm parport_pc parport i2c_piix4 shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt crc32c_intel fb_sys_fops ttm drm serio_raw ata_piix mptspi scsi_transport_spi mptscsih libata mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod CPU: 1 PID: 831 Comm: chronyd Tainted: G OEL ------------ 3.10.0-693.2.2.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020 task: ffff880034862f70 ti: ffff880034bc4000 task.ti: ffff880034bc4000 RIP: 0010:[<ffffffffc00554dd>] [<ffffffffc00554dd>] vmxnet3_xmit_frame+0xb5d/0xed0 [vmxnet3] RSP: 0018:ffff88042d643770 EFLAGS: 00000286 RAX: 0000000000000000 RBX: ffff880179f44000 RCX: 000000000000007f RDX: ffffc900018b4600 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff88042d643800 R08: 0c00bc92fc565000 R09: 0000000000c80036 R10: ffff88041a76c7e0 R11: 00405f9128001045 R12: ffff88042d6436e8 R13: ffffffff816b5c5d R14: ffff88042d643800 R15: ffff88042977e100 FS: 00007f39362f2740(0000) GS:ffff88042d640000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fde955a9180 CR3: 0000000034b6b000 CR4: 00000000003407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Stack: ffff880428b20000 ffff88042d643788 ffffffff815710d7 ffff88042d6437a8 ffff88041a76c7e0 0000000000000283 ffff880428b208e0 000088042d643800 0000002200000014 ffff88042977e100 0000000000000036 ffff880400000002 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
vmxnet3_xmit_frame
|
vmxnet3 |
0xffffffffc00743dc
Build id: 3.10.0-1062.12.1.el7.x86_64 |
0xaec |
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [WTJourn.Flusher:9656] Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm dns_resolver krg_12_4_1_8542_imRH7K1smp64(POE) ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock nfit sb_edac libnvdimm iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel pcspkr aesni_intel lrw gf128mul joydev vmw_balloon glue_helper ablk_helper vmw_vmci sg cryptd parport_pc parport i2c_piix4 nfsd nfs_acl lockd grace auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi crct10dif_pclmul crct10dif_common crc32c_intel scsi_transport_spi mptscsih serio_raw libata mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 1 PID: 9656 Comm: WTJourn.Flusher Kdump: loaded Tainted: P OE ------------ 3.10.0-1062.12.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017 task: ffff8f4183f7e2a0 ti: ffff8f416b790000 task.ti: ffff8f416b790000 RIP: 0010:[<ffffffffc00743dc>] [<ffffffffc00743dc>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0018:ffff8f41ffd036f0 EFLAGS: 00000286 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000078 RDX: ffffb56c018ba600 RSI: 0000000000000283 RDI: 0000000000000283 RBP: ffff8f41ffd03780 R08: 0000000000000013 R09: 0000000000c84042 R10: 01010000e4509005 R11: ffff8f41dfe40770 R12: ffff8f41ffd03668 R13: ffffffff9558eefa R14: ffff8f41ffd03780 R15: 0000000000000001 FS: 00007f50392b9700(0000) GS:ffff8f41ffd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f8651099000 CR3: 0000000422b4e000 CR4: 00000000001607e0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
vmxnet3_xmit_frame
|
vmxnet3 |
0xffffffffc03173dc
Build id: 3.10.0-1127.19.1.el7.x86_64 |
0xaec |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 30s! [Socket Thread:3390] Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm nfit libata crct10dif_pclmul crct10dif_common libnvdimm crc32c_intel mptspi serio_raw scsi_transport_spi mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 3390 Comm: Socket Thread Kdump: loaded Tainted: G L ------------ 3.10.0-1127.19.1.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018 task: ffff96de4bbe0000 ti: ffff96de35c90000 task.ti: ffff96de35c90000 RIP: 0010:[<ffffffffc03173dc>] [<ffffffffc03173dc>] vmxnet3_xmit_frame+0xaec/0xfd0 [vmxnet3] RSP: 0018:ffff96deffc036d0 EFLAGS: 00000286 RAX: 0000000000000000 RBX: ffffffffc07b1fad RCX: 0000000000000186 RDX: ffffa9b30062c600 RSI: 0000000000000293 RDI: 0000000000000293 RBP: ffff96deffc03760 R08: e43bbb01e6bfcad8 R09: 0000000000000036 R10: 00000000f9d4e43b R11: ffff96de72c7f850 R12: ffff96deffc03648 R13: ffffffffac793efa R14: ffff96deffc03760 R15: 0000000000000001 FS: 00007fa02f442700(0000) GS:ffff96deffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ff5f2ce5000 CR3: 000000008bb7a000 CR4: 00000000000407f0 Call Trace: