| Operating system | Count |
|---|---|
| CentOS 7 | 21 |
| Architecture | Count |
|---|---|
| x86_64 | 21 |
| Related packages | Count |
|---|---|
| kernel | 21 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
dec_pending
|
dm_mod |
0xffffffffc0090037
Build id: 3.10.0-1062.4.1.el7.x86_64 |
0xc7 |
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/2:1:7830] Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14203(E) falcon_lsm_pinned_14108(E) falcon_lsm_pinned_14006(E) devlink binfmt_misc ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_hda_codec_generic lrw gf128mul glue_helper ablk_helper cryptd joydev snd_hda_intel virtio_balloon snd_hda_codec sg snd_hda_core snd_hwdep parport_pc snd_seq parport snd_seq_device snd_pcm snd_timer snd i2c_piix4 pcspkr soundcore auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal] CPU: 2 PID: 7830 Comm: kworker/2:1 Kdump: loaded Tainted: P EL ------------ 3.10.0-1062.4.1.el7.x86_64 #1 Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Workqueue: events_freezable disk_events_workfn task: ffffa0474724c1c0 ti: ffffa04747300000 task.ti: ffffa04747300000 RIP: 0010:[<ffffffffc0090037>] [<ffffffffc0090037>] dec_pending+0xc7/0x350 [dm_mod] RSP: 0018:ffffa053df083db8 EFLAGS: 00010282 RAX: ffffa02ae068e000 RBX: ffffa02ae068e000 RCX: 0000000000000002 RDX: ffffa053df0a5140 RSI: 0000000000000008 RDI: ffffa053d8c9a660 RBP: ffffa053df083e08 R08: ffffa0474729d000 R09: 00000000000040d8 R10: ffffa049f82b4968 R11: ffffa0474729d000 R12: ffffa053df083d28 R13: ffffffffa238cefa R14: ffffa053df083e08 R15: ffffa02ae068e000 FS: 0000000000000000(0000) GS:ffffa053df080000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000005f5f6002 CR3: 00000027ebe3a000 CR4: 00000000001606e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
dec_pending
|
dm_mod |
0xffffffffc0250f99
Build id: 3.10.0-1160.59.1.el7.x86_64 |
0x29 |
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 9360s! [WRRende~ckend#1:3579] Modules linked in: nf_nat_ftp nf_conntrack_ftp binfmt_misc tcp_lp vboxsf(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc intel_powerclamp crc32_pclmul ghash_clmulni_intel snd_intel8x0 snd_ac97_codec ac97_bus snd_seq joydev snd_seq_device ppdev snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_timer pcspkr sg snd vboxguest(OE) soundcore i2c_piix4 parport_pc parport video ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci ata_piix libata e1000 crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 3579 Comm: WRRende~ckend#1 Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.59.1.el7.x86_64 #1 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 task: ffff98f225a55280 ti: ffff98f22c4b8000 task.ti: ffff98f22c4b8000 RIP: 0010:[<ffffffffc0250f99>] [<ffffffffc0250f99>] dec_pending+0x29/0x270 [dm_mod] RSP: 0000:ffff98f29fc03cb8 EFLAGS: 00000246 RAX: 000000000000000d RBX: ffff98f24c654000 RCX: 0000000000000000 RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff98f281373700 RBP: ffff98f29fc03d08 R08: ffff98f1b5518500 R09: 000000018010000f R10: 0000000000000001 R11: ffff98f1b5518500 R12: ffff98f29fc03c28 R13: ffffffffa7b9afba R14: ffff98f29fc03d08 R15: ffff98f1b5518518 FS: 00007f76e10fd700(0000) GS:ffff98f29fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000564716033000 CR3: 000000011727a000 CR4: 00000000000606f0 Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
dec_pending
|
dm_mod |
0xffffffffc0001e29
Build id: 3.10.0-693.el7.x86_64 |
0x59 |
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [kworker/5:0:6870] Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg pcspkr vmw_balloon joydev parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih mptbase libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw e1000 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 5 PID: 6870 Comm: kworker/5:0 Tainted: G W L ------------ 3.10.0-693.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 Workqueue: xfs-data/dm-0 xfs_end_io [xfs] task: ffff88083fec9fa0 ti: ffff880185390000 task.ti: ffff880185390000 RIP: 0010:[<ffffffffc0001e29>] [<ffffffffc0001e29>] dec_pending+0x59/0x350 [dm_mod] RSP: 0018:ffff88081ef43ca0 EFLAGS: 00000206 RAX: ffff88081a2f4800 RBX: ffff8807df2a4a60 RCX: 0000000180200008 RDX: 0000000000000411 RSI: 0000000000000000 RDI: ffff88022fbcde70 RBP: ffff88081ef43cf0 R08: ffff88038ea6f800 R09: 00000000000064b4 R10: ffff88022fbcde70 R11: ffffea000e3a9b80 R12: ffff88081ef43c18 R13: ffffffff816b5c1d R14: ffff88081ef43cf0 R15: ffff88081a2f4800 FS: 0000000000000000(0000) GS:ffff88081ef40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fe3ac871000 CR3: 00000000019f2000 CR4: 00000000000407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Stack: ffffffff81185477 ffff88081ef43cd0 ffffffff81185879 ffff88038ea6f818 ffff880818fbed80 ffff88038ea6f818 0000000000000000 ffff88022fbcde70 ffff88081a2f4800 0000000000000000 ffff88081ef43d28 ffffffffc0002c3e Call Trace:
| Frame # | Function | Binary | Source or offset | Line |
|---|---|---|---|---|
| 1 |
dec_pending
|
dm_mod |
0xffffffffa0000807
Build id: 3.10.0-327.28.3.el7.x86_64 |
0x57 |
BUG: soft lockup - CPU#6 stuck for 22s! [rcuos/3:21] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter binfmt_misc vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg pcspkr vmw_balloon parport_pc parport i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul ttm crct10dif_common mptspi ata_piix crc32c_intel scsi_transport_spi serio_raw drm mptscsih libata vmxnet3 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod CPU: 6 PID: 21 Comm: rcuos/3 Tainted: G L ------------ 3.10.0-327.28.3.el7.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015 task: ffff88032e5e6780 ti: ffff88032e614000 task.ti: ffff88032e614000 RIP: 0010:[<ffffffffa0000807>] [<ffffffffa0000807>] dec_pending+0x57/0x2e0 [dm_mod] RSP: 0018:ffff88033fd83d08 EFLAGS: 00000216 RAX: 0000000000000001 RBX: ffff88033fd83ca8 RCX: 000000018020001a RDX: 000000018020001b RSI: 0000000000000000 RDI: ffff880124810c08 RBP: ffff88033fd83d50 R08: ffff8802838ca200 R09: 000000000000206a R10: ffffffff8116c287 R11: 0000000000000431 R12: ffff88033fd83c78 R13: ffffffff8164779d R14: ffff88033fd83d50 R15: ffff880124810c08 FS: 0000000000000000(0000) GS:ffff88033fd80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f4c3943e172 CR3: 000000032d980000 CR4: 00000000003407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffff88033fd83d30 ffffffff8116c689 ffff8802838ca218 ffff8803291c6480 ffff8802838ca218 0000000000000000 ffff880124810c08 ffff880328727000 0000000000000000 ffff88033fd83d88 ffffffffa00016ce 00000000fffffffb Call Trace: