| Operating system | Count |
|---|---|
| CentOS 7 | 197 |
| CentOS 8 | 16 |
| Architecture | Count |
|---|---|
| x86_64 | 213 |
| Related packages | Count |
|---|---|
| kernel | 213 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
WARNING: CPU: 1 PID: 3780 at net/netfilter/nf_nat_core.c:404 nf_nat_setup_info+0x353/0x360 [nf_nat] Modules linked in: xt_nat xt_NFQUEUE nf_nat_ftp nf_conntrack_ftp nf_nat_pptp nf_nat_proto_gre nf_conntrack_pptp nf_conntrack_proto_gre ip_gre ip_tunnel gre xt_comment xt_recent xt_conntrack iptable_filter iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack nfnetlink_queue snapapi26(POE) tun lp sunrpc ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log dm_mod intel_powerclamp coretemp ppdev kvm_intel gpio_ich iTCO_wdt iTCO_vendor_support kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel ipmi_si lrw gf128mul glue_helper ablk_helper cryptd ipmi_devintf ipmi_msghandler pcspkr sg joydev i2c_i801 lpc_ich ioatdma dca i7core_edac parport_pc parport tpm_infineon acpi_cpufreq ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom uas mgag200 usb_storage i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm libata e1000e crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks ptp pps_core fuse [last unloaded: edd] CPU: 1 PID: 3780 Comm: billing Kdump: loaded Tainted: P IOE ------------ 3.10.0-1160.119.1.el7.x86_64 #1 Hardware name: Supermicro X8STi/X8STi, BIOS 1.0c 03/10/2010 Call Trace: [<ffffffffa41b1bec>] dump_stack+0x19/0x1f [<ffffffffa3a9f868>] __warn+0xd8/0x100 [<ffffffffa3a9f9ad>] warn_slowpath_null+0x1d/0x30 [<ffffffffc06e04b3>] nf_nat_setup_info+0x353/0x360 [nf_nat] [<ffffffffc07d51b4>] xt_snat_target_v0+0x74/0x95 [xt_nat] [<ffffffffc01ec70b>] ipt_do_table+0x30b/0x750 [ip_tables] [<ffffffffa3aa89b7>] ? local_bh_enable+0x17/0x20 [<ffffffffc01ec761>] ? ipt_do_table+0x361/0x750 [ip_tables] [<ffffffffa407edc6>] ? dev_hard_start_xmit+0x246/0x3b0 [<ffffffffc076a045>] iptable_nat_do_chain+0x25/0x30 [iptable_nat] [<ffffffffc04e2779>] nf_nat_ipv4_fn+0x179/0x230 [nf_nat_ipv4] [<ffffffffc076a020>] ? iptable_nat_ipv4_fn+0x20/0x20 [iptable_nat] [<ffffffffc04e291b>] nf_nat_ipv4_out+0x5b/0x110 [nf_nat_ipv4] [<ffffffffc076a088>] iptable_nat_ipv4_out+0x18/0x20 [iptable_nat] [<ffffffffa40c0ef8>] nf_iterate+0x98/0xe0 [<ffffffffa40c0fe8>] nf_hook_slow+0xa8/0x110 [<ffffffffa40d2c02>] ip_output+0xe2/0x100 [<ffffffffa40d1f90>] ? __ip_append_data.isra.50+0xa60/0xa60 [<ffffffffa40ce17c>] ip_forward_finish+0x6c/0x90 [<ffffffffa40c238a>] nf_reinject+0xda/0x1b0 [<ffffffffc06cb5ee>] nfqnl_recv_verdict+0x21e/0x320 [nfnetlink_queue] [<ffffffffa3dcc386>] ? nla_parse+0xb6/0x120 [<ffffffffa40c2bd2>] nfnetlink_rcv_msg+0x162/0x270 [<ffffffffa40c2a70>] ? nfnetlink_net_exit_batch+0x80/0x80 [<ffffffffa40be47b>] netlink_rcv_skb+0xab/0xc0 [<ffffffffa40c30ef>] nfnetlink_rcv+0x29f/0x590 [<ffffffffa40bc6d3>] ? __netlink_lookup+0xd3/0x130 [<ffffffffa40bdde8>] netlink_unicast+0x178/0x220 [<ffffffffa3db3e62>] ? memcpy_fromiovec+0x62/0xb0 [<ffffffffa40be198>] netlink_sendmsg+0x308/0x420 [<ffffffffa405e4f6>] sock_sendmsg+0xb6/0x100 [<ffffffffa3a2c671>] ? __switch_to+0x151/0x590 [<ffffffffa3b19428>] ? get_futex_key+0x1d8/0x2d0 [<ffffffffa405f3f9>] ___sys_sendmsg+0x3e9/0x400 [<ffffffffa41b7920>] ? __schedule+0x320/0x680 [<ffffffffa3acfca9>] ? hrtimer_try_to_cancel+0x29/0x130 [<ffffffffa41b6e5a>] ? do_nanosleep+0x9a/0x100 [<ffffffffa4060b41>] __sys_sendmsg+0x51/0x90 [<ffffffffa4060b92>] SyS_sendmsg+0x12/0x30 [<ffffffffa41c539a>] system_call_fastpath+0x25/0x2a
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [billing:4125] Modules linked in: snapapi26(POE) xt_nat xt_NFQUEUE nf_nat_ftp nf_conntrack_ftp nf_nat_pptp nf_nat_proto_gre nf_conntrack_pptp nf_conntrack_proto_gre ip_gre gre ip_tunnel xt_comment xt_recent xt_conntrack iptable_filter iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack nfnetlink_queue nfnetlink tun lp sunrpc dm_mirror dm_region_hash dm_log dm_mod ext4 mbcache jbd2 iTCO_wdt gpio_ich iTCO_vendor_support ppdev intel_powerclamp coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel sg aesni_intel ipmi_si lrw gf128mul glue_helper ipmi_devintf ablk_helper cryptd ipmi_msghandler pcspkr joydev lpc_ich i2c_i801 ioatdma i7core_edac dca parport_pc parport tpm_infineon acpi_cpufreq ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic uas usb_storage mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm libata e1000e crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks ptp pps_core [last unloaded: edd] CPU: 1 PID: 4125 Comm: billing Kdump: loaded Tainted: P IOEL ------------ 3.10.0-957.27.2.el7.x86_64 #1 Hardware name: Supermicro X8STi/X8STi, BIOS 1.0c 03/10/2010 task: ffff98d5417c30c0 ti: ffff98d53a7c0000 task.ti: ffff98d53a7c0000 RIP: 0010:[<ffffffffc0648e4a>] [<ffffffffc0648e4a>] get_unique_tuple+0x3fa/0x670 [nf_nat] RSP: 0018:ffff98d53a7c3470 EFLAGS: 00000246 RAX: 0000000000000006 RBX: 0000000000000001 RCX: ffff98d564200000 RDX: ffff98d5642743e0 RSI: ffff98d4a22312c0 RDI: 00000000760c3220 RBP: ffff98d53a7c34f8 R08: ffff98d4b0bc9528 R09: ffff98d53a7c3508 R10: ffff98d4aef88bd8 R11: ffff98d2b5d68000 R12: ffff98d4217b2e68 R13: ffff98d53a7c3508 R14: ffffffffbaf119c0 R15: ffff98d2b5d68000 FS: 00007f87a7fff700(0000) GS:ffff98d593240000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055df08cb04d0 CR3: 00000000c22e0000 CR4: 00000000000207e0 Call Trace: [<ffffffffc0788737>] ? nf_ct_invert_tuple+0x77/0x90 [nf_conntrack] [<ffffffffc064914c>] nf_nat_setup_info+0x8c/0x320 [nf_nat] [<ffffffffc0821194>] xt_snat_target_v0+0x74/0x8a [xt_nat] [<ffffffffc028a6ab>] ipt_do_table+0x2eb/0x710 [ip_tables] [<ffffffffba2a15d7>] ? local_bh_enable+0x17/0x20 [<ffffffffc028a6fe>] ? ipt_do_table+0x33e/0x710 [ip_tables] [<ffffffffc07ab045>] iptable_nat_do_chain+0x25/0x30 [iptable_nat] [<ffffffffc062b749>] nf_nat_ipv4_fn+0x179/0x230 [nf_nat_ipv4] [<ffffffffc07ab020>] ? iptable_nat_ipv4_fn+0x20/0x20 [iptable_nat] [<ffffffffc062b8d8>] nf_nat_ipv4_out+0x48/0xf0 [nf_nat_ipv4] [<ffffffffc07ab088>] iptable_nat_ipv4_out+0x18/0x20 [iptable_nat] [<ffffffffba87a528>] nf_iterate+0x98/0xe0 [<ffffffffba87a618>] nf_hook_slow+0xa8/0x110 [<ffffffffba88ad6e>] ip_output+0xce/0xe0 [<ffffffffba88a260>] ? __ip_append_data.isra.51+0xa50/0xa50 [<ffffffffba88660c>] ip_forward_finish+0x6c/0x90 [<ffffffffba87b7ea>] nf_reinject+0xda/0x1b0 [<ffffffffc06318de>] nfqnl_recv_verdict+0x24e/0x370 [nfnetlink_queue] [<ffffffffba5a53e6>] ? nla_parse+0xb6/0x120 [<ffffffffc054c402>] nfnetlink_rcv_msg+0x162/0x270 [nfnetlink] [<ffffffffc054c2a0>] ? nfnetlink_bind+0x60/0x60 [nfnetlink] [<ffffffffba877bab>] netlink_rcv_skb+0xab/0xc0 [<ffffffffc054c87f>] nfnetlink_rcv+0x27f/0x518 [nfnetlink] [<ffffffffba875ec0>] ? __netlink_lookup+0xc0/0x110 [<ffffffffba877530>] netlink_unicast+0x170/0x210 [<ffffffffba58dd02>] ? memcpy_fromiovec+0x62/0xb0 [<ffffffffba8778d8>] netlink_sendmsg+0x308/0x420 [<ffffffffba81b3a6>] sock_sendmsg+0xb6/0xf0 [<ffffffffba81c269>] ___sys_sendmsg+0x3e9/0x400 [<ffffffffba22a621>] ? __switch_to+0x151/0x580 [<ffffffffba969aba>] ? __schedule+0x42a/0x860 [<ffffffffba2c6efd>] ? hrtimer_start_range_ns+0x1ed/0x3c0 [<ffffffffba968f67>] ? do_nanosleep+0xa7/0xf0 [<ffffffffba2c7cbb>] ? hrtimer_nanosleep+0xbb/0x180 [<ffffffffba81d921>] __sys_sendmsg+0x51/0x90 [<ffffffffba81d972>] SyS_sendmsg+0x12/0x20 [<ffffffffba976ddb>] system_call_fastpath+0x22/0x27 [<ffffffffba976d21>] ? system_call_after_swapgs+0xae/0x146 Code: 24 4c 89 d3 4d 89 cf 4d 89 c5 eb 16 0f 1f 44 00 00 48 8b 1b 48 85 db 0f 84 f4 01 00 00 41 0f b6 47 26 48 8b 73 10 38 46 46 75 e6 <41> 8b 07 39 46 20 75 de 41 8b 47 04 39 46 24 75 d5 41 8b 47 08
WARNING: CPU: 1 PID: 3208 at net/netfilter/nf_nat_core.c:404 nf_nat_setup_info+0x353/0x360 [nf_nat] Modules linked in: arc4 md4 nls_utf8 cifs ccm dns_resolver snapapi26(POE) xt_nat xt_NFQUEUE nf_nat_ftp nf_conntrack_ftp nf_nat_pptp nf_nat_proto_gre nf_conntrack_pptp nf_conntrack_proto_gre ip_gre ip_tunnel gre xt_comment xt_recent xt_conntrack iptable_filter iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack nfnetlink_queue tun lp sunrpc ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log dm_mod intel_powerclamp coretemp kvm_intel iTCO_wdt iTCO_vendor_support gpio_ich kvm ppdev irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel sg ipmi_si lrw gf128mul glue_helper ablk_helper cryptd ipmi_devintf ipmi_msghandler joydev pcspkr lpc_ich i2c_i801 ioatdma i7core_edac dca parport_pc parport tpm_infineon acpi_cpufreq ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic uas mgag200 i2c_algo_bit drm_kms_helper usb_storage syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci e1000e libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks ptp pps_core fuse [last unloaded: edd] CPU: 1 PID: 3208 Comm: billing Kdump: loaded Tainted: P IOE ------------ 3.10.0-1160.102.1.el7.x86_64 #1 Hardware name: Supermicro X8STi/X8STi, BIOS 1.0c 03/10/2010 Call Trace: [<ffffffff839b1bec>] dump_stack+0x19/0x1f [<ffffffff8329f868>] __warn+0xd8/0x100 [<ffffffff8329f9ad>] warn_slowpath_null+0x1d/0x30 [<ffffffffc08544b3>] nf_nat_setup_info+0x353/0x360 [nf_nat] [<ffffffffc08c61b4>] xt_snat_target_v0+0x74/0x95 [xt_nat] [<ffffffffc02ec70b>] ipt_do_table+0x30b/0x750 [ip_tables] [<ffffffff832a89b7>] ? local_bh_enable+0x17/0x20 [<ffffffffc02ec761>] ? ipt_do_table+0x361/0x750 [ip_tables] [<ffffffffc084a045>] iptable_nat_do_chain+0x25/0x30 [iptable_nat] [<ffffffffc054b779>] nf_nat_ipv4_fn+0x179/0x230 [nf_nat_ipv4] [<ffffffffc084a020>] ? iptable_nat_ipv4_fn+0x20/0x20 [iptable_nat] [<ffffffffc054b91b>] nf_nat_ipv4_out+0x5b/0x110 [nf_nat_ipv4] [<ffffffffc084a088>] iptable_nat_ipv4_out+0x18/0x20 [iptable_nat] [<ffffffff838c0ef8>] nf_iterate+0x98/0xe0 [<ffffffff838c0fe8>] nf_hook_slow+0xa8/0x110 [<ffffffff838d2c02>] ip_output+0xe2/0x100 [<ffffffff838d1f90>] ? __ip_append_data.isra.50+0xa60/0xa60 [<ffffffff838ce17c>] ip_forward_finish+0x6c/0x90 [<ffffffff838c238a>] nf_reinject+0xda/0x1b0 [<ffffffffc05d85ee>] nfqnl_recv_verdict+0x21e/0x320 [nfnetlink_queue] [<ffffffff835cc346>] ? nla_parse+0xb6/0x120 [<ffffffff838c2bd2>] nfnetlink_rcv_msg+0x162/0x270 [<ffffffff838c2a70>] ? nfnetlink_net_exit_batch+0x80/0x80 [<ffffffff838be47b>] netlink_rcv_skb+0xab/0xc0 [<ffffffff838c30ef>] nfnetlink_rcv+0x29f/0x590 [<ffffffff838bc6d3>] ? __netlink_lookup+0xd3/0x130 [<ffffffff838bdde8>] netlink_unicast+0x178/0x220 [<ffffffff835b3e22>] ? memcpy_fromiovec+0x62/0xb0 [<ffffffff838be198>] netlink_sendmsg+0x308/0x420 [<ffffffff8385e4f6>] sock_sendmsg+0xb6/0x100 [<ffffffff83319408>] ? get_futex_key+0x1d8/0x2d0 [<ffffffff8385f3f9>] ___sys_sendmsg+0x3e9/0x400 [<ffffffff839b7920>] ? __schedule+0x320/0x680 [<ffffffff832cfca9>] ? hrtimer_try_to_cancel+0x29/0x130 [<ffffffff839b6e5a>] ? do_nanosleep+0x9a/0x100 [<ffffffff83860b41>] __sys_sendmsg+0x51/0x90 [<ffffffff83860b92>] SyS_sendmsg+0x12/0x30 [<ffffffff839c539a>] system_call_fastpath+0x25/0x2a