| Operating system | Count |
|---|---|
| CentOS 7 | 15 |
| Architecture | Count |
|---|---|
| x86_64 | 15 |
| Related packages | Count |
|---|---|
| kernel | 15 |
Packages names with count less than the total count of reports are most probably not the cause of the problem.
NMI watchdog: BUG: soft lockup - CPU#144 stuck for 22s! [llama-server:23849] Modules linked in: tcp_diag inet_diag binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nvidia(POE) vfat fat ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt rpcrdma sunrpc rdma_ucm ib_iser ib_umad joydev cdc_ether usbnet mii sg i2c_piix4 ipmi_si ipmi_devintf ipmi_msghandler rdma_cm ib_ipoib iw_cm libiscsi scsi_transport_iscsi i2c_designware_platform pinctrl_amd i2c_designware_core ib_cm acpi_cpufreq mlx5_ib ib_uverbs ib_core ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast drm_kms_helper syscopyarea mlx5_core sysfillrect sysimgblt fb_sys_fops ttm ahci igb libahci drm mlxfw dca libata devlink i2c_algo_bit megaraid_sas ptp drm_panel_orientation_quirks pps_core nfit libnvdimm wmi dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 144 PID: 23849 Comm: llama-server Kdump: loaded Tainted: P OEL ------------ 3.10.0-1160.119.1.el7.x86_64 #1 Hardware name: ASUSTeK COMPUTER INC. ESC8000A-E11/KMPG-D32 Series, BIOS 2002 08/30/2024 task: ffff9ebb37bc0000 ti: ffff9e83b19c0000 task.ti: ffff9e83b19c0000 RIP: 0010:[<ffffffff8f424245>] [<ffffffff8f424245>] iommu_unmap_page+0x15/0x110 RSP: 0018:ffff9e83b19c39a8 EFLAGS: 00000206 RAX: 0000008000000000 RBX: 000000000000001b RCX: 0000000000000027 RDX: 0000000000001000 RSI: 0000897b8cb59000 RDI: ffff9ec3705ed000 RBP: ffff9e83b19c39d0 R08: 0000000000000002 R09: 0000000000000000 R10: 000000000000001b R11: 000ffffffffff000 R12: 000000000000001b R13: ffff9e83b19c39d0 R14: ffffffff8f5b7920 R15: ffff9e83b19c3988 FS: 00007faa1b695000(0000) GS:ffff9ec37e600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f259475df08 CR3: 0000017dfa10c000 CR4: 0000000000760ee0 PKRU: 55555554 Call Trace: [<ffffffff8f426523>] __unmap_single.isra.22+0x63/0x200 [<ffffffff8f426e7f>] unmap_sg+0x5f/0x80 [<ffffffffd13983ce>] nv_unmap_dma_map_scatterlist+0x8e/0xc0 [nvidia] [<ffffffffd139941b>] nv_dma_unmap_alloc+0x19b/0x1b0 [nvidia] [<ffffffffd1e90fd5>] _nv041699rm+0xc5/0x1d0 [nvidia] [<ffffffffd13f7ddf>] ? _nv032724rm+0x6f/0x90 [nvidia] [<ffffffffd145bdca>] ? _nv039226rm+0x25a/0x290 [nvidia] [<ffffffffd1ea4c80>] ? rm_gpu_ops_free_duped_handle+0x20/0x60 [nvidia] [<ffffffffd13a877a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x50 [nvidia] [<ffffffffc113db2a>] ? uvm_ext_gpu_map_free_internal+0x6a/0x90 [nvidia_uvm] [<ffffffffc113e8be>] ? uvm_ext_gpu_map_free+0xe/0x30 [nvidia_uvm] [<ffffffffc10eb5c4>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm] [<ffffffffc113fa10>] ? uvm_api_unmap_external+0x100/0x220 [nvidia_uvm] [<ffffffffc10d7c5d>] ? uvm_ioctl+0x1a9d/0x1dd0 [nvidia_uvm] [<ffffffffd13a1d3a>] ? os_release_spinlock+0x1a/0x20 [nvidia] [<ffffffffd13b87cd>] ? _nv049913rm+0xed/0x1d0 [nvidia] [<ffffffffd13a1d02>] ? os_acquire_spinlock+0x12/0x30 [nvidia] [<ffffffffd13a1d3a>] ? os_release_spinlock+0x1a/0x20 [nvidia] [<ffffffffd13b87cd>] ? _nv049913rm+0xed/0x1d0 [nvidia] [<ffffffffd1e93927>] ? rm_ioctl+0x67/0xb0 [nvidia] [<ffffffffc10d7fcc>] ? uvm_unlocked_ioctl+0x3c/0x60 [nvidia_uvm] [<ffffffffc10d80e4>] ? uvm_unlocked_ioctl_entry+0x64/0xd0 [nvidia_uvm] [<ffffffff8f0719d8>] ? do_vfs_ioctl+0x3a8/0x5c0 [<ffffffff8f071c71>] ? SyS_ioctl+0x81/0xa0 [<ffffffff8ef46986>] ? __audit_syscall_exit+0x1f6/0x2b0 [<ffffffff8f5c539a>] ? system_call_fastpath+0x25/0x2a Code: f8 c3 cc cc cc cc 31 ff 48 89 f8 5d c3 cc cc cc cc 0f 1f 44 00 00 66 66 66 66 90 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 89 d3 <48> 83 ec 10 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 85
NMI watchdog: BUG: soft lockup - CPU#96 stuck for 22s! [llama-server:25290] Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) binfmt_misc tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nvidia(POE) vfat fat ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt rpcrdma sunrpc rdma_ucm ib_iser joydev cdc_ether usbnet mii ib_umad sg i2c_piix4 ipmi_si ipmi_devintf ipmi_msghandler rdma_cm ib_ipoib iw_cm pinctrl_amd i2c_designware_platform ib_cm i2c_designware_core libiscsi scsi_transport_iscsi acpi_cpufreq mlx5_ib ib_uverbs ib_core ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mlx5_core ttm igb ahci drm libahci dca mlxfw i2c_algo_bit libata devlink megaraid_sas drm_panel_orientation_quirks ptp pps_core nfit libnvdimm wmi dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 96 PID: 25290 Comm: llama-server Kdump: loaded Tainted: P OEL ------------ T 3.10.0-1160.119.1.el7.x86_64 #1 Hardware name: ASUSTeK COMPUTER INC. ESC8000A-E11/KMPG-D32 Series, BIOS 2002 08/30/2024 task: ffff95bc737b1080 ti: ffff95f84c890000 task.ti: ffff95f84c890000 RIP: 0010:[<ffffffff96a240f1>] [<ffffffff96a240f1>] fetch_pte.isra.6+0x51/0x190 RSP: 0018:ffff95f84c893988 EFLAGS: 00000206 RAX: ffff95f4653a9898 RBX: ffffb2c4ef845008 RCX: 0000000000000027 RDX: 000089bf8f8cd000 RSI: 0000008000000000 RDI: 0000000000000003 RBP: ffff95f84c893988 R08: 0000000000000002 R09: ffff95f84c893998 R10: 000000000000001b R11: 000ffffffffff000 R12: ffff95bf2ba0eda0 R13: 0000000000000010 R14: ffffffff96633b6a R15: ffff95f84c893900 FS: 00007f6b88efc000(0000) GS:ffff95134e200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000002af47eb0 CR3: 000001557b646000 CR4: 0000000000760ee0 PKRU: 55555554 Call Trace: [<ffffffff96a24296>] iommu_unmap_page+0x66/0x110 [<ffffffff96a26523>] __unmap_single.isra.22+0x63/0x200 [<ffffffff96a26e7f>] unmap_sg+0x5f/0x80 [<ffffffffc11e83ce>] nv_unmap_dma_map_scatterlist+0x8e/0xc0 [nvidia] [<ffffffffc11e941b>] nv_dma_unmap_alloc+0x19b/0x1b0 [nvidia] [<ffffffffc1ce0fd5>] _nv041699rm+0xc5/0x1d0 [nvidia] [<ffffffffc1247ddf>] ? _nv032724rm+0x6f/0x90 [nvidia] [<ffffffffc12abdca>] ? _nv039226rm+0x25a/0x290 [nvidia] [<ffffffffc1cf4c80>] ? rm_gpu_ops_free_duped_handle+0x20/0x60 [nvidia] [<ffffffffc11f877a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x50 [nvidia] [<ffffffffc460db2a>] ? uvm_ext_gpu_map_free_internal+0x6a/0x90 [nvidia_uvm] [<ffffffffc460e8be>] ? uvm_ext_gpu_map_free+0xe/0x30 [nvidia_uvm] [<ffffffffc45bb5c4>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm] [<ffffffffc460fa10>] ? uvm_api_unmap_external+0x100/0x220 [nvidia_uvm] [<ffffffffc45a7c5d>] ? uvm_ioctl+0x1a9d/0x1dd0 [nvidia_uvm] [<ffffffff965fd4d4>] ? unmap_page_range+0xa94/0xc80 [<ffffffff965d9685>] ? lru_add_drain+0x15/0x20 [<ffffffff9661610d>] ? free_pages_and_swap_cache+0x1d/0xd0 [<ffffffffc45a7fcc>] ? uvm_unlocked_ioctl+0x3c/0x60 [nvidia_uvm] [<ffffffffc45a80e4>] ? uvm_unlocked_ioctl_entry+0x64/0xd0 [nvidia_uvm] [<ffffffff966719d8>] ? do_vfs_ioctl+0x3a8/0x5c0 [<ffffffff96671c71>] ? SyS_ioctl+0x81/0xa0 [<ffffffff96546986>] ? __audit_syscall_exit+0x1f6/0x2b0 [<ffffffff96bc539a>] ? system_call_fastpath+0x25/0x2a Code: 56 01 00 00 83 ef 01 48 8b 36 48 89 d0 44 8d 14 ff 41 8d 4a 0c 48 d3 e8 25 ff 01 00 00 48 8d 04 c6 be 01 00 00 00 48 d3 e6 85 ff <49> 89 31 48 8b 30 0f 8e b2 00 00 00 40 f6 c6 01 0f 84 19 01 00