Info

Permalink
Function
iommu_unmap_page
First occurrence
2023-09-07
Last occurrence
2023-09-13
State
NEW
Components
kernel Tainted
Quality
-76

Statistics

Operating system Count
CentOS 7 204
Architecture Count
x86_64 204
Related packages Count
kernel 204
 0:3.10.0-1160.119.1.el7 102
 0:3.10.0-1160.el7 44
 0:3.10.0-1160.102.1.el7 37
 0:3.10.0-1160.92.1.el7 21

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1420034
Frame # Function Binary Source or offset Line
1
fetch_pte.isra.6
vmlinux 0xffffffff96a24222
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x182
2
iommu_unmap_page
vmlinux 0xffffffff96a24296
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x66
3
__unmap_single.isra.22
vmlinux 0xffffffff96a26523
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x63
4
unmap_sg
vmlinux 0xffffffff96a26e7f
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x5f
5
nv_unmap_dma_map_scatterlist
nvidia 0xffffffffc11e83ce
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x8e
6
nv_dma_unmap_alloc
nvidia 0xffffffffc11e941b
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x19b
7
_nv041699rm
nvidia 0xffffffffc1ce0fd5
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xc5
8
? _nv032724rm
nvidia 0xffffffffc1247ddf
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x6f
9
? _nv039226rm
nvidia 0xffffffffc12abdca
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x25a
10
? rm_gpu_ops_free_duped_handle
nvidia 0xffffffffc1cf4c80
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x20
11
? nvUvmInterfaceFreeDupedHandle
nvidia 0xffffffffc11f877a
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x2a
12
? uvm_ext_gpu_map_free_internal
nvidia_uvm 0xffffffffc460db2a
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x6a
13
? uvm_ext_gpu_map_free
nvidia_uvm 0xffffffffc460e8be
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xe
14
? uvm_deferred_free_object_list
nvidia_uvm 0xffffffffc45bb5c4
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x64
15
? uvm_va_space_destroy
nvidia_uvm 0xffffffffc45bb915
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x295
16
? uvm_release.isra.12
nvidia_uvm 0xffffffffc45a5878
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xa8
17
? uvm_release_entry
nvidia_uvm 0xffffffffc45a59d4
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x54
18
? __fput
vmlinux 0xffffffff9665dfcc
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xec
19
? ____fput
vmlinux 0xffffffff9665e1fe
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xe
20
? task_work_run
vmlinux 0xffffffff964c7f6b
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xbb
21
? do_exit
vmlinux 0xffffffff964a6174
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x2d4
22
? futex_wait
vmlinux 0xffffffff9651a61f
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x11f
23
? do_group_exit
vmlinux 0xffffffff964a694f
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x3f
24
? get_signal_to_deliver
vmlinux 0xffffffff964b825e
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x1ce
25
? do_signal
vmlinux 0xffffffff9642d5b7
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x57
26
? do_futex
vmlinux 0xffffffff9651c376
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x106
27
? override_creds
vmlinux 0xffffffff964d2440
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x30
28
? do_notify_resume
vmlinux 0xffffffff9642dcca
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x7a
29
? int_signal
vmlinux 0xffffffff96bc56f7
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x12 
NMI watchdog: BUG: soft lockup - CPU#165 stuck for 22s! [llama-server:12200]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) binfmt_misc tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nvidia(POE) vfat fat ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt rpcrdma sunrpc rdma_ucm ib_iser joydev cdc_ether usbnet mii ib_umad sg i2c_piix4
ipmi_si ipmi_devintf ipmi_msghandler rdma_cm ib_ipoib iw_cm pinctrl_amd i2c_designware_platform ib_cm i2c_designware_core libiscsi scsi_transport_iscsi acpi_cpufreq mlx5_ib ib_uverbs ib_core ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mlx5_core ttm igb ahci drm libahci dca mlxfw i2c_algo_bit libata devlink megaraid_sas drm_panel_orientation_quirks ptp pps_core nfit libnvdimm wmi dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 165 PID: 12200 Comm: llama-server Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: ASUSTeK COMPUTER INC. ESC8000A-E11/KMPG-D32 Series, BIOS 2002 08/30/2024
task: ffff951192463180 ti: ffff95f9de620000 task.ti: ffff95f9de620000
RIP: 0010:[<ffffffff96a24222>]  [<ffffffff96a24222>] fetch_pte.isra.6+0x182/0x190
RSP: 0018:ffff95f9de6238f8  EFLAGS: 00000246
RAX: ffff960f5a3688d0 RBX: ffffffff96bb7920 RCX: 0000000000000027
RDX: 00008d751a68f000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff95f9de6238f8 R08: 0000000000000002 R09: ffff95f9de623908
R10: 000000000000001b R11: 000ffffffffff000 R12: 0000000000000001
R13: 0000000000000000 R14: ffff9612feb5acc0 R15: ffff9612dc0a2bc0
FS:  00007f703f79a700(0000) GS:ffff9612feb40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb870fde9e0 CR3: 00000020fe010000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffff96a24296>] iommu_unmap_page+0x66/0x110
[<ffffffff96a26523>] __unmap_single.isra.22+0x63/0x200
[<ffffffff96a26e7f>] unmap_sg+0x5f/0x80
[<ffffffffc11e83ce>] nv_unmap_dma_map_scatterlist+0x8e/0xc0 [nvidia]
[<ffffffffc11e941b>] nv_dma_unmap_alloc+0x19b/0x1b0 [nvidia]
[<ffffffffc1ce0fd5>] _nv041699rm+0xc5/0x1d0 [nvidia]
[<ffffffffc1247ddf>] ? _nv032724rm+0x6f/0x90 [nvidia]
[<ffffffffc12abdca>] ? _nv039226rm+0x25a/0x290 [nvidia]
[<ffffffffc1cf4c80>] ? rm_gpu_ops_free_duped_handle+0x20/0x60 [nvidia]
[<ffffffffc11f877a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x50 [nvidia]
[<ffffffffc460db2a>] ? uvm_ext_gpu_map_free_internal+0x6a/0x90 [nvidia_uvm]
[<ffffffffc460e8be>] ? uvm_ext_gpu_map_free+0xe/0x30 [nvidia_uvm]
[<ffffffffc45bb5c4>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm]
[<ffffffffc45bb915>] ? uvm_va_space_destroy+0x295/0x400 [nvidia_uvm]
[<ffffffffc45a5878>] ? uvm_release.isra.12+0xa8/0x140 [nvidia_uvm]
[<ffffffffc45a59d4>] ? uvm_release_entry+0x54/0xb0 [nvidia_uvm]
[<ffffffff9665dfcc>] ? __fput+0xec/0x230
[<ffffffff9665e1fe>] ? ____fput+0xe/0x20
[<ffffffff964c7f6b>] ? task_work_run+0xbb/0xe0
[<ffffffff964a6174>] ? do_exit+0x2d4/0xa30
[<ffffffff9651a61f>] ? futex_wait+0x11f/0x290
[<ffffffff964a694f>] ? do_group_exit+0x3f/0xa0
[<ffffffff964b825e>] ? get_signal_to_deliver+0x1ce/0x5e0
[<ffffffff9642d5b7>] ? do_signal+0x57/0x6f0
[<ffffffff9651c376>] ? do_futex+0x106/0x4d0
[<ffffffff964d2440>] ? override_creds+0x30/0x30
[<ffffffff9642dcca>] ? do_notify_resume+0x7a/0xd0
[<ffffffff96bc56f7>] ? int_signal+0x12/0x17
Code: 48 89 c8 48 f7 e2 48 c1 ea 03 48 8d 04 d2 48 29 c1 48 89 f0 48 d3 e0 48 c1 e0 03 48 f7 d8 48 21 c7 48 89 f8 c3 cc cc cc cc 31 ff <48> 89 f8 5d c3 cc cc cc cc 0f 1f 44 00 00 66 66 66 66 90 55 48
Complete report #1420019
Frame # Function Binary Source or offset Line
1
iommu_unmap_page
vmlinux 0xffffffff96a24329
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xf9
2
__unmap_single.isra.22
vmlinux 0xffffffff96a26523
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x63
3
unmap_sg
vmlinux 0xffffffff96a26e7f
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x5f
4
nv_unmap_dma_map_scatterlist
nvidia 0xffffffffc11e83ce
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x8e
5
nv_dma_unmap_alloc
nvidia 0xffffffffc11e941b
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x19b
6
_nv041699rm
nvidia 0xffffffffc1ce0fd5
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xc5
7
? _nv032724rm
nvidia 0xffffffffc1247ddf
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x6f
8
? _nv039226rm
nvidia 0xffffffffc12abdca
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x25a
9
? rm_gpu_ops_free_duped_handle
nvidia 0xffffffffc1cf4c80
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x20
10
? nvUvmInterfaceFreeDupedHandle
nvidia 0xffffffffc11f877a
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x2a
11
? uvm_ext_gpu_map_free_internal
nvidia_uvm 0xffffffffc460db2a
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x6a
12
? uvm_ext_gpu_map_free
nvidia_uvm 0xffffffffc460e8be
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xe
13
? uvm_deferred_free_object_list
nvidia_uvm 0xffffffffc45bb5c4
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x64
14
? uvm_va_space_destroy
nvidia_uvm 0xffffffffc45bb915
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x295
15
? uvm_release.isra.12
nvidia_uvm 0xffffffffc45a5878
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xa8
16
? uvm_release_entry
nvidia_uvm 0xffffffffc45a59d4
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x54
17
? __fput
vmlinux 0xffffffff9665dfcc
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xec
18
? ____fput
vmlinux 0xffffffff9665e1fe
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xe
19
? task_work_run
vmlinux 0xffffffff964c7f6b
Build id: 3.10.0-1160.119.1.el7.x86_64 		0xbb
20
? do_exit
vmlinux 0xffffffff964a6174
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x2d4
21
? futex_wait
vmlinux 0xffffffff9651a61f
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x11f
22
? do_group_exit
vmlinux 0xffffffff964a694f
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x3f
23
? get_signal_to_deliver
vmlinux 0xffffffff964b825e
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x1ce
24
? do_signal
vmlinux 0xffffffff9642d5b7
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x57
25
? do_futex
vmlinux 0xffffffff9651c376
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x106
26
? do_notify_resume
vmlinux 0xffffffff9642dcca
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x7a
27
? int_signal
vmlinux 0xffffffff96bc56f7
Build id: 3.10.0-1160.119.1.el7.x86_64 		0x12 
NMI watchdog: BUG: soft lockup - CPU#185 stuck for 22s! [llama-server:93669]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) binfmt_misc tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nvidia(POE) vfat fat ib_isert iscsi_target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt rpcrdma sunrpc rdma_ucm ib_iser joydev cdc_ether usbnet mii ib_umad sg i2c_piix4
ipmi_si ipmi_devintf ipmi_msghandler rdma_cm ib_ipoib iw_cm pinctrl_amd i2c_designware_platform ib_cm i2c_designware_core libiscsi scsi_transport_iscsi acpi_cpufreq mlx5_ib ib_uverbs ib_core ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mlx5_core ttm igb ahci drm libahci dca mlxfw i2c_algo_bit libata devlink megaraid_sas drm_panel_orientation_quirks ptp pps_core nfit libnvdimm wmi dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 185 PID: 93669 Comm: llama-server Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: ASUSTeK COMPUTER INC. ESC8000A-E11/KMPG-D32 Series, BIOS 2002 08/30/2024
task: ffff951192463180 ti: ffff95120ecb8000 task.ti: ffff95120ecb8000
RIP: 0010:[<ffffffff96a24329>]  [<ffffffff96a24329>] iommu_unmap_page+0xf9/0x110
RSP: 0018:ffff95120ecbb908  EFLAGS: 00000206
RAX: 0000007fffffffff RBX: 000000000000001b RCX: 0000000000000027
RDX: ffffff8000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff95120ecbb940 R08: 0000000000000002 R09: 0000000000000000
R10: 000000000000001b R11: 000ffffffffff000 R12: ffffffff96bb7920
R13: ffff95120ecbb8f8 R14: 0000000000000001 R15: 0000000000000000
FS:  00007f9ecac36700(0000) GS:ffff9612ff040000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000564d8347f6b8 CR3: 0000007ddceee000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffff96a26523>] __unmap_single.isra.22+0x63/0x200
[<ffffffff96a26e7f>] unmap_sg+0x5f/0x80
[<ffffffffc11e83ce>] nv_unmap_dma_map_scatterlist+0x8e/0xc0 [nvidia]
[<ffffffffc11e941b>] nv_dma_unmap_alloc+0x19b/0x1b0 [nvidia]
[<ffffffffc1ce0fd5>] _nv041699rm+0xc5/0x1d0 [nvidia]
[<ffffffffc1247ddf>] ? _nv032724rm+0x6f/0x90 [nvidia]
[<ffffffffc12abdca>] ? _nv039226rm+0x25a/0x290 [nvidia]
[<ffffffffc1cf4c80>] ? rm_gpu_ops_free_duped_handle+0x20/0x60 [nvidia]
[<ffffffffc11f877a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x50 [nvidia]
[<ffffffffc460db2a>] ? uvm_ext_gpu_map_free_internal+0x6a/0x90 [nvidia_uvm]
[<ffffffffc460e8be>] ? uvm_ext_gpu_map_free+0xe/0x30 [nvidia_uvm]
[<ffffffffc45bb5c4>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm]
[<ffffffffc45bb915>] ? uvm_va_space_destroy+0x295/0x400 [nvidia_uvm]
[<ffffffffc45a5878>] ? uvm_release.isra.12+0xa8/0x140 [nvidia_uvm]
[<ffffffffc45a59d4>] ? uvm_release_entry+0x54/0xb0 [nvidia_uvm]
[<ffffffff9665dfcc>] ? __fput+0xec/0x230
[<ffffffff9665e1fe>] ? ____fput+0xe/0x20
[<ffffffff964c7f6b>] ? task_work_run+0xbb/0xe0
[<ffffffff964a6174>] ? do_exit+0x2d4/0xa30
[<ffffffff9651a61f>] ? futex_wait+0x11f/0x290
[<ffffffff964a694f>] ? do_group_exit+0x3f/0xa0
[<ffffffff964b825e>] ? get_signal_to_deliver+0x1ce/0x5e0
[<ffffffff9642d5b7>] ? do_signal+0x57/0x6f0
[<ffffffff9651c376>] ? do_futex+0x106/0x4d0
[<ffffffff9642dcca>] ? do_notify_resume+0x7a/0xd0
[<ffffffff96bc56f7>] ? int_signal+0x12/0x17
Code: 48 8b 75 d0 65 48 33 34 25 28 00 00 00 4c 89 f0 75 20 48 83 c4 10 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc 0f 0b 49 8d 46 ff <4c> 85 f0 74 d0 0f 0b e8 6b b2 a7 ff 90 66 2e 0f 1f 84 00 00 00
Complete report #1158970
Frame # Function Binary Source or offset Line
1
iommu_unmap_page
vmlinux 0xffffffffabe2429c
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xfc
2
__unmap_single.isra.22
vmlinux 0xffffffffabe266f3
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x63
3
unmap_sg
vmlinux 0xffffffffabe27c6f
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x5f
4
nv_unmap_dma_map_scatterlist
nvidia 0xffffffffc03fe3be
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x8e
5
nv_dma_unmap_alloc
nvidia 0xffffffffc03ff40b
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x19b
6
_nv041656rm
nvidia 0xffffffffc0ef40e5
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xc5
7
? _nv032682rm
nvidia 0xffffffffc045ddaf
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x6f
8
? _nv039183rm
nvidia 0xffffffffc04c1daa
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x25a
9
? rm_gpu_ops_free_duped_handle
nvidia 0xffffffffc0f07d90
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x20
10
? nvUvmInterfaceFreeDupedHandle
nvidia 0xffffffffc040e74a
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x2a
11
? uvm_ext_gpu_map_free_internal
nvidia_uvm 0xffffffffc46a4aaa
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x6a
12
? uvm_ext_gpu_map_free
nvidia_uvm 0xffffffffc46a583e
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xe
13
? uvm_deferred_free_object_list
nvidia_uvm 0xffffffffc46525a4
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x64
14
? uvm_va_space_destroy
nvidia_uvm 0xffffffffc46528f5
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x295
15
? uvm_release.isra.12
nvidia_uvm 0xffffffffc463c858
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xa8
16
? uvm_release_entry
nvidia_uvm 0xffffffffc463c9b4
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x54
17
? __fput
vmlinux 0xffffffffaba5df7c
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xec
18
? ____fput
vmlinux 0xffffffffaba5e1ae
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xe
19
? task_work_run
vmlinux 0xffffffffab8c7f6b
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xbb
20
? do_exit
vmlinux 0xffffffffab8a6174
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x2d4
21
? __schedule
vmlinux 0xffffffffabfb7920
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x320
22
? do_group_exit
vmlinux 0xffffffffab8a694f
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x3f
23
? get_signal_to_deliver
vmlinux 0xffffffffab8b825e
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x1ce
24
? do_signal
vmlinux 0xffffffffab82d5b7
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x57
25
? do_futex
vmlinux 0xffffffffab91c356
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x106
26
? override_creds
vmlinux 0xffffffffab8d2440
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x30
27
? do_notify_resume
vmlinux 0xffffffffab82dcca
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x7a
28
? int_signal
vmlinux 0xffffffffabfc56f7
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x12 
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [llama-server:11063]
Modules linked in: nvidia_uvm(OE) cmac xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype br_netfilter ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter binfmt_misc overlay(T) bridge stp llc bnep sunrpc vfat fat ext4 arc4 mbcache jbd2 joydev iwlmvm snd_hda_codec_hdmi mac80211 iwlwifi edac_mce_amd kvm_amd snd_hda_codec_generic kvm snd_hda_intel irqbypass crc32_pclmul snd_hda_codec btusb ghash_clmulni_intel btrtl btbcm btintel snd_hda_core aesni_intel bluetooth snd_hwdep lrw gf128mul snd_seq glue_helper eeepc_wmi ablk_helper asus_wmi cryptd sparse_keymap snd_seq_device video cfg80211 pcspkr hid_logitech_dj snd_pcm snd_timer sg snd rfkill soundcore i2c_piix4
pinctrl_amd i2c_designware_platform i2c_designware_core acpi_cpufreq ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci libahci drm libata crct10dif_pclmul crct10dif_common crc32c_intel nvme r8169 serio_raw sdhci_acpi iosf_mbi nvme_core sdhci drm_panel_orientation_quirks mmc_core wmi dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 8 PID: 11063 Comm: llama-server Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/TUF GAMING X570-PLUS (WI-FI), BIOS 2407 07/01/2020
task: ffff8f27b858b180 ti: ffff8f18c1a08000 task.ti: ffff8f18c1a08000
RIP: 0010:[<ffffffffabe2429c>]  [<ffffffffabe2429c>] iommu_unmap_page+0xfc/0x110
RSP: 0018:ffff8f18c1a0b908  EFLAGS: 00000246
RAX: 0000007fffffffff RBX: 000000000000001b RCX: 0000000000000027
RDX: ffffff8000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff8f18c1a0b940 R08: 0000000000000003 R09: 0000000000000000
R10: 000000000000001b R11: 000ffffffffff000 R12: ffffffffc0407cd2
R13: ffff8f18c1a0b8f8 R14: ffff8f27bbbe49c8 R15: 0000000000000046
FS:  00007f204cff9700(0000) GS:ffff8f27ce400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff86d2a9ff0 CR3: 0000001006610000 CR4: 0000000000340fe0
Call Trace:
[<ffffffffabe266f3>] __unmap_single.isra.22+0x63/0x200
[<ffffffffabe27c6f>] unmap_sg+0x5f/0x80
[<ffffffffc03fe3be>] nv_unmap_dma_map_scatterlist+0x8e/0xc0 [nvidia]
[<ffffffffc03ff40b>] nv_dma_unmap_alloc+0x19b/0x1b0 [nvidia]
[<ffffffffc0ef40e5>] _nv041656rm+0xc5/0x1d0 [nvidia]
[<ffffffffc045ddaf>] ? _nv032682rm+0x6f/0x90 [nvidia]
[<ffffffffc04c1daa>] ? _nv039183rm+0x25a/0x290 [nvidia]
[<ffffffffc0f07d90>] ? rm_gpu_ops_free_duped_handle+0x20/0x60 [nvidia]
[<ffffffffc040e74a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x50 [nvidia]
[<ffffffffc46a4aaa>] ? uvm_ext_gpu_map_free_internal+0x6a/0x90 [nvidia_uvm]
[<ffffffffc46a583e>] ? uvm_ext_gpu_map_free+0xe/0x30 [nvidia_uvm]
[<ffffffffc46525a4>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm]
[<ffffffffc46528f5>] ? uvm_va_space_destroy+0x295/0x400 [nvidia_uvm]
[<ffffffffc463c858>] ? uvm_release.isra.12+0xa8/0x140 [nvidia_uvm]
[<ffffffffc463c9b4>] ? uvm_release_entry+0x54/0xb0 [nvidia_uvm]
[<ffffffffaba5df7c>] ? __fput+0xec/0x230
[<ffffffffaba5e1ae>] ? ____fput+0xe/0x20
[<ffffffffab8c7f6b>] ? task_work_run+0xbb/0xe0
[<ffffffffab8a6174>] ? do_exit+0x2d4/0xa30
[<ffffffffabfb7920>] ? __schedule+0x320/0x680
[<ffffffffab8a694f>] ? do_group_exit+0x3f/0xa0
[<ffffffffab8b825e>] ? get_signal_to_deliver+0x1ce/0x5e0
[<ffffffffab82d5b7>] ? do_signal+0x57/0x6f0
[<ffffffffab91c356>] ? do_futex+0x106/0x4d0
[<ffffffffab8d2440>] ? override_creds+0x30/0x30
[<ffffffffab82dcca>] ? do_notify_resume+0x7a/0xd0
[<ffffffffabfc56f7>] ? int_signal+0x12/0x17
Code: d0 65 48 33 34 25 28 00 00 00 4c 89 f0 75 20 48 83 c4 10 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc 0f 0b 49 8d 46 ff 4c 85 f0 <74> d0 0f 0b e8 fb b2 a7 ff 90 66 2e 0f 1f 84 00 00 00 00 00 66
Complete report #1133199
Frame # Function Binary Source or offset Line
1
fetch_pte.isra.6
vmlinux 0xffffffff95624067
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x57
2
iommu_unmap_page
vmlinux 0xffffffff95624206
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x66
3
__unmap_single.isra.22
vmlinux 0xffffffff956266f3
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x63
4
unmap_sg
vmlinux 0xffffffff95627c6f
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x5f
5
nv_unmap_dma_map_scatterlist
nvidia 0xffffffffc383e3be
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x8e
6
nv_dma_unmap_alloc
nvidia 0xffffffffc383f40b
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x19b
7
_nv041656rm
nvidia 0xffffffffc43340e5
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xc5
8
? _nv032682rm
nvidia 0xffffffffc389ddaf
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x6f
9
? _nv039183rm
nvidia 0xffffffffc3901daa
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x25a
10
? rm_gpu_ops_free_duped_handle
nvidia 0xffffffffc4347d90
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x20
11
? nvUvmInterfaceFreeDupedHandle
nvidia 0xffffffffc384e74a
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x2a
12
? uvm_ext_gpu_map_free_internal
nvidia_uvm 0xffffffffc0eceaaa
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x6a
13
? uvm_ext_gpu_map_free
nvidia_uvm 0xffffffffc0ecf83e
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xe
14
? uvm_deferred_free_object_list
nvidia_uvm 0xffffffffc0e7c5a4
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x64
15
? uvm_va_space_destroy
nvidia_uvm 0xffffffffc0e7c8f5
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x295
16
? uvm_release.isra.12
nvidia_uvm 0xffffffffc0e66858
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xa8
17
? uvm_release_entry
nvidia_uvm 0xffffffffc0e669b4
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x54
18
? __fput
vmlinux 0xffffffff9525df7c
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xec
19
? ____fput
vmlinux 0xffffffff9525e1ae
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xe
20
? task_work_run
vmlinux 0xffffffff950c7f6b
Build id: 3.10.0-1160.102.1.el7.x86_64 		0xbb
21
? do_exit
vmlinux 0xffffffff950a6174
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x2d4
22
? generic_file_aio_read2
vmlinux 0xffffffff951cb179
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x1b9
23
? do_group_exit
vmlinux 0xffffffff950a694f
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x3f
24
? get_signal_to_deliver
vmlinux 0xffffffff950b825e
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x1ce
25
? do_signal
vmlinux 0xffffffff9502d5b7
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x57
26
? do_notify_resume
vmlinux 0xffffffff9502dcca
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x7a
27
? int_signal
vmlinux 0xffffffff957c56f7
Build id: 3.10.0-1160.102.1.el7.x86_64 		0x12 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [llama-cli:28078]
Modules linked in: md4 nls_utf8 cifs ccm dns_resolver nvidia_uvm(OE) cmac ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter devlink binfmt_misc overlay(T) nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep sunrpc vfat fat arc4 joydev ext4 mbcache jbd2 iwlmvm mac80211 snd_hda_codec_hdmi edac_mce_amd kvm_amd snd_hda_codec_generic kvm btusb
iwlwifi btrtl irqbypass snd_hda_intel btbcm btintel crc32_pclmul ghash_clmulni_intel snd_hda_codec bluetooth aesni_intel hid_logitech_dj snd_hda_core lrw snd_hwdep eeepc_wmi gf128mul glue_helper cfg80211 ablk_helper snd_seq asus_wmi sparse_keymap cryptd snd_seq_device video sg snd_pcm pcspkr rfkill snd_timer snd soundcore i2c_piix4 pinctrl_amd i2c_designware_platform i2c_designware_core acpi_cpufreq ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci libahci drm libata crct10dif_pclmul crct10dif_common nvme crc32c_intel serio_raw sdhci_acpi r8169 iosf_mbi nvme_core sdhci drm_panel_orientation_quirks mmc_core wmi dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 28078 Comm: llama-cli Kdump: loaded Tainted: P           OE  ------------ T 3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/TUF GAMING X570-PLUS (WI-FI), BIOS 2407 07/01/2020
task: ffff90194268e300 ti: ffff901881780000 task.ti: ffff901881780000
RIP: 0010:[<ffffffff95624067>]  [<ffffffff95624067>] fetch_pte.isra.6+0x57/0x190
RSP: 0018:ffff9018817838f8  EFLAGS: 00000206
RAX: ffff9004700bb8e0 RBX: ffffffffc3847cd2 RCX: 0000000000000027
RDX: 00008e5cd4d5b000 RSI: 0000000000000000 RDI: 0000000000000003
RBP: ffff9018817838f8 R08: 0000000000000003 R09: ffff901881783908
R10: 000000000000001b R11: 000ffffffffff000 R12: ffff901487270988
R13: 0000000000000046 R14: ffffffffc3847cd2 R15: ffff901881783860
FS:  00007fe797877000(0000) GS:ffff90194e2c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f13f19f1000 CR3: 0000000c97610000 CR4: 0000000000340fe0
Call Trace:
[<ffffffff95624206>] iommu_unmap_page+0x66/0x110
[<ffffffff956266f3>] __unmap_single.isra.22+0x63/0x200
[<ffffffff95627c6f>] unmap_sg+0x5f/0x80
[<ffffffffc383e3be>] nv_unmap_dma_map_scatterlist+0x8e/0xc0 [nvidia]
[<ffffffffc383f40b>] nv_dma_unmap_alloc+0x19b/0x1b0 [nvidia]
[<ffffffffc43340e5>] _nv041656rm+0xc5/0x1d0 [nvidia]
[<ffffffffc389ddaf>] ? _nv032682rm+0x6f/0x90 [nvidia]
[<ffffffffc3901daa>] ? _nv039183rm+0x25a/0x290 [nvidia]
[<ffffffffc4347d90>] ? rm_gpu_ops_free_duped_handle+0x20/0x60 [nvidia]
[<ffffffffc384e74a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x50 [nvidia]
[<ffffffffc0eceaaa>] ? uvm_ext_gpu_map_free_internal+0x6a/0x90 [nvidia_uvm]
[<ffffffffc0ecf83e>] ? uvm_ext_gpu_map_free+0xe/0x30 [nvidia_uvm]
[<ffffffffc0e7c5a4>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm]
[<ffffffffc0e7c8f5>] ? uvm_va_space_destroy+0x295/0x400 [nvidia_uvm]
[<ffffffffc0e66858>] ? uvm_release.isra.12+0xa8/0x140 [nvidia_uvm]
[<ffffffffc0e669b4>] ? uvm_release_entry+0x54/0xb0 [nvidia_uvm]
[<ffffffff9525df7c>] ? __fput+0xec/0x230
[<ffffffff9525e1ae>] ? ____fput+0xe/0x20
[<ffffffff950c7f6b>] ? task_work_run+0xbb/0xe0
[<ffffffff950a6174>] ? do_exit+0x2d4/0xa30
[<ffffffff951cb179>] ? generic_file_aio_read2+0x1b9/0x330
[<ffffffff950a694f>] ? do_group_exit+0x3f/0xa0
[<ffffffff950b825e>] ? get_signal_to_deliver+0x1ce/0x5e0
[<ffffffff9502d5b7>] ? do_signal+0x57/0x6f0
[<ffffffff9502dcca>] ? do_notify_resume+0x7a/0xd0
[<ffffffff957c56f7>] ? int_signal+0x12/0x17
Code: 01 48 8b 36 48 89 d0 44 8d 14 ff 41 8d 4a 0c 48 d3 e8 25 ff 01 00 00 48 8d 04 c6 be 01 00 00 00 48 d3 e6 85 ff 49 89 31 48 8b 30 <0f> 8e b2 00 00 00 40 f6 c6 01 0f 84 19 01 00 00 49 89 f0 49 c1
Complete report #902992
Frame # Function Binary Source or offset Line
1
fetch_pte.isra.6
vmlinux 0xffffffff8b9fb2a4
Build id: 3.10.0-1160.el7.x86_64 		0x24
2
iommu_unmap_page
vmlinux 0xffffffff8b9fb476
Build id: 3.10.0-1160.el7.x86_64 		0x66
3
__unmap_single.isra.22
vmlinux 0xffffffff8b9fd893
Build id: 3.10.0-1160.el7.x86_64 		0x63
4
unmap_sg
vmlinux 0xffffffff8b9fed9f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
5
nv_unmap_dma_map_scatterlist
nvidia 0xffffffffc122d40e
Build id: 3.10.0-1160.el7.x86_64 		0x8e
6
nv_dma_unmap_pages
nvidia 0xffffffffc122e0d5
Build id: 3.10.0-1160.el7.x86_64 		0x115
7
nv_dma_unmap_alloc
nvidia 0xffffffffc122e41d
Build id: 3.10.0-1160.el7.x86_64 		0x3d
8
_nv033026rm
nvidia 0xffffffffc1b609e1
Build id: 3.10.0-1160.el7.x86_64 		0xc1
9
? _nv026824rm
nvidia 0xffffffffc127f027
Build id: 3.10.0-1160.el7.x86_64 		0x67
10
? _nv031350rm
nvidia 0xffffffffc12d1a78
Build id: 3.10.0-1160.el7.x86_64 		0x118
11
? rm_gpu_ops_free_duped_handle
nvidia 0xffffffffc1b72e7c
Build id: 3.10.0-1160.el7.x86_64 		0x1c
12
? nvUvmInterfaceFreeDupedHandle
nvidia 0xffffffffc123b97a
Build id: 3.10.0-1160.el7.x86_64 		0x2a
13
? uvm_ext_gpu_map_free_internal
nvidia_uvm 0xffffffffc0ea5092
Build id: 3.10.0-1160.el7.x86_64 		0x62
14
? uvm_ext_gpu_map_free
nvidia_uvm 0xffffffffc0ea5d8e
Build id: 3.10.0-1160.el7.x86_64 		0xe
15
? uvm_deferred_free_object_list
nvidia_uvm 0xffffffffc0e66a04
Build id: 3.10.0-1160.el7.x86_64 		0x64
16
? uvm_va_space_destroy
nvidia_uvm 0xffffffffc0e66d5d
Build id: 3.10.0-1160.el7.x86_64 		0x29d
17
? uvm_release.isra.5
nvidia_uvm 0xffffffffc0e53030
Build id: 3.10.0-1160.el7.x86_64 		0x80
18
? uvm_release_entry
nvidia_uvm 0xffffffffc0e53114
Build id: 3.10.0-1160.el7.x86_64 		0x54
19
? __fput
vmlinux 0xffffffff8b64fe4c
Build id: 3.10.0-1160.el7.x86_64 		0xec
20
? ____fput
vmlinux 0xffffffff8b65007e
Build id: 3.10.0-1160.el7.x86_64 		0xe
21
? task_work_run
vmlinux 0xffffffff8b4c275b
Build id: 3.10.0-1160.el7.x86_64 		0xbb
22
? do_exit
vmlinux 0xffffffff8b4a1904
Build id: 3.10.0-1160.el7.x86_64 		0x2d4
23
? futex_wait
vmlinux 0xffffffff8b512fb5
Build id: 3.10.0-1160.el7.x86_64 		0x195
24
? do_group_exit
vmlinux 0xffffffff8b4a20cf
Build id: 3.10.0-1160.el7.x86_64 		0x3f
25
? get_signal_to_deliver
vmlinux 0xffffffff8b4b307e
Build id: 3.10.0-1160.el7.x86_64 		0x1ce
26
? do_signal
vmlinux 0xffffffff8b42c527
Build id: 3.10.0-1160.el7.x86_64 		0x57
27
? do_nanosleep
vmlinux 0xffffffff8bb8619a
Build id: 3.10.0-1160.el7.x86_64 		0x9a
28
? hrtimer_get_res
vmlinux 0xffffffff8b4c9860
Build id: 3.10.0-1160.el7.x86_64 		0x50
29
? do_nanosleep
vmlinux 0xffffffff8bb8615b
Build id: 3.10.0-1160.el7.x86_64 		0x5b
30
? do_notify_resume
vmlinux 0xffffffff8b42cc32
Build id: 3.10.0-1160.el7.x86_64 		0x72
31
? int_signal
vmlinux 0xffffffff8bb942ef
Build id: 3.10.0-1160.el7.x86_64 		0x12 
NMI watchdog: BUG: soft lockup - CPU#183 stuck for 23s! [processor:31295]
Modules linked in: nvidia_uvm(OE) taurus(OE) vulcan_gvb_board(OE) vulcancl(OE) vulcan_cxp_board(OE) knem(OE) binfmt_misc uas usb_storage tcp_lp joydev xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun drbg ansi_cprng ccm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib
rdma_ucm ib_uverbs sunrpc ib_umad rdma_cm ib_cm iw_cm i40iw amd64_edac_mod edac_mce_amd kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nvidia_drm(POE) nvidia_modeset(POE) arc4 nvidia(POE) vfat fat rtl8192cu rtl_usb rtl8192c_common rtlwifi mac80211 ipmi_ssif bnxt_re cfg80211 ib_core rfkill sg i2c_piix4 k10temp ipmi_si ipmi_devintf ipmi_msghandler i2c_designware_platform i2c_designware_core pinctrl_amd acpi_cpufreq ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel nouveau video mxm_wmi ast wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt ahci fb_sys_fops ttm i40e libahci bnxt_en drm libata ptp megaraid_sas pps_core devlink drm_panel_orientation_quirks
nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: mst_pciconf]
CPU: 183 PID: 31295 Comm: processor Kdump: loaded Tainted: P           OEL ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: GIGABYTE MZ72-HB0-00/MZ72-HB0-00, BIOS R14 05/29/2022
task: ffff8ff5f8fb8000 ti: ffff8ff5b7788000 task.ti: ffff8ff5b7788000
RIP: 0010:[<ffffffff8b9fb2a4>]  [<ffffffff8b9fb2a4>] fetch_pte.isra.6+0x24/0x190
RSP: 0018:ffff8ff5b778b8d0  EFLAGS: 00000206
RAX: 0000ffffffffffff RBX: ffff8ff5b778b8f8 RCX: 0000000000000030
RDX: 00008a9b0fe7a000 RSI: ffff8ef139b46098 RDI: 0000000000000004
RBP: ffff8ff5b778b8d0 R08: 0000000000000003 R09: ffff8ff5b778b8e0
R10: 000000000000001b R11: 000ffffffffff000 R12: ffff8ff5df785b00
R13: ffffffffc1234eb4 R14: ffffffffc123513c R15: ffff8ff5b778b850
FS:  00007fa1cb7fe700(0000) GS:ffff90ebaefc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6b680039c8 CR3: 000000ab4de10000 CR4: 0000000000340fe0
Call Trace:
[<ffffffff8b9fb476>] iommu_unmap_page+0x66/0x110
[<ffffffff8b9fd893>] __unmap_single.isra.22+0x63/0x200
[<ffffffff8b9fed9f>] unmap_sg+0x5f/0x70
[<ffffffffc122d40e>] nv_unmap_dma_map_scatterlist+0x8e/0xb0 [nvidia]
[<ffffffffc122e0d5>] nv_dma_unmap_pages+0x115/0x120 [nvidia]
[<ffffffffc122e41d>] nv_dma_unmap_alloc+0x3d/0x60 [nvidia]
[<ffffffffc1b609e1>] _nv033026rm+0xc1/0x1b0 [nvidia]
[<ffffffffc127f027>] ? _nv026824rm+0x67/0x90 [nvidia]
[<ffffffffc12d1a78>] ? _nv031350rm+0x118/0x120 [nvidia]
[<ffffffffc1b72e7c>] ? rm_gpu_ops_free_duped_handle+0x1c/0x60 [nvidia]
[<ffffffffc123b97a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x40 [nvidia]
[<ffffffffc0ea5092>] ? uvm_ext_gpu_map_free_internal+0x62/0x90 [nvidia_uvm]
[<ffffffffc0ea5d8e>] ? uvm_ext_gpu_map_free+0xe/0x20 [nvidia_uvm]
[<ffffffffc0e66a04>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm]
[<ffffffffc0e66d5d>] ? uvm_va_space_destroy+0x29d/0x3e0 [nvidia_uvm]
[<ffffffffc0e53030>] ? uvm_release.isra.5+0x80/0xa0 [nvidia_uvm]
[<ffffffffc0e53114>] ? uvm_release_entry+0x54/0xb0 [nvidia_uvm]
[<ffffffff8b64fe4c>] ? __fput+0xec/0x230
[<ffffffff8b65007e>] ? ____fput+0xe/0x10
[<ffffffff8b4c275b>] ? task_work_run+0xbb/0xe0
[<ffffffff8b4a1904>] ? do_exit+0x2d4/0xa20
[<ffffffff8b512fb5>] ? futex_wait+0x195/0x280
[<ffffffff8b4a20cf>] ? do_group_exit+0x3f/0xa0
[<ffffffff8b4b307e>] ? get_signal_to_deliver+0x1ce/0x5e0
[<ffffffff8b42c527>] ? do_signal+0x57/0x6f0
[<ffffffff8bb8619a>] ? do_nanosleep+0x9a/0x100
[<ffffffff8b4c9860>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8bb8615b>] ? do_nanosleep+0x5b/0x100
[<ffffffff8b42cc32>] ? do_notify_resume+0x72/0xc0
[<ffffffff8bb942ef>] ? int_signal+0x12/0x17
Code: 1f 84 00 00 00 00 00 66 66 66 66 90 55 83 ff 05 49 89 c9 48 89 e5 7f 19 8d 4c ff 0c b8 01 00 00 00 48 d3 e0 48 83 e8 01 48 39 d0 <0f> 82 56 01 00 00 83 ef 01 48 8b 36 48 89 d0 44 8d 14 ff 41 8d
Complete report #902989
Frame # Function Binary Source or offset Line
1
iommu_unmap_page
vmlinux 0xffffffff8b9fb432
Build id: 3.10.0-1160.el7.x86_64 		0x22
2
__unmap_single.isra.22
vmlinux 0xffffffff8b9fd893
Build id: 3.10.0-1160.el7.x86_64 		0x63
3
unmap_sg
vmlinux 0xffffffff8b9fed9f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
4
nv_unmap_dma_map_scatterlist
nvidia 0xffffffffc122d40e
Build id: 3.10.0-1160.el7.x86_64 		0x8e
5
nv_dma_unmap_pages
nvidia 0xffffffffc122e0d5
Build id: 3.10.0-1160.el7.x86_64 		0x115
6
nv_dma_unmap_alloc
nvidia 0xffffffffc122e41d
Build id: 3.10.0-1160.el7.x86_64 		0x3d
7
_nv033026rm
nvidia 0xffffffffc1b609e1
Build id: 3.10.0-1160.el7.x86_64 		0xc1
8
? _nv026824rm
nvidia 0xffffffffc127f027
Build id: 3.10.0-1160.el7.x86_64 		0x67
9
? _nv031350rm
nvidia 0xffffffffc12d1a78
Build id: 3.10.0-1160.el7.x86_64 		0x118
10
? rm_gpu_ops_free_duped_handle
nvidia 0xffffffffc1b72e7c
Build id: 3.10.0-1160.el7.x86_64 		0x1c
11
? nvUvmInterfaceFreeDupedHandle
nvidia 0xffffffffc123b97a
Build id: 3.10.0-1160.el7.x86_64 		0x2a
12
? uvm_ext_gpu_map_free_internal
nvidia_uvm 0xffffffffc0ea5092
Build id: 3.10.0-1160.el7.x86_64 		0x62
13
? uvm_ext_gpu_map_free
nvidia_uvm 0xffffffffc0ea5d8e
Build id: 3.10.0-1160.el7.x86_64 		0xe
14
? uvm_deferred_free_object_list
nvidia_uvm 0xffffffffc0e66a04
Build id: 3.10.0-1160.el7.x86_64 		0x64
15
? uvm_va_space_destroy
nvidia_uvm 0xffffffffc0e66d5d
Build id: 3.10.0-1160.el7.x86_64 		0x29d
16
? uvm_release.isra.5
nvidia_uvm 0xffffffffc0e53030
Build id: 3.10.0-1160.el7.x86_64 		0x80
17
? uvm_release_entry
nvidia_uvm 0xffffffffc0e53114
Build id: 3.10.0-1160.el7.x86_64 		0x54
18
? __fput
vmlinux 0xffffffff8b64fe4c
Build id: 3.10.0-1160.el7.x86_64 		0xec
19
? ____fput
vmlinux 0xffffffff8b65007e
Build id: 3.10.0-1160.el7.x86_64 		0xe
20
? task_work_run
vmlinux 0xffffffff8b4c275b
Build id: 3.10.0-1160.el7.x86_64 		0xbb
21
? do_exit
vmlinux 0xffffffff8b4a1904
Build id: 3.10.0-1160.el7.x86_64 		0x2d4
22
? futex_wait
vmlinux 0xffffffff8b512f3f
Build id: 3.10.0-1160.el7.x86_64 		0x11f
23
? task_numa_fault
vmlinux 0xffffffff8b4e38f0
Build id: 3.10.0-1160.el7.x86_64 		0x8d0
24
? do_group_exit
vmlinux 0xffffffff8b4a20cf
Build id: 3.10.0-1160.el7.x86_64 		0x3f
25
? get_signal_to_deliver
vmlinux 0xffffffff8b4b307e
Build id: 3.10.0-1160.el7.x86_64 		0x1ce
26
? do_signal
vmlinux 0xffffffff8b42c527
Build id: 3.10.0-1160.el7.x86_64 		0x57
27
? do_futex
vmlinux 0xffffffff8b514d06
Build id: 3.10.0-1160.el7.x86_64 		0x106
28
? do_notify_resume
vmlinux 0xffffffff8b42cc32
Build id: 3.10.0-1160.el7.x86_64 		0x72
29
? int_signal
vmlinux 0xffffffff8bb942ef
Build id: 3.10.0-1160.el7.x86_64 		0x12 
NMI watchdog: BUG: soft lockup - CPU#149 stuck for 23s! [processor:28076]
Modules linked in: nvidia_uvm(OE) taurus(OE) vulcan_gvb_board(OE) vulcancl(OE) vulcan_cxp_board(OE) knem(OE) binfmt_misc uas usb_storage tcp_lp joydev xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun drbg ansi_cprng ccm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib
rdma_ucm ib_uverbs sunrpc ib_umad rdma_cm ib_cm iw_cm i40iw amd64_edac_mod edac_mce_amd kvm_amd kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr nvidia_drm(POE) nvidia_modeset(POE) arc4 nvidia(POE) vfat fat rtl8192cu rtl_usb rtl8192c_common rtlwifi mac80211 ipmi_ssif bnxt_re cfg80211 ib_core rfkill sg i2c_piix4 k10temp ipmi_si ipmi_devintf ipmi_msghandler i2c_designware_platform i2c_designware_core pinctrl_amd acpi_cpufreq ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel nouveau video mxm_wmi ast wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt ahci fb_sys_fops ttm i40e libahci bnxt_en drm libata ptp megaraid_sas pps_core devlink drm_panel_orientation_quirks
nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: mst_pciconf]
CPU: 149 PID: 28076 Comm: processor Kdump: loaded Tainted: P           OE  ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: GIGABYTE MZ72-HB0-00/MZ72-HB0-00, BIOS R14 05/29/2022
task: ffff8ff78bfce300 ti: ffff8ff612bd0000 task.ti: ffff8ff612bd0000
RIP: 0010:[<ffffffff8b9fb432>]  [<ffffffff8b9fb432>] iommu_unmap_page+0x22/0x110
RSP: 0018:ffff8ff612bd38e0  EFLAGS: 00000282
RAX: 00000000788e0cea RBX: 000000000000001b RCX: 0000000000000027
RDX: 0000000000001000 RSI: 00008ab5d4429000 RDI: ffff8ef139b46000
RBP: ffff8ff612bd3918 R08: 0000000000000003 R09: 0000000000000000
R10: 000000000000001b R11: 000ffffffffff000 R12: ffff8ff612bd38f8
R13: ffff8ff612bd38d0 R14: ffff8ef23fc07500 R15: 000000000020001f
FS:  00007f82ed08e000(0000) GS:ffff90ebae740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbc486a97e8 CR3: 000000ab4de10000 CR4: 0000000000340fe0
Call Trace:
[<ffffffff8b9fd893>] __unmap_single.isra.22+0x63/0x200
[<ffffffff8b9fed9f>] unmap_sg+0x5f/0x70
[<ffffffffc122d40e>] nv_unmap_dma_map_scatterlist+0x8e/0xb0 [nvidia]
[<ffffffffc122e0d5>] nv_dma_unmap_pages+0x115/0x120 [nvidia]
[<ffffffffc122e41d>] nv_dma_unmap_alloc+0x3d/0x60 [nvidia]
[<ffffffffc1b609e1>] _nv033026rm+0xc1/0x1b0 [nvidia]
[<ffffffffc127f027>] ? _nv026824rm+0x67/0x90 [nvidia]
[<ffffffffc12d1a78>] ? _nv031350rm+0x118/0x120 [nvidia]
[<ffffffffc1b72e7c>] ? rm_gpu_ops_free_duped_handle+0x1c/0x60 [nvidia]
[<ffffffffc123b97a>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x40 [nvidia]
[<ffffffffc0ea5092>] ? uvm_ext_gpu_map_free_internal+0x62/0x90 [nvidia_uvm]
[<ffffffffc0ea5d8e>] ? uvm_ext_gpu_map_free+0xe/0x20 [nvidia_uvm]
[<ffffffffc0e66a04>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm]
[<ffffffffc0e66d5d>] ? uvm_va_space_destroy+0x29d/0x3e0 [nvidia_uvm]
[<ffffffffc0e53030>] ? uvm_release.isra.5+0x80/0xa0 [nvidia_uvm]
[<ffffffffc0e53114>] ? uvm_release_entry+0x54/0xb0 [nvidia_uvm]
[<ffffffff8b64fe4c>] ? __fput+0xec/0x230
[<ffffffff8b65007e>] ? ____fput+0xe/0x10
[<ffffffff8b4c275b>] ? task_work_run+0xbb/0xe0
[<ffffffff8b4a1904>] ? do_exit+0x2d4/0xa20
[<ffffffff8b512f3f>] ? futex_wait+0x11f/0x280
[<ffffffff8b4e38f0>] ? task_numa_fault+0x8d0/0xbb0
[<ffffffff8b4a20cf>] ? do_group_exit+0x3f/0xa0
[<ffffffff8b4b307e>] ? get_signal_to_deliver+0x1ce/0x5e0
[<ffffffff8b42c527>] ? do_signal+0x57/0x6f0
[<ffffffff8b514d06>] ? do_futex+0x106/0x5a0
[<ffffffff8b42cc32>] ? do_notify_resume+0x72/0xc0
[<ffffffff8bb942ef>] ? int_signal+0x12/0x17
Code: 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 89 d3 48 83 ec 10 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 85 d2 0f 84 be 00 00 00 48 8d 42 ff 48 85
Complete report #795889
Frame # Function Binary Source or offset Line
1
iommu_unmap_page
vmlinux 0xffffffffaaa24111
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xb1
2
__unmap_single.isra.22
vmlinux 0xffffffffaaa265b3
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x63
3
unmap_sg
vmlinux 0xffffffffaaa27b2f
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x5f
4
nv_unmap_dma_map_scatterlist
nvidia 0xffffffffc05be43b
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x6b
5
nv_dma_unmap_pages
nvidia 0xffffffffc05beee5
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x115
6
nv_dma_unmap_alloc
nvidia 0xffffffffc05bf1e6
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x16
7
? _nv022217rm
nvidia 0xffffffffc0622e10
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x60
8
_nv032865rm
nvidia 0xffffffffc0e7b211
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xc1
9
? _nv027692rm
nvidia 0xffffffffc063f3db
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x9b
10
? _nv030893rm
nvidia 0xffffffffc067b8f4
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x124
11
? rm_gpu_ops_free_duped_handle
nvidia 0xffffffffc0e8382c
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x1c
12
? nvUvmInterfaceFreeDupedHandle
nvidia 0xffffffffc05cb3ba
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x2a
13
? uvm_ext_gpu_map_free_internal
nvidia_uvm 0xffffffffc30edb5f
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x6f
14
? uvm_ext_gpu_map_free
nvidia_uvm 0xffffffffc30ee80e
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xe
15
? uvm_deferred_free_object_list
nvidia_uvm 0xffffffffc30b1434
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x64
16
? uvm_va_space_destroy
nvidia_uvm 0xffffffffc30b17ff
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x30f
17
? uvm_release.isra.5
nvidia_uvm 0xffffffffc309f068
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x88
18
? uvm_release_entry
nvidia_uvm 0xffffffffc309f144
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x54
19
? __fput
vmlinux 0xffffffffaa65df0c
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xec
20
? ____fput
vmlinux 0xffffffffaa65e13e
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xe
21
? task_work_run
vmlinux 0xffffffffaa4c7f6b
Build id: 3.10.0-1160.92.1.el7.x86_64 		0xbb
22
? do_exit
vmlinux 0xffffffffaa4a6174
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x2d4
23
? futex_wait
vmlinux 0xffffffffaa51a58f
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x11f
24
? do_group_exit
vmlinux 0xffffffffaa4a694f
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x3f
25
? get_signal_to_deliver
vmlinux 0xffffffffaa4b825e
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x1ce
26
? do_signal
vmlinux 0xffffffffaa42d5b7
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x57
27
? do_futex
vmlinux 0xffffffffaa51c2e6
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x106
28
? do_notify_resume
vmlinux 0xffffffffaa42dcca
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x7a
29
? int_signal
vmlinux 0xffffffffaabc56f7
Build id: 3.10.0-1160.92.1.el7.x86_64 		0x12 
NMI watchdog: BUG: soft lockup - CPU#46 stuck for 23s! [colabfold_batch:4617]
Modules linked in: nvidia_uvm(OE) tun nfsv3 bridge stp llc nfs fscache xt_multiport ebtable_filter ebtables ip6table_filter ip6_tables falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_15604(E) ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c xt_CHECKSUM iptable_mangle rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_uverbs ib_umad rdma_cm ib_cm dm_mirror dm_region_hash dm_log dm_mod iw_cm kvm_amd zfs(POE) kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr snd_hda_codec_hdmi
zunicode(POE) zzstd(OE) snd_hda_intel snd_hda_codec zlua(OE) snd_hda_core joydev snd_hwdep zcommon(POE) snd_seq snd_seq_device znvpair(POE) zavl(POE) snd_pcm icp(POE) snd_timer snd spl(OE) soundcore bnxt_re sg ib_core i2c_piix4 ipmi_si ipmi_devintf ipmi_msghandler pinctrl_amd i2c_designware_platform i2c_designware_core acpi_cpufreq wmi nfsd nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 nvidia_drm(POE) nvidia_modeset(POE) raid1 rndis_host cdc_ether usbnet mii nvidia(POE) sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel i2c_algo_bit drm_kms_helper ttm syscopyarea sysfillrect sysimgblt fb_sys_fops ahci drm libahci bnxt_en libata nvme devlink nvme_core drm_panel_orientation_quirks nfit libnvdimm fuse
CPU: 46 PID: 4617 Comm: colabfold_batch Kdump: loaded Tainted: P           OEL ------------   3.10.0-1160.92.1.el7.x86_64 #1
Hardware name: Supermicro Super Server/H12SSL-NT, BIOS 2.3 10/20/2021
task: ffff8d977a315280 ti: ffff8de88cda0000 task.ti: ffff8de88cda0000
RIP: 0010:[<ffffffffaaa24111>]  [<ffffffffaaa24111>] iommu_unmap_page+0xb1/0x110
RSP: 0018:ffff8de88cda38d8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 000000000000001b RCX: 0000000000000027
RDX: 00008e86c8ffa000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff8de88cda3910 R08: 0000000000000001 R09: 0000000000000000
R10: 000000000000001b R11: 000ffffffffff000 R12: ffff8d83b6148000
R13: ffff8de88cda38c8 R14: ffff8de88cda3fd8 R15: ffff8de88cda3fd8
FS:  00007fd751fff700(0000) GS:ffff8dffcdf80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f219f7f7000 CR3: 0000007d93992000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffffaaa265b3>] __unmap_single.isra.22+0x63/0x200
[<ffffffffaaa27b2f>] unmap_sg+0x5f/0x80
[<ffffffffc05be43b>] nv_unmap_dma_map_scatterlist+0x6b/0xc0 [nvidia]
[<ffffffffc05beee5>] nv_dma_unmap_pages+0x115/0x120 [nvidia]
[<ffffffffc05bf1e6>] nv_dma_unmap_alloc+0x16/0x30 [nvidia]
[<ffffffffc0622e10>] ? _nv022217rm+0x60/0x60 [nvidia]
[<ffffffffc0e7b211>] _nv032865rm+0xc1/0x1b0 [nvidia]
[<ffffffffc063f3db>] ? _nv027692rm+0x9b/0xc0 [nvidia]
[<ffffffffc067b8f4>] ? _nv030893rm+0x124/0x130 [nvidia]
[<ffffffffc0e8382c>] ? rm_gpu_ops_free_duped_handle+0x1c/0x60 [nvidia]
[<ffffffffc05cb3ba>] ? nvUvmInterfaceFreeDupedHandle+0x2a/0x40 [nvidia]
[<ffffffffc30edb5f>] ? uvm_ext_gpu_map_free_internal+0x6f/0x80 [nvidia_uvm]
[<ffffffffc30ee80e>] ? uvm_ext_gpu_map_free+0xe/0x30 [nvidia_uvm]
[<ffffffffc30b1434>] ? uvm_deferred_free_object_list+0x64/0x120 [nvidia_uvm]
[<ffffffffc30b17ff>] ? uvm_va_space_destroy+0x30f/0x440 [nvidia_uvm]
[<ffffffffc309f068>] ? uvm_release.isra.5+0x88/0xa0 [nvidia_uvm]
[<ffffffffc309f144>] ? uvm_release_entry+0x54/0xb0 [nvidia_uvm]
[<ffffffffaa65df0c>] ? __fput+0xec/0x230
[<ffffffffaa65e13e>] ? ____fput+0xe/0x20
[<ffffffffaa4c7f6b>] ? task_work_run+0xbb/0xe0
[<ffffffffaa4a6174>] ? do_exit+0x2d4/0xa30
[<ffffffffaa51a58f>] ? futex_wait+0x11f/0x290
[<ffffffffaa4a694f>] ? do_group_exit+0x3f/0xa0
[<ffffffffaa4b825e>] ? get_signal_to_deliver+0x1ce/0x5e0
[<ffffffffaa42d5b7>] ? do_signal+0x57/0x6f0
[<ffffffffaa51c2e6>] ? do_futex+0x106/0x4d0
[<ffffffffaa42dcca>] ? do_notify_resume+0x7a/0xd0
[<ffffffffaabc56f7>] ? int_signal+0x12/0x17
Code: f7 e1 48 c1 ea 03 48 8d 04 d2 48 29 c1 b8 01 00 00 00 48 d3 e0 48 89 c1 66 90 4b c7 04 c1 00 00 00 00 49 83 c0 01 44 39 c1 7f ef <48> 8b 45 c8 48 89 c2 49 01 c6 48 f7 da 49 21 d7 49 01 c7 4c 39