Complete report #923448
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
get_request
|
vmlinux |
0xffffffff89969248
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x338
|
|
2
|
|
vmlinux |
0xffffffff896cc790
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x40
|
|
3
|
blk_get_request_flags
|
vmlinux |
0xffffffff8996a07c
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0xec
|
|
4
|
scsi_execute
|
vmlinux |
0xffffffff89b092dd
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x3d
|
|
5
|
scsi_execute_req_flags
|
vmlinux |
0xffffffff89b0b37e
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x8e
|
|
6
|
sr_check_events
|
sr_mod |
0xffffffffc03e635c
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0xbc
|
|
7
|
cdrom_check_events
|
cdrom |
0xffffffffc03d906e
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x1e
|
|
8
|
sr_block_check_events
|
sr_mod |
0xffffffffc03e7101
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0xb1
|
|
9
|
disk_check_events
|
vmlinux |
0xffffffff8997f106
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x66
|
|
10
|
disk_events_workfn
|
vmlinux |
0xffffffff8997f256
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x16
|
|
11
|
process_one_work
|
vmlinux |
0xffffffff896c32ef
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x17f
|
|
12
|
worker_thread
|
vmlinux |
0xffffffff896c4436
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x126
|
|
13
|
|
vmlinux |
0xffffffff896c4310
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x2b0
|
|
14
|
kthread
|
vmlinux |
0xffffffff896cb621
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0xd1
|
|
15
|
|
vmlinux |
0xffffffff896cb550
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x40
|
|
16
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffff89dc51f7
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x21
|
|
17
|
|
vmlinux |
0xffffffff896cb550
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [kworker/0:2:5115]
Modules linked in: nf_conntrack_netlink ts_kmp ts_bm nf_log_ipv4 nf_log_common xt_LOG xt_string xt_set ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ip_set_hash_ip ip_set_hash_net ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c dahdi(OE) crc_ccitt ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr joydev ppdev vmw_balloon vmw_vmci parport_pc parport i2c_piix4
ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod ata_generic crc_t10dif pata_acpi crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel mptspi e1000 serio_raw scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy
CPU: 0 PID: 5115 Comm: kworker/0:2 Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.90.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 06/22/2012
Workqueue: events_freezable disk_events_workfn
task: ffff9735c9c2b180 ti: ffff973637538000 task.ti: ffff973637538000
RIP: 0010:[<ffffffff89969248>] [<ffffffff89969248>] get_request+0x338/0x7d0
RSP: 0000:ffff97363753bb68 EFLAGS: 00000202
RAX: 0000000010200000 RBX: ffff9735c9c2b1e8 RCX: ffff9736b4f58030
RDX: 0000000042112002 RSI: 0000000000000001 RDI: ffff9736b4f58450
RBP: ffff97363753bc10 R08: 0000000000000008 R09: ffff9735c2c29d80
R10: ffff9735c2c29d80 R11: ffff973639d150c0 R12: 00000000e1047810
R13: 000000000001e18c R14: 0000000000000000 R15: ffffffff897668ef
FS: 0000000000000000(0000) GS:ffff9736bfc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000003a3a2f8 CR3: 00000000bb172000 CR4: 00000000000407f0
Call Trace:
[<ffffffff896cc790>] ? wake_up_atomic_t+0x40/0x40
[<ffffffff8996a07c>] blk_get_request_flags+0xec/0x190
[<ffffffff89b092dd>] scsi_execute+0x3d/0x170
[<ffffffff89b0b37e>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc03e635c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc03d906e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc03e7101>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffff8997f106>] disk_check_events+0x66/0x1a0
[<ffffffff8997f256>] disk_events_workfn+0x16/0x20
[<ffffffff896c32ef>] process_one_work+0x17f/0x440
[<ffffffff896c4436>] worker_thread+0x126/0x3c0
[<ffffffff896c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff896cb621>] kthread+0xd1/0xe0
[<ffffffff896cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff89dc51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff896cb550>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 97 40 04 00 00 44 89 e0 49 8b bf 58 04 00 00 0d 00 00 00 10 80 e6 20 44 0f 45 e0 c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <48> 8b 45 a0 89 de 48 8b 78 20 e8 29 31 e6 ff 48 85 c0 49 89 c6
Complete report #314392
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
get_request
|
vmlinux |
0xffffffffa8b54808
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x338
|
|
2
|
|
vmlinux |
0xffffffffa88c6f20
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x30
|
|
3
|
blk_get_request_flags
|
vmlinux |
0xffffffffa8b55374
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0xe4
|
|
4
|
scsi_execute
|
vmlinux |
0xffffffffa8cea10d
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x3d
|
|
5
|
scsi_execute_req_flags
|
vmlinux |
0xffffffffa8cec07e
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x8e
|
|
6
|
sr_check_events
|
sr_mod |
0xffffffffc04ac31c
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0xbc
|
|
7
|
cdrom_check_events
|
cdrom |
0xffffffffc05dc05e
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x1e
|
|
8
|
sr_block_check_events
|
sr_mod |
0xffffffffc04ad091
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0xb1
|
|
9
|
disk_check_events
|
vmlinux |
0xffffffffa8b69b06
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x66
|
|
10
|
disk_events_workfn
|
vmlinux |
0xffffffffa8b69c46
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x16
|
|
11
|
process_one_work
|
vmlinux |
0xffffffffa88bde5f
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x17f
|
|
12
|
worker_thread
|
vmlinux |
0xffffffffa88bef76
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x126
|
|
13
|
|
vmlinux |
0xffffffffa88bee50
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x2a0
|
|
14
|
kthread
|
vmlinux |
0xffffffffa88c5e31
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0xd1
|
|
15
|
|
vmlinux |
0xffffffffa88c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x40
|
|
16
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa8f95ddd
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x7
|
|
17
|
|
vmlinux |
0xffffffffa88c5d60
Build id: 3.10.0-1160.31.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [kworker/1:0:25064]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd joydev pcspkr sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod
syscopyarea crc_t10dif crct10dif_generic sysfillrect sysimgblt fb_sys_fops ttm ahci drm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw libahci nfit libnvdimm ata_piix vmxnet3 libata vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 25064 Comm: kworker/1:0 Kdump: loaded Not tainted 3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Workqueue: events_freezable disk_events_workfn
task: ffff98a20f8ad280 ti: ffff98a20442c000 task.ti: ffff98a20442c000
RIP: 0010:[<ffffffffa8b54808>] [<ffffffffa8b54808>] get_request+0x338/0x7d0
RSP: 0018:ffff98a20442fb68 EFLAGS: 00000202
RAX: 0000000010200000 RBX: ffff98a20f8ad2e8 RCX: ffff98a228a08030
RDX: 0000000042112002 RSI: 0000000000000001 RDI: ffff98a228a08450
RBP: ffff98a20442fc10 R08: 0000000000000008 R09: ffff98a227810f60
R10: ffff98a227810f60 R11: ffff989eb8623f00 R12: ffff98a228a08000
R13: ffff98a2299c4800 R14: ffff989e35556800 R15: ffffffffa8b56e7d
FS: 0000000000000000(0000) GS:ffff98a23fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcd70787b30 CR3: 000000042d0aa000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffa88c6f20>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffa8b55374>] blk_get_request_flags+0xe4/0x180
[<ffffffffa8cea10d>] scsi_execute+0x3d/0x170
[<ffffffffa8cec07e>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc04ac31c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc05dc05e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc04ad091>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffffa8b69b06>] disk_check_events+0x66/0x190
[<ffffffffa8b69c46>] disk_events_workfn+0x16/0x20
[<ffffffffa88bde5f>] process_one_work+0x17f/0x440
[<ffffffffa88bef76>] worker_thread+0x126/0x3c0
[<ffffffffa88bee50>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa88c5e31>] kthread+0xd1/0xe0
[<ffffffffa88c5d60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa8f95ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffa88c5d60>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 97 40 04 00 00 44 89 e0 49 8b bf 58 04 00 00 0d 00 00 00 10 80 e6 20 44 0f 45 e0 c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <48> 8b 45 a0 89 de 48 8b 78 20 e8 d9 c8 e6 ff 48 85 c0 49 89 c6
Complete report #106605
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
get_request
|
vmlinux |
0xffffffff9fd44248
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x338
|
|
2
|
|
vmlinux |
0xffffffff9fac2d40
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x30
|
|
3
|
blk_get_request_flags
|
vmlinux |
0xffffffff9fd44f54
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xe4
|
|
4
|
scsi_execute
|
vmlinux |
0xffffffff9fed700d
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x3d
|
|
5
|
scsi_execute_req_flags
|
vmlinux |
0xffffffff9fed8d8e
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x8e
|
|
6
|
sr_check_events
|
sr_mod |
0xffffffffc049a31c
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xbc
|
|
7
|
cdrom_check_events
|
cdrom |
0xffffffffc04b305e
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x1e
|
|
8
|
sr_block_check_events
|
sr_mod |
0xffffffffc049b0b1
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xb1
|
|
9
|
disk_check_events
|
vmlinux |
0xffffffff9fd591c6
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x66
|
|
10
|
disk_events_workfn
|
vmlinux |
0xffffffff9fd59306
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x16
|
|
11
|
process_one_work
|
vmlinux |
0xffffffff9fab9d8f
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x17f
|
|
12
|
worker_thread
|
vmlinux |
0xffffffff9fabae26
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x126
|
|
13
|
|
vmlinux |
0xffffffff9fabad00
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x2a0
|
|
14
|
kthread
|
vmlinux |
0xffffffff9fac1c71
Build id: 3.10.0-957.5.1.el7.x86_64
|
0xd1
|
|
15
|
|
vmlinux |
0xffffffff9fac1ba0
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x40
|
|
16
|
ret_from_fork_nospec_begin
|
vmlinux |
0xffffffffa0174c37
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x21
|
|
17
|
|
vmlinux |
0xffffffff9fac1ba0
Build id: 3.10.0-957.5.1.el7.x86_64
|
0x40
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:2:18410]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ipt_REJECT nf_reject_ipv4 ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev sg pcspkr i2c_piix4 parport_pc parport vmw_vmci xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix drm_panel_orientation_quirks libata vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nf_defrag_ipv4]
CPU: 0 PID: 18410 Comm: kworker/0:2 Kdump: loaded Tainted: G L ------------ 3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: events_freezable disk_events_workfn
task: ffffa0da2e266180 ti: ffffa0d997918000 task.ti: ffffa0d997918000
RIP: 0010:[<ffffffff9fd44248>] [<ffffffff9fd44248>] get_request+0x338/0x7d0
RSP: 0018:ffffa0d99791bb68 EFLAGS: 00000202
RAX: 0000000010200000 RBX: 0000000000000000 RCX: ffffa0dab1e78030
RDX: 0000000042112002 RSI: 0000000000000001 RDI: ffffa0dab1e78450
RBP: ffffa0d99791bc10 R08: 0000000000000008 R09: ffffa0da4cbd4960
R10: ffffa0da4cbd4960 R11: 0000000000000005 R12: 00000000e48c10eb
R13: 00000000000190da R14: 0000000000000000 R15: ffffffff9fb5822f
FS: 0000000000000000(0000) GS:ffffa0dabfc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f16035d9000 CR3: 00000000158a2000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9fac2d40>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff9fd44f54>] blk_get_request_flags+0xe4/0x180
[<ffffffff9fed700d>] scsi_execute+0x3d/0x170
[<ffffffff9fed8d8e>] scsi_execute_req_flags+0x8e/0x100
[<ffffffffc049a31c>] sr_check_events+0xbc/0x2d0 [sr_mod]
[<ffffffffc04b305e>] cdrom_check_events+0x1e/0x40 [cdrom]
[<ffffffffc049b0b1>] sr_block_check_events+0xb1/0x120 [sr_mod]
[<ffffffff9fd591c6>] disk_check_events+0x66/0x190
[<ffffffff9fd59306>] disk_events_workfn+0x16/0x20
[<ffffffff9fab9d8f>] process_one_work+0x17f/0x440
[<ffffffff9fabae26>] worker_thread+0x126/0x3c0
[<ffffffff9fabad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff9fac1c71>] kthread+0xd1/0xe0
[<ffffffff9fac1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa0174c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fac1ba0>] ? insert_kthread_work+0x40/0x40
Code: 49 8b 97 40 04 00 00 44 89 e0 49 8b bf 58 04 00 00 0d 00 00 00 10 80 e6 20 44 0f 45 e0 c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <48> 8b 45 a0 89 de 48 8b 78 20 e8 29 54 e7 ff 48 85 c0 49 89 c6