Complete report #21436
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff9d7883c5
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x15
|
|
2
|
pvscsi_queue
|
vmw_pvscsi |
0xffffffffc02a0632
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x4a2
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [in:imjournal:1277]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink arc4 md4 nls_utf8 cifs ccm dns_resolver ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel sg lrw gf128mul glue_helper pcspkr joydev vmw_vmci ablk_helper cryptd i2c_piix4 parport_pc parport ip_tables xfs libcrc32c
sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod crc_t10dif sysimgblt fb_sys_fops crct10dif_generic ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ahci libahci libata vmxnet3 nfit vmw_pvscsi libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1277 Comm: in:imjournal Kdump: loaded Tainted: G L ------------ 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8b3f387e62a0 ti: ffff8b3f36088000 task.ti: ffff8b3f36088000
RIP: 0010:[<ffffffff9d7883c5>] [<ffffffff9d7883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff8b3f3608baf0 EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff9d3b14b3 RCX: ffffffff9dc34e40
RDX: ffff9bb240eb0000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8b3f3608baf0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff8b3f346ab000 R11: ffff8b3db5f6b800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff8b3f259b7980 R15: ffffffff9d4e9d59
FS: 00007ff82a83b700(0000) GS:ffff8b3f3fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff82401e088 CR3: 00000000b42ee000 CR4: 00000000007607f0
PKRU: 55555554
Call Trace:
[<ffffffffc02a0632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
Complete report #406486
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffb398e9f5
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x15
|
|
2
|
pvscsi_queue
|
vmw_pvscsi |
0xffffffffc0317632
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x4a2
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 35s! [kworker/u4:1:15379]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_13109(E) tcp_lp falcon_lsm_pinned_12905(E) nfsv3 nfs_acl nfs lockd grace fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg parport_pc vmw_vmci parport i2c_piix4 binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic
drm_kms_helper ahci crct10dif_pclmul crct10dif_common syscopyarea sysfillrect ata_piix libahci sysimgblt fb_sys_fops crc32c_intel ttm serio_raw libata drm vmxnet3 nfit libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 1 PID: 15379 Comm: kworker/u4:1 Kdump: loaded Tainted: P EL ------------ 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff918bbb4ad280 ti: ffff918c1ae6c000 task.ti: ffff918c1ae6c000
RIP: 0010:[<ffffffffb398e9f5>] [<ffffffffb398e9f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff918c1ae6f8f0 EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffb35b2d63 RCX: ffffffffb3e34f40
RDX: ffffb6c540780000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff918c1ae6f8f0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff918b34815c00 R11: ffff918c33752800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff918b4ae36480 R15: ffffffffb36eba09
FS: 0000000000000000(0000) GS:ffff918c3fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1f90edc000 CR3: 000000008acda000 CR4: 00000000000607e0
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 33s! [xfsaild/dm-0:505]
Call Trace:
[<ffffffffc0317632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
Complete report #343913
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff9a58b4d5
Build id: 3.10.0-1160.24.1.el7.x86_64
|
0x15
|
|
2
|
pvscsi_queue
|
vmw_pvscsi |
0xffffffffc0320632
Build id: 3.10.0-1160.24.1.el7.x86_64
|
0x4a2
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u4:1:14531]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [mysqld:2331]
Modules linked in: tcp_diag inet_diag talpa_vfshook(OE) talpa_pedconnector(OE) talpa_pedevice(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) vmw_vsock_vmci_transport vsock nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack xt_multiport iptable_filter sb_edac iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd pcspkr joydev sg parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci ata_piix crct10dif_pclmul crct10dif_common crc32c_intel nfit serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks libnvdimm
dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 2331 Comm: mysqld Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff91a9f656a100 ti: ffff91aa7aebc000 task.ti: ffff91aa7aebc000
RIP: 0010:[<ffffffff9a58b4d5>] [<ffffffff9a58b4d5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff91aa7aebf9d0 EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffff9a1b28d3 RCX: ffff91abffcb9060
RDX: ffffa72c00e90000 RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff91aa7aebf9d0 R08: 0000000000000000 R09: ffff91ab31ba2500
R10: ffff91a9f5dd5c00 R11: ffff91ab4e3d0800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff91abcfd5f980 R15: ffffffff9a2eb369
FS: 00007f02015cc700(0000) GS:ffff91abffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd00c6f9624 CR3: 0000000173bb8000 CR4: 00000000003607e0
Call Trace:
[<ffffffffc0320632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
Complete report #269264
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffffb851b2c5
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x15
|
|
2
|
pvscsi_queue
|
vmw_pvscsi |
0xffffffffc0255622
Build id: 3.10.0-862.11.6.el7.x86_64
|
0x4a2
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [kworker/u21:2:9837]
Modules linked in: veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_1_9_8557(POE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon
joydev pcspkr sg parport_pc parport shpchp vmw_vmci i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod vmwgfx crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 9837 Comm: kworker/u21:2 Kdump: loaded Tainted: P OEL ------------ 3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:7)
task: ffff998205e76eb0 ti: ffff9979faf4c000 task.ti: ffff9979faf4c000
RIP: 0010:[<ffffffffb851b2c5>] [<ffffffffb851b2c5>] _raw_spin_unlock_irqrestore+0x15/0x20
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 28s! [kworker/u21:1:9259]
RSP: 0000:ffff9979faf4f8f0 EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffffb81795b3 RCX: ffff998f0d170060
RDX: ffffc1a54ce30000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9979faf4f8f0 R08: 0000000000000000 R09: ffff998164f56e00
R10: ffff997c3955e8d8 R11: ffff998f0d2ef800 R12: 0000000000000000
R13: 0000000000000020 R14: ffff998032686480 R15: ffffffffb82a7259
FS: 0000000000000000(0000) GS:ffff99828d900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4200e0000 CR3: 00000000a4800000 CR4: 00000000003607e0
Modules linked in: veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_1_9_8557(POE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon
joydev pcspkr sg parport_pc parport shpchp vmw_vmci i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod vmwgfx crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 9259 Comm: kworker/u21:1 Kdump: loaded Tainted: P OEL ------------ 3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:8)
task: ffff998252e3dee0 ti: ffff9980c9eb0000 task.ti: ffff9980c9eb0000
RIP: 0010:[<ffffffffb851b2c5>] [<ffffffffb851b2c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff99828db03c18 EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff99828db03bd8 RCX: ffff998f0d170060
RDX: ffffc1a54ce30000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff99828db03c18 R08: 0000000000000000 R09: ffff99787eab1400
R10: ffff997c3955e8d8 R11: ffff998f0d2ef800 R12: ffff99828db03b88
R13: ffffffffb85267b2 R14: ffff99828db03c18 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff99828db00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7064a9f000 CR3: 0000001463b90000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffc0255622>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]
Complete report #198078
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
_raw_spin_unlock_irqrestore
|
vmlinux |
0xffffffff8478a3f5
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x15
|
|
2
|
pvscsi_queue
|
vmw_pvscsi |
0xffffffffc028d632
Build id: 3.10.0-1160.15.2.el7.x86_64
|
0x4a2
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [ora_dbw0_xe:2355]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon sg pcspkr joydev parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx
drm_kms_helper syscopyarea sysfillrect nfit sysimgblt fb_sys_fops ttm drm libnvdimm e1000e ahci ata_piix libahci sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw ptp pps_core vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 2355 Comm: ora_dbw0_xe Kdump: loaded Tainted: G W L ------------ 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9e87f5fcb180 ti: ffff9e87362a0000 task.ti: ffff9e87362a0000
RIP: 0010:[<ffffffff8478a3f5>] [<ffffffff8478a3f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9e87362a39e0 EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff843b2803 RCX: ffffffff84c34f40
RDX: ffffbe5440780000 RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9e87362a39e0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff9e87eed14c00 R11: ffff9e86f54bc000 R12: 0000000000000000
R13: 0000000000000020 R14: ffff9e86c42aa780 R15: ffffffff844eb239
FS: 00007fba0f817980(0000) GS:ffff9e87ffc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcc5a05e000 CR3: 0000000130944000 CR4: 00000000007607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffc028d632>] pvscsi_queue+0x4a2/0x920 [vmw_pvscsi]