Info

Permalink
Function
_raw_spin_unlock_irqrestore
First occurrence
2020-10-27
Last occurrence
2026-03-11
State
NEW
Components
kernel
Quality
-40

Statistics

Operating system Count
CentOS 7 212
Architecture Count
x86_64 212
Related packages Count
kernel 212
 0:3.10.0-1160.el7 48
 0:3.10.0-1160.41.1.el7 23
 0:3.10.0-1062.4.1.el7 19
 0:3.10.0-957.el7 17
 0:3.10.0-862.el7 16
 0:3.10.0-1127.19.1.el7 11
 0:3.10.0-1160.71.1.el7 7
 0:3.10.0-1160.80.1.el7 7
 0:3.10.0-693.el7 7
 0:3.10.0-1127.10.1.el7 6
 0:3.10.0-1127.el7 5
 0:3.10.0-1127.18.2.el7 3
 0:3.10.0-1160.102.1.el7 3
 0:3.10.0-1160.114.2.el7 3
 0:3.10.0-1160.119.1.el7 3
 0:3.10.0-1160.76.1.el7 3
 0:3.10.0-1160.90.1.el7 3
 0:3.10.0-1062.el7 2
 0:3.10.0-1127.13.1.el7 2
 0:3.10.0-1160.108.1.el7 2
 0:3.10.0-1160.25.1.el7 2
 0:3.10.0-1160.53.1.el7 2
 0:3.10.0-1160.62.1.el7 2
 0:3.10.0-957.27.2.el7 2
 0:3.10.0-693.5.2.el7 2
 0:3.10.0-1160.105.1.el7 1
 0:3.10.0-1160.118.1.el7 1
 0:3.10.0-1160.15.2.el7 1
 0:3.10.0-1160.31.1.el7 1
 0:3.10.0-1160.36.2.el7 1
 0:3.10.0-1160.49.1.el7 1
 0:3.10.0-1160.59.1.el7 1
 0:3.10.0-1160.6.1.el7 1
 0:3.10.0-862.3.2.el7 1
 0:3.10.0-957.12.1.el7 1
 0:3.10.0-514.el7 1
 0:3.10.0-693.2.2.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1030634
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa0f8b795
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffffa0d44839
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffffa0d10c60
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x180
4
usb_hcd_poll_rh_status
vmlinux 0xffffffffa0d10b41
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x61
5
? usb_hcd_poll_rh_status
vmlinux 0xffffffffa0d10c60
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x180
6
rh_timer_func
vmlinux 0xffffffffa0d10c75
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x15
7
call_timer_fn
vmlinux 0xffffffffa08abdb8
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x38
8
? usb_hcd_poll_rh_status
vmlinux 0xffffffffa0d10c60
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x180
9
run_timer_softirq
vmlinux 0xffffffffa08ae3cd
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x25d
10
__do_softirq
vmlinux 0xffffffffa08a4bf5
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xf5
11
run_ksoftirqd
vmlinux 0xffffffffa08a4db8
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x38
12
smpboot_thread_fn
vmlinux 0xffffffffa08ce8b4
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x144
13
? lg_double_unlock
vmlinux 0xffffffffa08ce770
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40
14
kthread
vmlinux 0xffffffffa08c5e61
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xd1
15
? insert_kthread_work
vmlinux 0xffffffffa08c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40
16
ret_from_fork_nospec_begin
vmlinux 0xffffffffa0f95df7
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x21
17
? insert_kthread_work
vmlinux 0xffffffffa08c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 28s! [ksoftirqd/2:19]
Modules linked in: veeamsnap(OE) tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm iosf_mbi snd_timer crc32_pclmul snd ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper
ablk_helper soundcore cryptd ppdev vmw_balloon joydev i2c_piix4 pcspkr vmw_vmci sg parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit libnvdimm drm ata_piix libata e1000 crct10dif_pclmul crct10dif_common crc32c_intel mptspi scsi_transport_spi serio_raw mptscsih mptbase drm_panel_orientation_quirks fuse
CPU: 2 PID: 19 Comm: ksoftirqd/2 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9f29bb43d280 ti: ffff9f29bb478000 task.ti: ffff9f29bb478000
RIP: 0010:[<ffffffffa0f8b795>]  [<ffffffffa0f8b795>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9f29bb47bcd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffffa08e7508 RCX: 0000000000000003
RDX: 0000000000002090 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff9f29bb47bcd0 R08: 0000000000002090 R09: 000000000000080a
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa0d42103
R13: ffff9f29bb47bcd0 R14: ffff9f29f769acc0 R15: ffff9f29f769ad40
FS:  0000000000000000(0000) GS:ffff9f29f7680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001493510 CR3: 00000000b4f1a000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa0d44839>] uhci_hub_status_data+0x59/0x220
[<ffffffffa0d10c60>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffa0d10b41>] usb_hcd_poll_rh_status+0x61/0x180
[<ffffffffa0d10c60>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffa0d10c75>] rh_timer_func+0x15/0x20
[<ffffffffa08abdb8>] call_timer_fn+0x38/0x110
[<ffffffffa0d10c60>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffa08ae3cd>] run_timer_softirq+0x25d/0x340
[<ffffffffa08a4bf5>] __do_softirq+0xf5/0x280
[<ffffffffa08a4db8>] run_ksoftirqd+0x38/0x50
[<ffffffffa08ce8b4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffa08ce770>] ? lg_double_unlock+0x40/0x40
[<ffffffffa08c5e61>] kthread+0xd1/0xe0
[<ffffffffa08c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffa0f95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa08c5d90>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #990725
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff9a589785
Build id: 3.10.0-1160.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffff9a343969
Build id: 3.10.0-1160.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9a30fd90
Build id: 3.10.0-1160.el7.x86_64 		0x180
4
usb_hcd_poll_rh_status
vmlinux 0xffffffff9a30fc71
Build id: 3.10.0-1160.el7.x86_64 		0x61
5
? walk_page_test
vmlinux 0xffffffff9a006e82
Build id: 3.10.0-1160.el7.x86_64 		0x22
6
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9a30fd90
Build id: 3.10.0-1160.el7.x86_64 		0x180
7
rh_timer_func
vmlinux 0xffffffff9a30fda5
Build id: 3.10.0-1160.el7.x86_64 		0x15
8
call_timer_fn
vmlinux 0xffffffff99eabd58
Build id: 3.10.0-1160.el7.x86_64 		0x38
9
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9a30fd90
Build id: 3.10.0-1160.el7.x86_64 		0x180
10
run_timer_softirq
vmlinux 0xffffffff99eae1ed
Build id: 3.10.0-1160.el7.x86_64 		0x24d
11
__do_softirq
vmlinux 0xffffffff99ea4b95
Build id: 3.10.0-1160.el7.x86_64 		0xf5
12
run_ksoftirqd
vmlinux 0xffffffff99ea4d58
Build id: 3.10.0-1160.el7.x86_64 		0x38
13
smpboot_thread_fn
vmlinux 0xffffffff99ece674
Build id: 3.10.0-1160.el7.x86_64 		0x144
14
? lg_double_unlock
vmlinux 0xffffffff99ece530
Build id: 3.10.0-1160.el7.x86_64 		0x40
15
kthread
vmlinux 0xffffffff99ec5c21
Build id: 3.10.0-1160.el7.x86_64 		0xd1
16
? insert_kthread_work
vmlinux 0xffffffff99ec5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40
17
ret_from_fork_nospec_begin
vmlinux 0xffffffff9a593df7
Build id: 3.10.0-1160.el7.x86_64 		0x21
18
? insert_kthread_work
vmlinux 0xffffffff99ec5b50
Build id: 3.10.0-1160.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 36s! [ksoftirqd/0:6]
Modules linked in: xt_nat veth tcp_lp nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer iosf_mbi crc32_pclmul snd soundcore ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom
ata_generic pata_acpi sd_mod crc_t10dif vmwgfx crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm ata_piix mptspi scsi_transport_spi libata serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G               ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff963378775280 ti: ffff963370ff8000 task.ti: ffff963370ff8000
RIP: 0010:[<ffffffff9a589785>]  [<ffffffff9a589785>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff963370ffbcd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffff9a2eb608 RCX: 0000000000000003
RDX: 0000000000002090 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff963370ffbcd0 R08: 0000000000002090 R09: 000000000000080a
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff9a341233
R13: ffff963370ffbcd0 R14: ffffffff9a153565 R15: ffff963370ffbc48
FS:  0000000000000000(0000) GS:ffff963379600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcad24aaca0 CR3: 00000000b481c000 CR4: 00000000003607f0
Call Trace:
[<ffffffff9a343969>] uhci_hub_status_data+0x59/0x220
[<ffffffff9a30fd90>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff9a30fc71>] usb_hcd_poll_rh_status+0x61/0x180
[<ffffffff9a006e82>] ? walk_page_test+0x22/0x50
[<ffffffff9a30fd90>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff9a30fda5>] rh_timer_func+0x15/0x20
[<ffffffff99eabd58>] call_timer_fn+0x38/0x110
[<ffffffff9a30fd90>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff99eae1ed>] run_timer_softirq+0x24d/0x300
[<ffffffff99ea4b95>] __do_softirq+0xf5/0x280
[<ffffffff99ea4d58>] run_ksoftirqd+0x38/0x50
[<ffffffff99ece674>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff99ece530>] ? lg_double_unlock+0x40/0x40
[<ffffffff99ec5c21>] kthread+0xd1/0xe0
[<ffffffff99ec5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff9a593df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff99ec5b50>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #763045
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff816ab865
Build id: 3.10.0-693.2.2.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffff814c06a9
Build id: 3.10.0-693.2.2.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8148e1a0
Build id: 3.10.0-693.2.2.el7.x86_64 		0x170
4
usb_hcd_poll_rh_status
vmlinux 0xffffffff8148e08b
Build id: 3.10.0-693.2.2.el7.x86_64 		0x5b
5
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8148e1a0
Build id: 3.10.0-693.2.2.el7.x86_64 		0x170
6
rh_timer_func
vmlinux 0xffffffff8148e1ae
Build id: 3.10.0-693.2.2.el7.x86_64 		0xe
7
call_timer_fn
vmlinux 0xffffffff81097316
Build id: 3.10.0-693.2.2.el7.x86_64 		0x36
8
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8148e1a0
Build id: 3.10.0-693.2.2.el7.x86_64 		0x170
9
run_timer_softirq
vmlinux 0xffffffff8109982d
Build id: 3.10.0-693.2.2.el7.x86_64 		0x22d
10
__do_softirq
vmlinux 0xffffffff81090b3f
Build id: 3.10.0-693.2.2.el7.x86_64 		0xef
11
run_ksoftirqd
vmlinux 0xffffffff81090d08
Build id: 3.10.0-693.2.2.el7.x86_64 		0x38
12
smpboot_thread_fn
vmlinux 0xffffffff810b909f
Build id: 3.10.0-693.2.2.el7.x86_64 		0x12f
13
? lg_double_unlock
vmlinux 0xffffffff810b8f70
Build id: 3.10.0-693.2.2.el7.x86_64 		0x40
14
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.2.2.el7.x86_64 		0xcf
15
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.2.2.el7.x86_64 		0x40
16
ret_from_fork
vmlinux 0xffffffff816b4f58
Build id: 3.10.0-693.2.2.el7.x86_64 		0x58
17
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.2.2.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ksoftirqd/1:13]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat vmw_vsock_vmci_transport ebtable_broute vsock bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack fuse iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmhgfs(OE) snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ext4 mbcache ghash_clmulni_intel jbd2 ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus joydev pcspkr snd_seq snd_seq_device
snd_pcm snd_timer snd sg soundcore vmw_vmci nfit i2c_piix4 libnvdimm parport_pc parport shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi mptscsih mptbase vmxnet3 crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 13 Comm: ksoftirqd/1 Tainted: G           OEL ------------   3.10.0-693.2.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff88017614af70 ti: ffff880176164000 task.ti: ffff880176164000
RIP: 0010:[<ffffffff816ab865>]  [<ffffffff816ab865>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff880176167cd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffff810c76f5 RCX: 0000000000000003
RDX: 0000000000002050 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff880176167cd0 R08: 0000000000002050 R09: 000000000000080a
R10: 0000000000000000 R11: 0000000000000001 R12: ffffffff814bdf83
R13: ffff880176167cd0 R14: ffff8801765d8000 R15: 0000000100000000
FS:  0000000000000000(0000) GS:ffff880635440000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000562a342455ec CR3: 000000062b69e000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880176167d08 ffffffff814c06a9 ffff88063732a800 ffff88063732a8c0
0000000000000100 ffffffff8148e1a0 ffff88063732a800 ffff880176167d48
ffffffff8148e08b 000006de6700807f 0000000060378d44 ffff880176180000
Call Trace:
[<ffffffff814c06a9>] uhci_hub_status_data+0x59/0x220
[<ffffffff8148e1a0>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff8148e08b>] usb_hcd_poll_rh_status+0x5b/0x170
[<ffffffff8148e1a0>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff8148e1ae>] rh_timer_func+0xe/0x10
[<ffffffff81097316>] call_timer_fn+0x36/0x110
[<ffffffff8148e1a0>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff8109982d>] run_timer_softirq+0x22d/0x310
[<ffffffff81090b3f>] __do_softirq+0xef/0x280
[<ffffffff81090d08>] run_ksoftirqd+0x38/0x50
[<ffffffff810b909f>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f70>] ? lg_double_unlock+0x40/0x40
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f58>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #628079
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8168d7ab
Build id: 3.10.0-514.el7.x86_64 		0x1b
2
uhci_hub_status_data
vmlinux 0xffffffff814acd39
Build id: 3.10.0-514.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8147a020
Build id: 3.10.0-514.el7.x86_64 		0x170
4
usb_hcd_poll_rh_status
vmlinux 0xffffffff81479f0b
Build id: 3.10.0-514.el7.x86_64 		0x5b
5
? xen_hypercall_arch_6
vmlinux 0xffffffff81001ab7
Build id: 3.10.0-514.el7.x86_64 		0x3d7
6
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8147a020
Build id: 3.10.0-514.el7.x86_64 		0x170
7
rh_timer_func
vmlinux 0xffffffff8147a02e
Build id: 3.10.0-514.el7.x86_64 		0xe
8
call_timer_fn
vmlinux 0xffffffff81095976
Build id: 3.10.0-514.el7.x86_64 		0x36
9
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8147a020
Build id: 3.10.0-514.el7.x86_64 		0x170
10
run_timer_softirq
vmlinux 0xffffffff81098667
Build id: 3.10.0-514.el7.x86_64 		0x237
11
__do_softirq
vmlinux 0xffffffff8108f0ff
Build id: 3.10.0-514.el7.x86_64 		0xef
12
run_ksoftirqd
vmlinux 0xffffffff8108f2c8
Build id: 3.10.0-514.el7.x86_64 		0x38
13
smpboot_thread_fn
vmlinux 0xffffffff810b90af
Build id: 3.10.0-514.el7.x86_64 		0x12f
14
? lg_double_unlock
vmlinux 0xffffffff810b8f80
Build id: 3.10.0-514.el7.x86_64 		0x90
15
kthread
vmlinux 0xffffffff810b052f
Build id: 3.10.0-514.el7.x86_64 		0xcf
16
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140
17
ret_from_fork
vmlinux 0xffffffff81696518
Build id: 3.10.0-514.el7.x86_64 		0x58
18
? kthread_create_on_node
vmlinux 0xffffffff810b0460
Build id: 3.10.0-514.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 25s! [ksoftirqd/1:13]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec intel_powerclamp ac97_bus snd_seq iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_seq_device lrw gf128mul glue_helper ablk_helper snd_pcm cryptd ppdev snd_timer snd vmw_balloon soundcore pcspkr i2c_piix4 sg nfit vmw_vmci shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl
lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper mptspi scsi_transport_spi syscopyarea sysfillrect sysimgblt fb_sys_fops mptscsih mptbase ttm e1000 drm ata_piix libata i2c_core fjes
CPU: 1 PID: 13 Comm: ksoftirqd/1 Tainted: G               ------------ T 3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff880184a8af10 ti: ffff880184aa0000 task.ti: ffff880184aa0000
RIP: 0010:[<ffffffff8168d7ab>]  [<ffffffff8168d7ab>] _raw_spin_unlock_irqrestore+0x1b/0x40
RSP: 0018:ffff880184aa3cc8  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffff814aa433 RCX: 0000000000000003
RDX: 0000000000002090 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff880184aa3cd0 R08: 0000000000002090 R09: 000000000000080a
R10: 0000000000000000 R11: ffff8800af39a098 R12: ffffffff815b43d1
R13: ffff880184aa3c48 R14: ffff88022663c5c0 R15: ffff8800ad6ff300
FS:  0000000000000000(0000) GS:ffff880235640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcf93ea7000 CR3: 00000000bb92e000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff880231e98000 ffff880184aa3d08 ffffffff814acd39 ffff880231e98000
ffff880231e980c0 0000000000000100 ffffffff8147a020 ffff880231e98000
ffff880184aa3d48 ffffffff81479f0b ffffffff81001ab7 000000000ee0b221
Call Trace:
[<ffffffff814acd39>] uhci_hub_status_data+0x59/0x220
[<ffffffff8147a020>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff81479f0b>] usb_hcd_poll_rh_status+0x5b/0x170
[<ffffffff81001ab7>] ? xen_hypercall_arch_6+0x3d7/0x920
[<ffffffff8147a020>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff8147a02e>] rh_timer_func+0xe/0x10
[<ffffffff81095976>] call_timer_fn+0x36/0x110
[<ffffffff8147a020>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff81098667>] run_timer_softirq+0x237/0x340
[<ffffffff8108f0ff>] __do_softirq+0xef/0x280
[<ffffffff8108f2c8>] run_ksoftirqd+0x38/0x50
[<ffffffff810b90af>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f80>] ? lg_double_unlock+0x90/0x90
[<ffffffff810b052f>] kthread+0xcf/0xe0
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81696518>] ret_from_fork+0x58/0x90
[<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Code: 08 e8 da 11 a0 ff 5d c3 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 53 48 89 f3 0f 1f 44 00 00 66 83 07 02 48 89 df 57 9d <0f> 1f 44 00 00 5b 5d c3 0f 1f 44 00 00 8b 37 f0 66 83 07 02 f6
Complete report #462017
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffaf3817c5
Build id: 3.10.0-1062.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffffaf13c319
Build id: 3.10.0-1062.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffffaf108750
Build id: 3.10.0-1062.el7.x86_64 		0x180
4
usb_hcd_poll_rh_status
vmlinux 0xffffffffaf108631
Build id: 3.10.0-1062.el7.x86_64 		0x61
5
? create_of_modalias
vmlinux 0xffffffffaf00eb31
Build id: 3.10.0-1062.el7.x86_64 		0xa0
6
? usb_hcd_poll_rh_status
vmlinux 0xffffffffaf108750
Build id: 3.10.0-1062.el7.x86_64 		0x180
7
rh_timer_func
vmlinux 0xffffffffaf108765
Build id: 3.10.0-1062.el7.x86_64 		0x15
8
call_timer_fn
vmlinux 0xffffffffaecab238
Build id: 3.10.0-1062.el7.x86_64 		0x38
9
? usb_hcd_poll_rh_status
vmlinux 0xffffffffaf108750
Build id: 3.10.0-1062.el7.x86_64 		0x180
10
run_timer_softirq
vmlinux 0xffffffffaecad69d
Build id: 3.10.0-1062.el7.x86_64 		0x24d
11
__do_softirq
vmlinux 0xffffffffaeca41e5
Build id: 3.10.0-1062.el7.x86_64 		0xf5
12
run_ksoftirqd
vmlinux 0xffffffffaeca43a8
Build id: 3.10.0-1062.el7.x86_64 		0x38
13
smpboot_thread_fn
vmlinux 0xffffffffaeccdb04
Build id: 3.10.0-1062.el7.x86_64 		0x144
14
? lg_double_unlock
vmlinux 0xffffffffaeccd9c0
Build id: 3.10.0-1062.el7.x86_64 		0x40
15
kthread
vmlinux 0xffffffffaecc50d1
Build id: 3.10.0-1062.el7.x86_64 		0xd1
16
? insert_kthread_work
vmlinux 0xffffffffaecc5000
Build id: 3.10.0-1062.el7.x86_64 		0x40
17
ret_from_fork_nospec_begin
vmlinux 0xffffffffaf38bd37
Build id: 3.10.0-1062.el7.x86_64 		0x21
18
? insert_kthread_work
vmlinux 0xffffffffaecc5000
Build id: 3.10.0-1062.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:6]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq iosf_mbi crc32_pclmul snd_seq_device ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper snd_pcm cryptd snd_timer snd soundcore vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper
nfit syscopyarea sysfillrect libnvdimm sysimgblt ata_piix fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm libata mptspi scsi_transport_spi serio_raw e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 6 Comm: ksoftirqd/0 Tainted: G             L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff9d433a635230 ti: ffff9d433a6d0000 task.ti: ffff9d433a6d0000
RIP: 0010:[<ffffffffaf3817c5>]  [<ffffffffaf3817c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9d433a6d3cd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffffaf0e44a8 RCX: 0000000000000003
RDX: 0000000000002090 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff9d433a6d3cd0 R08: 0000000000002090 R09: 000000000000080a
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffaf139be3
R13: ffff9d433a6d3cd0 R14: ffffffffaef4cfe5 R15: ffff9d433a6d3c48
FS:  0000000000000000(0000) GS:ffff9d433b600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f25bb299000 CR3: 0000000062dde000 CR4: 00000000003607f0
Call Trace:
[<ffffffffaf13c319>] uhci_hub_status_data+0x59/0x220
[<ffffffffaf108750>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffaf108631>] usb_hcd_poll_rh_status+0x61/0x180
[<ffffffffaf00eb31>] ? create_of_modalias+0xa0/0x12f
[<ffffffffaf108750>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffaf108765>] rh_timer_func+0x15/0x20
[<ffffffffaecab238>] call_timer_fn+0x38/0x110
[<ffffffffaf108750>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffaecad69d>] run_timer_softirq+0x24d/0x300
[<ffffffffaeca41e5>] __do_softirq+0xf5/0x280
[<ffffffffaeca43a8>] run_ksoftirqd+0x38/0x50
[<ffffffffaeccdb04>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffaeccd9c0>] ? lg_double_unlock+0x40/0x40
[<ffffffffaecc50d1>] kthread+0xd1/0xe0
[<ffffffffaecc5000>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf38bd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaecc5000>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #229632
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa0115375
Build id: 3.10.0-862.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffff9fef7829
Build id: 3.10.0-862.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9fec4db0
Build id: 3.10.0-862.el7.x86_64 		0x180
4
usb_hcd_poll_rh_status
vmlinux 0xffffffff9fec4c91
Build id: 3.10.0-862.el7.x86_64 		0x61
5
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9fec4db0
Build id: 3.10.0-862.el7.x86_64 		0x180
6
rh_timer_func
vmlinux 0xffffffff9fec4dbe
Build id: 3.10.0-862.el7.x86_64 		0xe
7
call_timer_fn
vmlinux 0xffffffff9faa1828
Build id: 3.10.0-862.el7.x86_64 		0x38
8
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9fec4db0
Build id: 3.10.0-862.el7.x86_64 		0x180
9
run_timer_softirq
vmlinux 0xffffffff9faa3d1d
Build id: 3.10.0-862.el7.x86_64 		0x22d
10
__do_softirq
vmlinux 0xffffffff9fa9a945
Build id: 3.10.0-862.el7.x86_64 		0xf5
11
run_ksoftirqd
vmlinux 0xffffffff9fa9ab08
Build id: 3.10.0-862.el7.x86_64 		0x38
12
smpboot_thread_fn
vmlinux 0xffffffff9fac35e4
Build id: 3.10.0-862.el7.x86_64 		0x144
13
? lg_double_unlock
vmlinux 0xffffffff9fac34a0
Build id: 3.10.0-862.el7.x86_64 		0x40
14
kthread
vmlinux 0xffffffff9fabae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
15
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64 		0x40
16
ret_from_fork_nospec_begin
vmlinux 0xffffffffa011f637
Build id: 3.10.0-862.el7.x86_64 		0x21
17
? insert_kthread_work
vmlinux 0xffffffff9fabad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 71s! [ksoftirqd/2:19]
Modules linked in: esn_cfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul
syscopyarea crct10dif_common sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 19 Comm: ksoftirqd/2 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
task: ffff8870bb068fd0 ti: ffff8870bb070000 task.ti: ffff8870bb070000
RIP: 0010:[<ffffffffa0115375>]  [<ffffffffa0115375>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8870bb073cd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000054ab RCX: 0000000000000003
RDX: 0000000000002090 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff8870bb073cd0 R08: 0000000000002090 R09: 000000000000080a
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff9fef50e3
R13: ffff8870bb073cd0 R14: 0000000000000002 R15: 0000000077fe3f64
FS:  0000000000000000(0000) GS:ffff8870bc680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006dee1c CR3: 0000000079688000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9fef7829>] uhci_hub_status_data+0x59/0x220
[<ffffffff9fec4db0>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff9fec4c91>] usb_hcd_poll_rh_status+0x61/0x180
[<ffffffff9fec4db0>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff9fec4dbe>] rh_timer_func+0xe/0x10
[<ffffffff9faa1828>] call_timer_fn+0x38/0x110
[<ffffffff9fec4db0>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff9faa3d1d>] run_timer_softirq+0x22d/0x310
[<ffffffff9fa9a945>] __do_softirq+0xf5/0x280
[<ffffffff9fa9ab08>] run_ksoftirqd+0x38/0x50
[<ffffffff9fac35e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff9fac34a0>] ? lg_double_unlock+0x40/0x40
[<ffffffff9fabae31>] kthread+0xd1/0xe0
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa011f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9fabad60>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #23046
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa89883c5
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffffa8742ba9
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffffa870efe0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x180
4
usb_hcd_poll_rh_status
vmlinux 0xffffffffa870eec1
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x61
5
? usb_hcd_poll_rh_status
vmlinux 0xffffffffa870efe0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x180
6
rh_timer_func
vmlinux 0xffffffffa870eff5
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x15
7
call_timer_fn
vmlinux 0xffffffffa82ac7c8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x38
8
? usb_hcd_poll_rh_status
vmlinux 0xffffffffa870efe0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x180
9
run_timer_softirq
vmlinux 0xffffffffa82aec5d
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x24d
10
__do_softirq
vmlinux 0xffffffffa82a5695
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xf5
11
run_ksoftirqd
vmlinux 0xffffffffa82a5858
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x38
12
smpboot_thread_fn
vmlinux 0xffffffffa82cf0e4
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x144
13
? lg_double_unlock
vmlinux 0xffffffffa82cefa0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
14
kthread
vmlinux 0xffffffffa82c6691
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xd1
15
? insert_kthread_work
vmlinux 0xffffffffa82c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
16
ret_from_fork_nospec_begin
vmlinux 0xffffffffa8992d37
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
17
? insert_kthread_work
vmlinux 0xffffffffa82c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 37s! [ksoftirqd/2:19]
Modules linked in: tcp_lp fuse xt_nat veth nf_conntrack_netlink xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun xt_addrtype br_netfilter devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc
parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops nfit ttm libnvdimm ata_piix crct10dif_pclmul crct10dif_common drm crc32c_intel serio_raw mptspi libata e1000 scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 19 Comm: ksoftirqd/2 Tainted: G        W    L ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff9c59fc83d230 ti: ffff9c59fc858000 task.ti: ffff9c59fc858000
RIP: 0010:[<ffffffffa89883c5>]  [<ffffffffa89883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9c59fc85bcd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffff9c59fc85bc88 RCX: 0000000000000003
RDX: 0000000000002090 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff9c59fc85bcd0 R08: 0000000000002090 R09: 000000000000080a
R10: 0000000000000000 R11: ffff9c5a756ac000 R12: ffffffffa8740473
R13: ffff9c59fc85bcd0 R14: ffff9c59fc85bc50 R15: ffff9c5a4b0dd540
FS:  0000000000000000(0000) GS:ffff9c5a76680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f46f6f5d14b CR3: 0000000044410000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa8742ba9>] uhci_hub_status_data+0x59/0x220
[<ffffffffa870efe0>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffa870eec1>] usb_hcd_poll_rh_status+0x61/0x180
[<ffffffffa870efe0>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffa870eff5>] rh_timer_func+0x15/0x20
[<ffffffffa82ac7c8>] call_timer_fn+0x38/0x110
[<ffffffffa870efe0>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffffa82aec5d>] run_timer_softirq+0x24d/0x300
[<ffffffffa82a5695>] __do_softirq+0xf5/0x280
[<ffffffffa82a5858>] run_ksoftirqd+0x38/0x50
[<ffffffffa82cf0e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffffa82cefa0>] ? lg_double_unlock+0x40/0x40
[<ffffffffa82c6691>] kthread+0xd1/0xe0
[<ffffffffa82c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa8992d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa82c65c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #16819
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff816ab845
Build id: 3.10.0-693.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffff814c0499
Build id: 3.10.0-693.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8148df90
Build id: 3.10.0-693.el7.x86_64 		0x170
4
usb_hcd_poll_rh_status
vmlinux 0xffffffff8148de7b
Build id: 3.10.0-693.el7.x86_64 		0x5b
5
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8148df90
Build id: 3.10.0-693.el7.x86_64 		0x170
6
rh_timer_func
vmlinux 0xffffffff8148df9e
Build id: 3.10.0-693.el7.x86_64 		0xe
7
call_timer_fn
vmlinux 0xffffffff81097316
Build id: 3.10.0-693.el7.x86_64 		0x36
8
? usb_hcd_poll_rh_status
vmlinux 0xffffffff8148df90
Build id: 3.10.0-693.el7.x86_64 		0x170
9
run_timer_softirq
vmlinux 0xffffffff8109982d
Build id: 3.10.0-693.el7.x86_64 		0x22d
10
__do_softirq
vmlinux 0xffffffff81090b3f
Build id: 3.10.0-693.el7.x86_64 		0xef
11
run_ksoftirqd
vmlinux 0xffffffff81090d08
Build id: 3.10.0-693.el7.x86_64 		0x38
12
smpboot_thread_fn
vmlinux 0xffffffff810b909f
Build id: 3.10.0-693.el7.x86_64 		0x12f
13
? lg_double_unlock
vmlinux 0xffffffff810b8f70
Build id: 3.10.0-693.el7.x86_64 		0x40
14
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64 		0xcf
15
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40
16
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64 		0x58
17
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [ksoftirqd/0:3]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun binfmt_misc ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event ppdev snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul ac97_bus snd_seq snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr snd_timer
snd sg vmw_vmci soundcore shpchp i2c_piix4 nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix serio_raw mptspi libata scsi_transport_spi mptscsih e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 3 Comm: ksoftirqd/0 Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff88003bed1fa0 ti: ffff88003b800000 task.ti: ffff88003b800000
RIP: 0010:[<ffffffff816ab845>]  [<ffffffff816ab845>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff88003b803cd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffff810c149d RCX: 0000000000000003
RDX: 0000000000002090 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff88003b803cd0 R08: 0000000000002090 R09: 000000000000080a
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff814bdd73
R13: ffff88003b803cd0 R14: ffff88003b860000 R15: ffffffff810c1309
FS:  0000000000000000(0000) GS:ffff88003c600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000556f0bb65090 CR3: 0000000033d03000 CR4: 00000000003407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88003b803d08 ffffffff814c0499 ffff880035ccb000 ffff880035ccb0c0
0000000000000100 ffffffff8148df90 ffff880035ccb000 ffff88003b803d48
ffffffff8148de7b 0000000e370051e1 0000000096225287 ffffffff81e87180
Call Trace:
[<ffffffff814c0499>] uhci_hub_status_data+0x59/0x220
[<ffffffff8148df90>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff8148de7b>] usb_hcd_poll_rh_status+0x5b/0x170
[<ffffffff8148df90>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff8148df9e>] rh_timer_func+0xe/0x10
[<ffffffff81097316>] call_timer_fn+0x36/0x110
[<ffffffff8148df90>] ? usb_hcd_poll_rh_status+0x170/0x170
[<ffffffff8109982d>] run_timer_softirq+0x22d/0x310
[<ffffffff81090b3f>] __do_softirq+0xef/0x280
[<ffffffff81090d08>] run_ksoftirqd+0x38/0x50
[<ffffffff810b909f>] smpboot_thread_fn+0x12f/0x180
[<ffffffff810b8f70>] ? lg_double_unlock+0x40/0x40
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #11671
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff93b883a5
Build id: 3.10.0-1127.el7.x86_64 		0x15
2
uhci_hub_status_data
vmlinux 0xffffffff939429c9
Build id: 3.10.0-1127.el7.x86_64 		0x59
3
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9390ee00
Build id: 3.10.0-1127.el7.x86_64 		0x180
4
usb_hcd_poll_rh_status
vmlinux 0xffffffff9390ece1
Build id: 3.10.0-1127.el7.x86_64 		0x61
5
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9390ee00
Build id: 3.10.0-1127.el7.x86_64 		0x180
6
rh_timer_func
vmlinux 0xffffffff9390ee15
Build id: 3.10.0-1127.el7.x86_64 		0x15
7
call_timer_fn
vmlinux 0xffffffff934ac7c8
Build id: 3.10.0-1127.el7.x86_64 		0x38
8
? usb_hcd_poll_rh_status
vmlinux 0xffffffff9390ee00
Build id: 3.10.0-1127.el7.x86_64 		0x180
9
run_timer_softirq
vmlinux 0xffffffff934aec5d
Build id: 3.10.0-1127.el7.x86_64 		0x24d
10
__do_softirq
vmlinux 0xffffffff934a5695
Build id: 3.10.0-1127.el7.x86_64 		0xf5
11
run_ksoftirqd
vmlinux 0xffffffff934a5858
Build id: 3.10.0-1127.el7.x86_64 		0x38
12
smpboot_thread_fn
vmlinux 0xffffffff934cf0e4
Build id: 3.10.0-1127.el7.x86_64 		0x144
13
? lg_double_unlock
vmlinux 0xffffffff934cefa0
Build id: 3.10.0-1127.el7.x86_64 		0x40
14
kthread
vmlinux 0xffffffff934c6691
Build id: 3.10.0-1127.el7.x86_64 		0xd1
15
? insert_kthread_work
vmlinux 0xffffffff934c65c0
Build id: 3.10.0-1127.el7.x86_64 		0x40
16
ret_from_fork_nospec_begin
vmlinux 0xffffffff93b92d37
Build id: 3.10.0-1127.el7.x86_64 		0x21
17
? insert_kthread_work
vmlinux 0xffffffff934c65c0
Build id: 3.10.0-1127.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ksoftirqd/0:6]
Modules linked in: tcp_lp nls_utf8 isofs rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock bnep ext4 mbcache iosf_mbi kvm_intel kvm jbd2 irqbypass crc32_pclmul btusb btrtl btbcm btintel ghash_clmulni_intel aesni_intel lrw bluetooth gf128mul glue_helper ppdev ablk_helper cryptd rfkill pcspkr vmw_balloon joydev sg parport_pc parport i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm nfit libnvdimm ata_piix serio_raw libata mptspi scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 6 Comm: ksoftirqd/0 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8b34ba635230 ti: ffff8b34ba6f4000 task.ti: ffff8b34ba6f4000
RIP: 0010:[<ffffffff93b883a5>]  [<ffffffff93b883a5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0000:ffff8b34ba6f7cd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffff8b34b9f4a098 RCX: 0000000000000003
RDX: 0000000000002050 RSI: 0000000000000202 RDI: 0000000000000202
RBP: ffff8b34ba6f7cd0 R08: 0000000000002050 R09: 000000000000080a
R10: 0000000000000000 R11: ffff8b347fbcc000 R12: ffffffff93940293
R13: ffff8b34ba6f7cd0 R14: ffff8b34ba6f7cd8 R15: ffff8b34b84dc8c0
FS:  0000000000000000(0000) GS:ffff8b34bb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000dff0d000 CR3: 00000000789e8000 CR4: 00000000003607f0
Call Trace:
[<ffffffff939429c9>] uhci_hub_status_data+0x59/0x220
[<ffffffff9390ee00>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff9390ece1>] usb_hcd_poll_rh_status+0x61/0x180
[<ffffffff9390ee00>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff9390ee15>] rh_timer_func+0x15/0x20
[<ffffffff934ac7c8>] call_timer_fn+0x38/0x110
[<ffffffff9390ee00>] ? usb_hcd_poll_rh_status+0x180/0x180
[<ffffffff934aec5d>] run_timer_softirq+0x24d/0x300
[<ffffffff934a5695>] __do_softirq+0xf5/0x280
[<ffffffff934a5858>] run_ksoftirqd+0x38/0x50
[<ffffffff934cf0e4>] smpboot_thread_fn+0x144/0x1a0
[<ffffffff934cefa0>] ? lg_double_unlock+0x40/0x40
[<ffffffff934c6691>] kthread+0xd1/0xe0
[<ffffffff934c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff93b92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff934c65c0>] ? insert_kthread_work+0x40/0x40