Complete report #250216
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
|
vmlinux |
0xffffffff89586d4f
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x3af
|
|
2
|
|
vmlinux |
0xffffffff8958f975
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x35
|
|
3
|
schedule_user
|
vmlinux |
0xffffffff89588091
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x31
|
|
4
|
retint_careful
|
vmlinux |
0xffffffff8958b516
Build id: 3.10.0-1160.6.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 795s! [yumBackend.py:15160]
INFO: rcu_sched self-detected stall on CPU { 1} (t=853343 jiffies g=6029067 c=6029066 q=9)
Task dump for CPU 0:
yumBackend.py R running task 0 15160 3067 0x00000088
Call Trace:
[<ffffffff89586d4f>] ? __schedule+0x3af/0x860
[<ffffffff8958f975>] ? do_page_fault+0x35/0x90
[<ffffffff89588091>] schedule_user+0x31/0xc0
[<ffffffff8958b516>] retint_careful+0x14/0x32
Complete report #402343
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffffa54d5137
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffffa5b80d92
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x402
|
|
3
|
|
vmlinux |
0xffffffffa543c1e4
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0xb4
|
|
4
|
schedule_user
|
vmlinux |
0xffffffffa5b82061
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x31
|
|
5
|
retint_careful
|
vmlinux |
0xffffffffa5b84516
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [elasticsearch[B:20846]
Modules linked in: veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM tun devlink overlay(T) xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_mark ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr
sg joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom vmwgfx sd_mod crc_t10dif crct10dif_generic ata_generic drm_kms_helper pata_acpi ahci syscopyarea sysfillrect sysimgblt fb_sys_fops libahci ttm ata_piix nfit drm libnvdimm crct10dif_pclmul libata crct10dif_common vmw_pvscsi serio_raw crc32c_intel drm_panel_orientation_quirks vmxnet3
CPU: 5 PID: 20846 Comm: elasticsearch[B Kdump: loaded Tainted: G L ------------ T 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9a35298ba0e0 ti: ffff9a3572b68000 task.ti: ffff9a3572b68000
RIP: 0010:[<ffffffffa54d5137>] [<ffffffffa54d5137>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff9a3572b6beb8 EFLAGS: 00000282
RAX: ffff9a32fa371070 RBX: ffffffffa5437918 RCX: 0000000000000048
RDX: 0000000000000000 RSI: ffff9a35298ba0e0 RDI: ffff9a35bfd5ac80
RBP: ffff9a3572b6bed8 R08: ffff9a3572b68000 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000007 R12: ffff9a357dfce0c0
R13: 00007f9464002618 R14: 000000008fbc3660 R15: 0000000000000618
FS: 00007f94729e4700(0000) GS:ffff9a35bfd40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f81e9568000 CR3: 00000003ce840000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa5b80d92>] __schedule+0x402/0x840
[<ffffffffa543c1e4>] ? syscall_trace_leave+0xb4/0x110
[<ffffffffa5b82061>] schedule_user+0x31/0xc0
[<ffffffffa5b84516>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 80 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #985673
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffff9feda8a7
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffffa05b7920
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x320
|
|
3
|
schedule_user
|
vmlinux |
0xffffffffa05b8b41
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x31
|
|
4
|
retint_careful
|
vmlinux |
0xffffffffa05bb556
Build id: 3.10.0-1160.90.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [falcon-sensor:6357]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_16703(E) tcp_diag udp_diag inet_diag binfmt_misc drbg ansi_cprng cmac arc4 md4 nls_utf8 cifs ccm dns_resolver bmhook(OE) tmhook(OE) falcon_lsm_pinned_16604(E) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod ext4 mbcache jbd2 xfs libcrc32c iosf_mbi
crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd joydev vmw_balloon pcspkr parport_pc sg vmw_vmci parport i2c_piix4 ip_tables btrfs xor raid6_pq sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul sysimgblt crct10dif_common fb_sys_fops crc32c_intel ttm serio_raw drm ata_piix vmxnet3 libata mptspi nfit scsi_transport_spi mptscsih libnvdimm mptbase drm_panel_orientation_quirks floppy fuse [last unloaded: falcon_kal]
CPU: 3 PID: 6357 Comm: falcon-sensor Kdump: loaded Tainted: P OEL ------------ 3.10.0-1160.90.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8f3d4f83d280 ti: ffff8f3dc120c000 task.ti: ffff8f3dc120c000
RIP: 0010:[<ffffffff9feda8a7>] [<ffffffff9feda8a7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff8f3dc120fed8 EFLAGS: 00000286
RAX: ffff8f3dc4ca3180 RBX: ffff8f3dc120fe88 RCX: 00000000c0000100
RDX: ffff8f3dc120dfd8 RSI: ffff8f3d4f83d280 RDI: ffff8f3e7fd9acc0
RBP: ffff8f3dc120fef8 R08: ffff8f3dc120c000 R09: 0000000000000001
R10: 000000000000000b R11: 0000000000000293 R12: ffff8f3dc120fe88
R13: ffffffff9fee5d08 R14: ffff8f3dc120fe40 R15: ffff8f3d4f83d2e8
FS: 00007ffad5fc5700(0000) GS:ffff8f3e7fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0e0295e000 CR3: 0000000049c3e000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa05b7920>] __schedule+0x320/0x680
[<ffffffffa05b8b41>] schedule_user+0x31/0xc0
[<ffffffffa05bb556>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #724357
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffffa0ad4ac7
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffffa118c060
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x320
|
|
3
|
schedule_user
|
vmlinux |
0xffffffffa118d251
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x31
|
|
4
|
retint_careful
|
vmlinux |
0xffffffffa1190516
Build id: 3.10.0-1160.59.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [java:4198]
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 21s! [irqbalance:920]
Modules linked in: xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock ext4 mbcache jbd2 nfit libnvdimm coretemp ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm pata_acpi mptspi drm scsi_transport_spi ata_piix mptscsih mptbase crct10dif_pclmul libata crct10dif_common crc32c_intel serio_raw vmxnet3 drm_panel_orientation_quirks
floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 15 PID: 920 Comm: irqbalance Kdump: loaded Tainted: G W L ------------ T 3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9054f6fc6300 ti: ffff9058f6c2c000 task.ti: ffff9058f6c2c000
RIP: 0010:[<ffffffffa0ad4ac7>] [<ffffffffa0ad4ac7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff9058f6c2fed8 EFLAGS: 00000286
RAX: ffff9058f09c5280 RBX: 000008ddf6c2fe88 RCX: 00000000c0000100
RDX: ffff9058f6c2dfd8 RSI: ffff9054f6fc6300 RDI: ffff9058ffd9acc0
RBP: ffff9058f6c2fef8 R08: ffff9058f6c2c000 R09: 0000000000000001
R10: 0000000000000001 R11: ffff9053dc880300 R12: ffff9058f6c2fe88
R13: ffffffffa0adfbd8 R14: ffff9058f6c2fe40 R15: ffff9054f641d680
FS: 00007f8c21257780(0000) GS:ffff9058ffd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000c80a36fc CR3: 000000036f2b4000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa118c060>] __schedule+0x320/0x680
[<ffffffffa118d251>] schedule_user+0x31/0xc0
[<ffffffffa1190516>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #699479
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffff8b6da8a7
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffff8bdb7920
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x320
|
|
3
|
schedule_user
|
vmlinux |
0xffffffff8bdb8b41
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x31
|
|
4
|
retint_careful
|
vmlinux |
0xffffffff8bdbb556
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [wpa_supplicant:2711]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vboxsf(OE) sunrpc nfit libnvdimm snd_intel8x0 snd_ac97_codec ac97_bus iosf_mbi crc32_pclmul snd_seq joydev ppdev snd_seq_device ghash_clmulni_intel aesni_intel lrw snd_pcm gf128mul glue_helper ablk_helper cryptd snd_timer snd sg parport_pc vboxguest(OE) soundcore parport video i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common ahci libahci ata_piix vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm crc32c_intel drm libata e1000 serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 2711 Comm: wpa_supplicant Kdump: loaded Tainted: G W OEL ------------ 3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9ee424e91080 ti: ffff9ee43e050000 task.ti: ffff9ee43e050000
RIP: 0010:[<ffffffff8b6da8a7>] [<ffffffff8b6da8a7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff9ee43e053ed8 EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff9ee43e053e88 RCX: 00000000c0000100
RDX: ffff9ee43e051fd8 RSI: ffff9ee424e91080 RDI: ffff9ee45fc9acc0
RBP: ffff9ee43e053ef8 R08: ffff9ee43e050000 R09: 0000000000000001
R10: 000000000000001d R11: ffff9ee445303f00 R12: ffff9ee43e053e88
R13: ffffffff8b6e5d08 R14: ffff9ee43e053e40 R15: ffff9ee424e910e8
FS: 00007f3b62155880(0000) GS:ffff9ee45fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f198e21a000 CR3: 00000005e4f20000 CR4: 00000000000206e0
Call Trace:
[<ffffffff8bdb7920>] __schedule+0x320/0x680
[<ffffffff8bdb8b41>] schedule_user+0x31/0xc0
[<ffffffff8bdbb556>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df e8 da 3c 04 00 66 90 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #467789
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffffa7cd4aa7
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffffa8388e10
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x320
|
|
3
|
schedule_user
|
vmlinux |
0xffffffffa838a001
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x31
|
|
4
|
retint_careful
|
vmlinux |
0xffffffffa838c516
Build id: 3.10.0-1160.36.2.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 41s! [X:2250]
Modules linked in: uvcvideo videobuf2_vmalloc snd_usb_audio videobuf2_memops snd_usbmidi_lib videobuf2_core videodev snd_hwdep hid_wacom xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink vmhgfs(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 aesni_intel snd_rawmidi snd_ac97_codec
vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd ac97_bus snd_seq snd_seq_device snd_pcm joydev pcspkr sg snd_timer snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit libnvdimm ata_piix libata mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 2250 Comm: X Tainted: G OE ------------ 3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8dd2b4813180 ti: ffff8dd2a558c000 task.ti: ffff8dd2a558c000
RIP: 0010:[<ffffffffa7cd4aa7>] [<ffffffffa7cd4aa7>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff8dd2a558fed8 EFLAGS: 00003286
RAX: ffff8dd1fafe5280 RBX: ffffffffa7c37882 RCX: 00000000c0000100
RDX: 0000000000000000 RSI: ffff8dd2b4813180 RDI: ffff8dd2b561acc0
RBP: ffff8dd2a558fef8 R08: ffff8dd2a558c000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8dd2a558fe88
R13: ffffffffa7cdfbb8 R14: ffff8dd2a558fe40 R15: ffff8dd2b48131e8
FS: 00007f6759cdfa00(0000) GS:ffff8dd2b5600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f26707f4000 CR3: 00000002256f6000 CR4: 00000000003607f0
Call Trace:
[<ffffffffa8388e10>] __schedule+0x320/0x680
[<ffffffffa838a001>] schedule_user+0x31/0xc0
[<ffffffffa838c516>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #143016
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffff96ad5407
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffff97185942
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x402
|
|
3
|
|
vmlinux |
0xffffffff96b3efd6
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x1e6
|
|
4
|
schedule_user
|
vmlinux |
0xffffffff97186c11
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x31
|
|
5
|
retint_careful
|
vmlinux |
0xffffffff97189516
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [ora_ofsd_xe:30794]
<EOI>
Modules linked in:
xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ppdev joydev vmxnet3 vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata nfit crct10dif_pclmul crct10dif_common libnvdimm drm crc32c_intel
serio_raw vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 30794 Comm: ora_ofsd_xe Kdump: loaded Tainted: G L ------------ 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8a5dab27d230 ti: ffff8a5d5dfd0000 task.ti: ffff8a5d5dfd0000
RIP: 0010:[<ffffffff96ad5407>] [<ffffffff96ad5407>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff8a5d5dfd3eb8 EFLAGS: 00000286
RAX: ffff8a5daa985230 RBX: ffff8a5d5dfd3e68 RCX: 00000000c0000100
RDX: ffff8a5d5dfd1fd8 RSI: ffff8a5dab27d230 RDI: ffff8a5dbfd9acc0
RBP: ffff8a5d5dfd3ed8 R08: ffff8a5d5dfd0000 R09: 0000000000000001
R10: 000000000000000f R11: 0000000000000100 R12: ffff8a5d5dfd3e68
R13: ffffffff96ae0628 R14: ffff8a5d5dfd3e20 R15: ffff8a5dab27d298
FS: 00007ff69f28a700(0000) GS:ffff8a5dbfd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7d682b2000 CR3: 00000003ddf52000 CR4: 00000000001607e0
Call Trace:
[<ffffffff97185942>] __schedule+0x402/0x840
[<ffffffff96b3efd6>] ? __audit_syscall_exit+0x1e6/0x280
[<ffffffff97186c11>] schedule_user+0x31/0xc0
[<ffffffff97189516>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #115641
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffffa58d4cf7
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffffa5f868ef
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x3af
|
|
3
|
|
vmlinux |
0xffffffffa5f8e975
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x35
|
|
4
|
schedule_user
|
vmlinux |
0xffffffffa5f87c31
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x31
|
|
5
|
retint_careful
|
vmlinux |
0xffffffffa5f8a516
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [dovecot:1981]
Modules linked in: ebtable_filter ebtables ip6table_filter ip6_tables devlink binfmt_misc overlay ipt_REJECT nf_reject_ipv4 xt_conntrack xt_multiport iptable_filter iptable_raw iptable_security iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_REDIRECT nf_nat_redirect xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sunrpc intel_pmc_core iTCO_wdt ipmi_ssif intel_powerclamp coretemp intel_rapl iTCO_vendor_support ipmi_si ipmi_devintf ipmi_msghandler iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg video pinctrl_sunrisepoint pinctrl_intel mei_me mei tpm_crb i2c_i801 ie31200_edac acpi_power_meter ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ast drm_kms_helper ahci syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libahci crct10dif_pclmul crct10dif_common crc32c_intel drm libata igb uas ptp pps_core dca i2c_algo_bit drm_panel_orientation_quirks usb_storage i2c_hid dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 1981 Comm: dovecot Tainted: G W L ------------ 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: LENOVO ThinkServer TS460 /ThinkServer TS460 , BIOS TB1TS298 05/29/2020
task: ffff9e4db67c5280 ti: ffff9e4c75274000 task.ti: ffff9e4c75274000
RIP: 0010:[<ffffffffa58d4cf7>] [<ffffffffa58d4cf7>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff9e4c75277eb8 EFLAGS: 00000286
RAX: ffff9e51aefce300 RBX: ffff9e4c75277e68 RCX: 00000000c0000100
RDX: ffff9e4c75275fd8 RSI: ffff9e4db67c5280 RDI: ffff9e54afd1acc0
RBP: ffff9e4c75277ed8 R08: ffff9e4c75274000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000420 R12: ffff9e4c75277e68
R13: ffffffffa58dfca8 R14: ffff9e4c75277e20 R15: ffff9e4da77d3900
FS: 00007fc3f5c00740(0000) GS:ffff9e54afd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f95af4f6aa0 CR3: 00000000348a4000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa5f868ef>] __schedule+0x3af/0x860
[<ffffffffa5f8e975>] ? do_page_fault+0x35/0x90
[<ffffffffa5f87c31>] schedule_user+0x31/0xc0
[<ffffffffa5f8a516>] retint_careful+0x14/0x32
Complete report #18595
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffff98ad5407
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffff99185942
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x402
|
|
3
|
schedule_user
|
vmlinux |
0xffffffff99186c11
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x31
|
|
4
|
retint_careful
|
vmlinux |
0xffffffff99189516
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [titan_monitor:17361]
Modules linked in: binfmt_misc vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper joydev sg cryptd pcspkr vmw_balloon vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops pata_acpi ttm drm mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi mptscsih ata_piix crc32c_intel serio_raw mptbase libata vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 17361 Comm: titan_monitor Kdump: loaded Tainted: G L ------------ 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8842f742c1c0 ti: ffff8842ebd50000 task.ti: ffff8842ebd50000
RIP: 0010:[<ffffffff98ad5407>] [<ffffffff98ad5407>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff8842ebd53eb8 EFLAGS: 00000286
RAX: ffff8842f95f20e0 RBX: ffff8842ebd53e68 RCX: 00000000c0000100
RDX: ffff8842ebd51fd8 RSI: ffff8842f742c1c0 RDI: ffff88437fc1acc0
RBP: ffff8842ebd53ed8 R08: ffff8842ebd50000 R09: 0000000000000001
R10: 0000000000000004 R11: 0000000000000000 R12: ffff8842ebd53e68
R13: ffffffff98ae0628 R14: ffff8842ebd53e20 R15: ffff8842f742c228
FS: 00007ff605b2d740(0000) GS:ffff88437fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcec4569000 CR3: 00000000bb6f6000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff99185942>] __schedule+0x402/0x840
[<ffffffff99186c11>] schedule_user+0x31/0xc0
[<ffffffff99189516>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #18001
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffffba0d5407
Build id: 3.10.0-1127.18.2.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffffba785942
Build id: 3.10.0-1127.18.2.el7.x86_64
|
0x402
|
|
3
|
|
vmlinux |
0xffffffffba793e7e
Build id: 3.10.0-1127.18.2.el7.x86_64
|
0xee
|
|
4
|
schedule_user
|
vmlinux |
0xffffffffba786c11
Build id: 3.10.0-1127.18.2.el7.x86_64
|
0x31
|
|
5
|
retint_careful
|
vmlinux |
0xffffffffba789516
Build id: 3.10.0-1127.18.2.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 55s! [redis-server:2194]
Modules linked in: tcp_lp fuse veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) sunrpc iosf_mbi crc32_pclmul vmwgfx ghash_clmulni_intel joydev ttm ppdev iTCO_wdt iTCO_vendor_support drm_kms_helper aesni_intel syscopyarea sysfillrect sysimgblt lrw fb_sys_fops gf128mul glue_helper drm sg ablk_helper cryptd pcspkr vboxguest(OE) lpc_ich video parport_pc parport i2c_piix4 drm_panel_orientation_quirks ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi ahci ata_piix
libahci crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw e1000 dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 2194 Comm: redis-server Kdump: loaded Tainted: G OE ------------ T 3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9fa47bac0000 ti: ffff9fa47bac8000 task.ti: ffff9fa47bac8000
RIP: 0010:[<ffffffffba0d5407>] [<ffffffffba0d5407>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff9fa47bacbeb8 EFLAGS: 00000286
RAX: ffff9fa3c47fd230 RBX: ffff9fa47bacbe68 RCX: 00000000c0000100
RDX: ffff9fa47bac9fd8 RSI: ffff9fa47bac0000 RDI: ffff9fa4e3d1acc0
RBP: ffff9fa47bacbed8 R08: ffff9fa47bac8000 R09: 0000000000000001
R10: 0000000000000003 R11: ffff9fa43db07b86 R12: ffff9fa47bacbe68
R13: ffffffffba0e0628 R14: ffff9fa47bacbe20 R15: ffff9fa4dbf7e780
FS: 00007f79dab49f80(0000) GS:ffff9fa4e3d00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000373b56ca2000 CR3: 00000000b10a2000 CR4: 00000000000606e0
Call Trace:
[<ffffffffba785942>] __schedule+0x402/0x840
[<ffffffffba793e7e>] ? apic_timer_interrupt+0xee/0x170
[<ffffffffba786c11>] schedule_user+0x31/0xc0
[<ffffffffba789516>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22
Complete report #9200
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
finish_task_switch
|
vmlinux |
0xffffffff8ccd0887
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x57
|
|
2
|
__schedule
|
vmlinux |
0xffffffff8d36778f
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x3ff
|
|
3
|
|
vmlinux |
0xffffffff8d36f915
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x35
|
|
4
|
schedule_user
|
vmlinux |
0xffffffff8d368ab1
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x31
|
|
5
|
retint_careful
|
vmlinux |
0xffffffff8d36b4f6
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x14
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [sleep:32581]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic
cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 ata_generic pata_acpi vmwgfx floppy drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi libata scsi_transport_spi drm_panel_orientation_quirks mptscsih mptbase dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 32581 Comm: sleep Kdump: loaded Tainted: G L ------------ 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff96f4b4b04100 ti: ffff96f3a1998000 task.ti: ffff96f3a1998000
RIP: 0010:[<ffffffff8ccd0887>] [<ffffffff8ccd0887>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff96f3a199beb8 EFLAGS: 00000282
RAX: ffff96f4b4a76180 RBX: 00007fe5e8e664a0 RCX: 00000000c0000100
RDX: ffff96f3a1999fd8 RSI: ffff96f4b4b04100 RDI: ffff96f4bfd1ab80
RBP: ffff96f3a199bed8 R08: ffff96f3a1998000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: 00000000000002a8
R13: ffff96f4b4b04100 R14: ffff96f3a199be80 R15: ffffffff8cdebedd
FS: 00007fe5e93cb740(0000) GS:ffff96f4bfd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe5e8e664a0 CR3: 000000005deba000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8d36778f>] __schedule+0x3ff/0x890
[<ffffffff8d36f915>] ? do_page_fault+0x35/0x90
[<ffffffff8d368ab1>] schedule_user+0x31/0xc0
[<ffffffff8d36b4f6>] retint_careful+0x14/0x32
Code: 1f 44 00 00 65 48 8b 34 25 80 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22