Info

Permalink
Function
finish_task_switch
First occurrence
2020-10-04
Last occurrence
2022-12-09
State
NEW
Components
kernel
Quality
-20

Statistics

Operating system Count
CentOS 7 101
Architecture Count
x86_64 101
Related packages Count
kernel 101
 0:3.10.0-1127.el7 17
 0:3.10.0-1160.el7 11
 0:3.10.0-693.el7 11
 0:3.10.0-862.el7 6
 0:3.10.0-1062.18.1.el7 5
 0:3.10.0-1160.95.1.el7 4
 0:3.10.0-1160.25.1.el7 3
 0:3.10.0-1160.76.1.el7 2
 0:3.10.0-1127.19.1.el7 2
 0:3.10.0-1160.119.1.el7 2
 0:3.10.0-1160.99.1.el7 2
 0:3.10.0-1127.8.2.el7 2
 0:3.10.0-862.14.4.el7 2
 0:3.10.0-957.21.3.el7 2
 0:3.10.0-957.el7 2
 0:3.10.0-1160.15.2.el7 2
 0:3.10.0-957.27.2.el7 2
 0:3.10.0-1160.59.1.el7 2
 0:3.10.0-1062.1.1.el7 1
 0:3.10.0-1062.7.1.el7 1
 0:3.10.0-1062.9.1.el7 1
 0:3.10.0-1160.36.2.el7 1
 0:3.10.0-1160.6.1.el7 1
 0:3.10.0-1160.108.1.el7 1
 0:3.10.0-1160.80.1.el7 1
 0:3.10.0-1160.92.1.el7 1
 0:3.10.0-514.26.2.el7 1
 0:3.10.0-862.2.3.el7 1
 0:3.10.0-693.21.1.el7 1
 0:3.10.0-1160.102.1.el7 1
 0:3.10.0-1160.21.1.el7 1
 0:3.10.0-1160.49.1.el7 1
 0:3.10.0-957.1.3.el7 1
 0:3.10.0-1127.10.1.el7 1
 0:3.10.0-1160.71.1.el7 1
 0:3.10.0-327.el7 1
 0:3.10.0-1062.12.1.el7 1
 0:3.10.0-1160.114.2.el7 1
 0:3.10.0-1160.83.1.el7 1
 0:3.10.0-862.3.2.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #1421111
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810bf107
Build id: 3.10.0-693.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff816a8f8d
Build id: 3.10.0-693.el7.x86_64
0x39d
3
schedule
vmlinux 0xffffffff816a94c9
Build id: 3.10.0-693.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810f5126
Build id: 3.10.0-693.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff810f5deb
Build id: 3.10.0-693.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff810b4370
Build id: 3.10.0-693.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810f5104
Build id: 3.10.0-693.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff810f7b26
Build id: 3.10.0-693.el7.x86_64
0x106
9
? set_next_entity
vmlinux 0xffffffff810cb62c
Build id: 3.10.0-693.el7.x86_64
0x3c
10
? __switch_to
vmlinux 0xffffffff810295da
Build id: 3.10.0-693.el7.x86_64
0x15a
11
SyS_futex
vmlinux 0xffffffff810f8040
Build id: 3.10.0-693.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#12 stuck for 24s! [java:19591]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter arc4 md4 nls_utf8 cifs dns_resolver vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptsas drm scsi_transport_sas
ata_piix mptscsih crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 12 PID: 19591 Comm: java Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880bf134cf10 ti: ffff880b6737c000 task.ti: ffff880b6737c000
RIP: 0010:[<ffffffff810bf107>]  [<ffffffff810bf107>] finish_task_switch+0x57/0x160
RSP: 0000:ffff880b6737fc58  EFLAGS: 00000286
RAX: ffff88016d6f8000 RBX: ffff880c0f496d40 RCX: 00000000c0000100
RDX: ffff880b6737dfd8 RSI: ffff880bf134cf10 RDI: ffff880c0f316cc0
RBP: ffff880b6737fc78 R08: ffff880b6737c000 R09: 0000000000000000
R10: 0000000000000002 R11: 0000000000000000 R12: 0000000200000000
R13: 0000000000000000 R14: ffffffff810cae7b R15: ffff880b6737fc20
FS:  00007faeac2d8700(0000) GS:ffff880c0f300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f351ff88060 CR3: 0000000bf0e86000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880035525780 ffff880c0f316cc0 ffff880beca1c4c0 000000000000000c
ffff880b6737fcd8 ffffffff816a8f8d ffff880bf134cf10 ffff880b6737ffd8
ffff880b6737ffd8 ffff880b6737ffd8 ffff880bf134cf10 ffff880bf134cf10
Call Trace:
[<ffffffff816a8f8d>] __schedule+0x39d/0x8b0
[<ffffffff816a94c9>] schedule+0x29/0x70
[<ffffffff810f5126>] futex_wait_queue_me+0xc6/0x130
[<ffffffff810f5deb>] futex_wait+0x17b/0x280
[<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
[<ffffffff810f5104>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff810f7b26>] do_futex+0x106/0x5a0
[<ffffffff810cb62c>] ? set_next_entity+0x3c/0xe0
[<ffffffff810295da>] ? __switch_to+0x15a/0x510
[<ffffffff810f8040>] SyS_futex+0x80/0x180
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 1f 44 00 00 65 48 8b 34 25 00 ce 00 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 00 ce 00 00 48 8b 98 78 01 00 00 48 85 db 74 1c 
Complete report #937235
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff82ada8a7
Build id: 3.10.0-1160.114.2.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff831b7920
Build id: 3.10.0-1160.114.2.el7.x86_64
0x320
3
schedule
vmlinux 0xffffffff831b7ca9
Build id: 3.10.0-1160.114.2.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff82b19876
Build id: 3.10.0-1160.114.2.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff82b1a67b
Build id: 3.10.0-1160.114.2.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff82acf340
Build id: 3.10.0-1160.114.2.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff82b19854
Build id: 3.10.0-1160.114.2.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff82b1c376
Build id: 3.10.0-1160.114.2.el7.x86_64
0x106
9
? pick_next_entity
vmlinux 0xffffffff82ae8699
Build id: 3.10.0-1160.114.2.el7.x86_64
0xa9
10
? set_next_entity
vmlinux 0xffffffff82ae918c
Build id: 3.10.0-1160.114.2.el7.x86_64
0x3c
11
? __switch_to
vmlinux 0xffffffff82a2c671
Build id: 3.10.0-1160.114.2.el7.x86_64
0x151
12
SyS_futex
vmlinux 0xffffffff82b1c7c0
Build id: 3.10.0-1160.114.2.el7.x86_64
0x80
13
system_call_fastpath
vmlinux 0xffffffff831c539a
Build id: 3.10.0-1160.114.2.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [java:9170]
pcnet32: ens33: transmit timed out, status 96f3, resetting
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 sb_edac snd_rawmidi coretemp snd_ac97_codec iosf_mbi crc32_pclmul ac97_bus ppdev snd_seq ghash_clmulni_intel vmw_balloon snd_seq_device snd_pcm aesni_intel lrw gf128mul glue_helper ablk_helper snd_timer cryptd
pcspkr joydev snd soundcore sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common mptspi libata crc32c_intel scsi_transport_spi serio_raw mptscsih pcnet32 mptbase mii drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 9170 Comm: java Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.114.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff8e260ae2a100 ti: ffff8e2608f0c000 task.ti: ffff8e2608f0c000
RIP: 0010:[<ffffffff82ada8a7>]  [<ffffffff82ada8a7>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff8e2608f0fc28  EFLAGS: 00000286
RAX: ffff8e1f6c595280 RBX: ffff8e2608f0fbd8 RCX: 00000000c0000100
RDX: ffff8e2608f0dfd8 RSI: ffff8e260ae2a100 RDI: ffff8e261f31acc0
RBP: ffff8e2608f0fc48 R08: ffff8e2608f0c000 R09: 0000000000000001
R10: 000000000000000d R11: 0000000000000005 R12: ffff8e2608f0fbd8
R13: ffffffff82aeb01c R14: ffff8e2608f0fbd0 R15: 0000000000000001
FS:  00007fdbbc77c700(0000) GS:ffff8e261f300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f02e3d5f100 CR3: 000000065815a000 CR4: 00000000001607e0
Call Trace:
[<ffffffff831b7920>] __schedule+0x320/0x680
[<ffffffff831b7ca9>] schedule+0x29/0x70
[<ffffffff82b19876>] futex_wait_queue_me+0xc6/0x130
[<ffffffff82b1a67b>] futex_wait+0x17b/0x290
[<ffffffff82acf340>] ? hrtimer_get_res+0x50/0x50
[<ffffffff82b19854>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff82b1c376>] do_futex+0x106/0x4d0
[<ffffffff82ae8699>] ? pick_next_entity+0xa9/0x190
[<ffffffff82ae918c>] ? set_next_entity+0x3c/0x100
[<ffffffff82a2c671>] ? __switch_to+0x151/0x590
[<ffffffff82b1c7c0>] SyS_futex+0x80/0x190
[<ffffffff831c539a>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #1420180
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff83ad49d7
Build id: 3.10.0-1160.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff84186c8f
Build id: 3.10.0-1160.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffff84187169
Build id: 3.10.0-1160.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff83b121f6
Build id: 3.10.0-1160.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff83b12f9b
Build id: 3.10.0-1160.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff83ac9860
Build id: 3.10.0-1160.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff83b121d4
Build id: 3.10.0-1160.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff83b14d06
Build id: 3.10.0-1160.el7.x86_64
0x106
9
? __check_object_size
vmlinux 0xffffffff83c4a6c8
Build id: 3.10.0-1160.el7.x86_64
0x198
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 21s! [ib_log_wr_notif:3210]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink cmac arc4 md4 nls_utf8 cifs ccm dns_resolver ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper vmw_balloon ablk_helper sg cryptd vmw_vmci parport_pc pcspkr parport joydev i2c_piix4 ip_tables xfs libcrc32c sr_mod
cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci ttm crct10dif_pclmul crct10dif_common crc32c_intel libahci drm ata_piix serio_raw nfit libata vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 3210 Comm: ib_log_wr_notif Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
task: ffff96dca93aa100 ti: ffff96dca93b4000 task.ti: ffff96dca93b4000
RIP: 0010:[<ffffffff83ad49d7>]  [<ffffffff83ad49d7>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff96dca93b7c08  EFLAGS: 00000286
RAX: ffff96dca93d3180 RBX: ffff96dca93b7bb8 RCX: 00000000c0000100
RDX: ffff96dca93b5fd8 RSI: ffff96dca93aa100 RDI: ffff96fabf35acc0
RBP: ffff96dca93b7c28 R08: ffff96dca93b4000 R09: 0000000000000001
R10: 0000000000000006 R11: 0000000000000000 R12: ffff96dca93b7bb8
R13: ffffffff83ae48bc R14: ffff96dca93b7bb0 R15: 0000000000000001
FS:  00007f831ca39700(0000) GS:ffff96fabf340000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc4c0001338 CR3: 00000001cd7dc000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff84186c8f>] __schedule+0x3af/0x860
[<ffffffff84187169>] schedule+0x29/0x70
[<ffffffff83b121f6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff83b12f9b>] futex_wait+0x17b/0x280
[<ffffffff83ac9860>] ? hrtimer_get_res+0x50/0x50
[<ffffffff83b121d4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff83b14d06>] do_futex+0x106/0x5a0
[<ffffffff83c4a6c8>] ? __check_object_size+0x198/0x250
Complete report #1408139
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff8d8d4ac7
Build id: 3.10.0-1160.59.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff8df8c060
Build id: 3.10.0-1160.59.1.el7.x86_64
0x320
3
schedule
vmlinux 0xffffffff8df8c3e9
Build id: 3.10.0-1160.59.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff8d912826
Build id: 3.10.0-1160.59.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff8d9135eb
Build id: 3.10.0-1160.59.1.el7.x86_64
0x17b
6
? kmem_cache_alloc
vmlinux 0xffffffff8da29015
Build id: 3.10.0-1160.59.1.el7.x86_64
0x35
7
? hrtimer_get_res
vmlinux 0xffffffff8d8c9aa0
Build id: 3.10.0-1160.59.1.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff8d912804
Build id: 3.10.0-1160.59.1.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff8d9152a6
Build id: 3.10.0-1160.59.1.el7.x86_64
0x106
10
? __seccomp_filter
vmlinux 0xffffffff8d95729b
Build id: 3.10.0-1160.59.1.el7.x86_64
0x5b
11
SyS_futex
vmlinux 0xffffffff8d9156f0
Build id: 3.10.0-1160.59.1.el7.x86_64
0x80
12
tracesys
vmlinux 0xffffffff8df9a226
Build id: 3.10.0-1160.59.1.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 84s! [java:29389]
Modules linked in: tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper pcspkr cryptd joydev ppdev vmw_vmci vmw_balloon
sg parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod sysimgblt fb_sys_fops ttm crc_t10dif crct10dif_generic drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 29389 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff99dbb7e60000 ti: ffff99dae7f2c000 task.ti: ffff99dae7f2c000
RIP: 0010:[<ffffffff8d8d4ac7>]  [<ffffffff8d8d4ac7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff99dae7f2fc28  EFLAGS: 00000286
RAX: ffff99db90911080 RBX: ffffffffc0901137 RCX: 00000000c0000100
RDX: ffff99dae7f2dfd8 RSI: ffff99dbb7e60000 RDI: ffff99dcbfcdacc0
RBP: ffff99dae7f2fc48 R08: ffff99dae7f2c000 R09: 0000000000000001
R10: 0000000000000001 R11: ffff99dad411ff00 R12: ffffffff8da5a148
R13: ffffffff8d8e4d1c R14: ffff99dae7f2fbd0 R15: 0000000000000001
FS:  00007f77ff2fb700(0000) GS:ffff99dcbfcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f28cac7f280 CR3: 00000003f6a1c000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8df8c060>] __schedule+0x320/0x680
[<ffffffff8df8c3e9>] schedule+0x29/0x70
[<ffffffff8d912826>] futex_wait_queue_me+0xc6/0x130
[<ffffffff8d9135eb>] futex_wait+0x17b/0x280
[<ffffffff8da29015>] ? kmem_cache_alloc+0x35/0x1f0
[<ffffffff8d8c9aa0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8d912804>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff8d9152a6>] do_futex+0x106/0x4d0
[<ffffffff8d95729b>] ? __seccomp_filter+0x5b/0x300
[<ffffffff8d9156f0>] SyS_futex+0x80/0x190
[<ffffffff8df9a226>] tracesys+0xa6/0xcc
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #1217051
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffae4d4ac7
Build id: 3.10.0-1160.59.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffaeb8c060
Build id: 3.10.0-1160.59.1.el7.x86_64
0x320
3
schedule
vmlinux 0xffffffffaeb8c3e9
Build id: 3.10.0-1160.59.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffae512826
Build id: 3.10.0-1160.59.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffae5135eb
Build id: 3.10.0-1160.59.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffae4c9aa0
Build id: 3.10.0-1160.59.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffffae512804
Build id: 3.10.0-1160.59.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffae5152a6
Build id: 3.10.0-1160.59.1.el7.x86_64
0x106
9
? ktime_get
vmlinux 0xffffffffae507c77
Build id: 3.10.0-1160.59.1.el7.x86_64
0x97
10
SyS_futex
vmlinux 0xffffffffae5156f0
Build id: 3.10.0-1160.59.1.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffffaeb99f92
Build id: 3.10.0-1160.59.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [java:12859]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 24s! [HangDetector:783]
Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev i2c_piix4 sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea crc_t10dif sysfillrect crct10dif_generic sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 783 Comm: HangDetector Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff976376806300 ti: ffff976375d24000 task.ti: ffff976375d24000
RIP: 0010:[<ffffffffae4d4ac7>]  [<ffffffffae4d4ac7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff976375d27c28  EFLAGS: 00000286
RAX: ffff9762bafca100 RBX: ffff976375d27b98 RCX: 00000000c0000100
RDX: ffff976375d25fd8 RSI: ffff976376806300 RDI: ffff97637fc9acc0
RBP: ffff976375d27c48 R08: ffff976375d24000 R09: 0000000000000001
R10: 0000000000000003 R11: 0000000000000005 R12: 0000000000000002
R13: ffffffffae4e4d1c R14: ffff976375d27bd0 R15: 0000000000000001
FS:  00007fd9b36e8700(0000) GS:ffff97637fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc9a7b44b88 CR3: 00000000366d4000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaeb8c060>] __schedule+0x320/0x680
[<ffffffffaeb8c3e9>] schedule+0x29/0x70
[<ffffffffae512826>] futex_wait_queue_me+0xc6/0x130
[<ffffffffae5135eb>] futex_wait+0x17b/0x280
[<ffffffffae4c9aa0>] ? hrtimer_get_res+0x50/0x50
[<ffffffffae512804>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffae5152a6>] do_futex+0x106/0x4d0
[<ffffffffae507c77>] ? ktime_get+0x97/0xe0
[<ffffffffae5156f0>] SyS_futex+0x80/0x190
[<ffffffffaeb99f92>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #1193047
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffb02d4ac7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffb0988a77
Build id: 3.10.0-1160.25.1.el7.x86_64
0x3c7
3
schedule
vmlinux 0xffffffffb0988f49
Build id: 3.10.0-1160.25.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffb03124d6
Build id: 3.10.0-1160.25.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffb031327b
Build id: 3.10.0-1160.25.1.el7.x86_64
0x17b
6
? ttwu_do_wakeup
vmlinux 0xffffffffb02d7449
Build id: 3.10.0-1160.25.1.el7.x86_64
0x19
7
? hrtimer_get_res
vmlinux 0xffffffffb02c99e0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffffb03124b4
Build id: 3.10.0-1160.25.1.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffffb0314fe6
Build id: 3.10.0-1160.25.1.el7.x86_64
0x106
10
SyS_futex
vmlinux 0xffffffffb0315500
Build id: 3.10.0-1160.25.1.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffffb0995f92
Build id: 3.10.0-1160.25.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 84s! [nessusd:1933]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm libnvdimm mptspi scsi_transport_spi ata_piix crct10dif_pclmul crct10dif_common crc32c_intel serio_raw libata mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 1933 Comm: nessusd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9656b918b180 ti: ffff965635468000 task.ti: ffff965635468000
RIP: 0010:[<ffffffffb02d4ac7>]  [<ffffffffb02d4ac7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff96563546bc08  EFLAGS: 00000286
RAX: ffff96583695b180 RBX: ffff96563546bbb8 RCX: 00000000c0000100
RDX: ffff965635469fd8 RSI: ffff9656b918b180 RDI: ffff96583fc9acc0
RBP: ffff96563546bc28 R08: ffff965635468000 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000000 R12: ffff96563546bbb8
R13: ffffffffb02e4bcc R14: ffff96563546bbb0 R15: 0000000000000001
FS:  00007f4a7a7ff700(0000) GS:ffff96583fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f485d18b000 CR3: 0000000236554000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb0988a77>] __schedule+0x3c7/0x870
[<ffffffffb0988f49>] schedule+0x29/0x70
[<ffffffffb03124d6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffb031327b>] futex_wait+0x17b/0x280
[<ffffffffb02d7449>] ? ttwu_do_wakeup+0x19/0xe0
[<ffffffffb02c99e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffffb03124b4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffb0314fe6>] do_futex+0x106/0x5a0
[<ffffffffb0315500>] SyS_futex+0x80/0x190
[<ffffffffb0995f92>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #1163212
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810bf107
Build id: 3.10.0-693.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff816a8f8d
Build id: 3.10.0-693.el7.x86_64
0x39d
3
schedule
vmlinux 0xffffffff816a94c9
Build id: 3.10.0-693.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810f5126
Build id: 3.10.0-693.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff810f5deb
Build id: 3.10.0-693.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff810b4370
Build id: 3.10.0-693.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810f5104
Build id: 3.10.0-693.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff810f7b26
Build id: 3.10.0-693.el7.x86_64
0x106
9
? set_next_entity
vmlinux 0xffffffff810cb62c
Build id: 3.10.0-693.el7.x86_64
0x3c
10
? __switch_to
vmlinux 0xffffffff810295da
Build id: 3.10.0-693.el7.x86_64
0x15a
11
SyS_futex
vmlinux 0xffffffff810f8040
Build id: 3.10.0-693.el7.x86_64
0x80
12
tracesys
vmlinux 0xffffffff816b51d2
Build id: 3.10.0-693.el7.x86_64
0xdd
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 42s! [VM Thread:2529]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmhgfs(OE) vmw_vsock_vmci_transport vsock bnep snd_seq_midi snd_seq_midi_event ext4 mbcache crc32_pclmul jbd2 ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper snd_ens1371 cryptd vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device btusb btrtl btbcm btintel snd_pcm pcspkr bluetooth joydev snd_timer snd rfkill vmw_vmci sg soundcore nfit libnvdimm i2c_piix4 shpchp parport_pc parport nfsd
auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih crc32c_intel e1000 mptbase i2c_core serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 2529 Comm: VM Thread Tainted: G           OE  ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8800ab5f6eb0 ti: ffff88020cb50000 task.ti: ffff88020cb50000
RIP: 0010:[<ffffffff810bf107>]  [<ffffffff810bf107>] finish_task_switch+0x57/0x160
RSP: 0018:ffff88020cb53c58  EFLAGS: 00000286
RAX: ffff8801d3ff9fa0 RBX: ffffffff810cdb19 RCX: 00000000c0000100
RDX: ffff88020cb51fd8 RSI: ffff8800ab5f6eb0 RDI: ffff880235696cc0
RBP: ffff88020cb53c78 R08: ffff88020cb50000 R09: ffff880236573fb9
R10: 0000000000000001 R11: 0000000000000001 R12: ffff880232e7f200
R13: ffff880232e7f200 R14: 0000000000000400 R15: ffff880235696d40
FS:  00007fd73f7f7700(0000) GS:ffff880235680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa847affa99 CR3: 000000020b66c000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88022ebd92c0 ffff880235696cc0 ffff8800ab4fa580 0000000000000002
ffff88020cb53cd8 ffffffff816a8f8d ffff8800ab5f6eb0 ffff88020cb53fd8
ffff88020cb53fd8 ffff88020cb53fd8 ffff8800ab5f6eb0 ffff8800ab5f6eb0
Call Trace:
[<ffffffff816a8f8d>] __schedule+0x39d/0x8b0
[<ffffffff816a94c9>] schedule+0x29/0x70
[<ffffffff810f5126>] futex_wait_queue_me+0xc6/0x130
[<ffffffff810f5deb>] futex_wait+0x17b/0x280
[<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
[<ffffffff810f5104>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff810f7b26>] do_futex+0x106/0x5a0
[<ffffffff810cb62c>] ? set_next_entity+0x3c/0xe0
[<ffffffff810295da>] ? __switch_to+0x15a/0x510
[<ffffffff810f8040>] SyS_futex+0x80/0x180
[<ffffffff816b51d2>] tracesys+0xdd/0xe2
Code: 1f 44 00 00 65 48 8b 34 25 00 ce 00 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 00 ce 00 00 48 8b 98 78 01 00 00 48 85 db 74 1c 
Complete report #1002234
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff9f2c9997
Build id: 3.10.0-862.3.2.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff9f91394c
Build id: 3.10.0-862.3.2.el7.x86_64
0x41c
3
? hrtimer_start_range_ns
vmlinux 0xffffffff9f2bf040
Build id: 3.10.0-862.3.2.el7.x86_64
0x1d0
4
schedule
vmlinux 0xffffffff9f913f79
Build id: 3.10.0-862.3.2.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff9f303886
Build id: 3.10.0-862.3.2.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff9f30456b
Build id: 3.10.0-862.3.2.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff9f2beba0
Build id: 3.10.0-862.3.2.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff9f303864
Build id: 3.10.0-862.3.2.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff9f3062c6
Build id: 3.10.0-862.3.2.el7.x86_64
0x106
10
? __switch_to
vmlinux 0xffffffff9f229621
Build id: 3.10.0-862.3.2.el7.x86_64
0x151
11
SyS_futex
vmlinux 0xffffffff9f3067e0
Build id: 3.10.0-862.3.2.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff9f92082f
Build id: 3.10.0-862.3.2.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [java:3002]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log dm_mod sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg ppdev pcspkr vmw_balloon joydev parport_pc parport shpchp i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi ata_piix vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm libata mptspi scsi_transport_spi
mptscsih crct10dif_pclmul crct10dif_common mptbase crc32c_intel serio_raw vmxnet3 i2c_core floppy
CPU: 1 PID: 3002 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-862.3.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9542f737dee0 ti: ffff954319ef4000 task.ti: ffff954319ef4000
RIP: 0010:[<ffffffff9f2c9997>]  [<ffffffff9f2c9997>] finish_task_switch+0x57/0x170
RSP: 0000:ffff954319ef7c08  EFLAGS: 00000286
RAX: ffff9542bb210000 RBX: ffff95433fd18b80 RCX: 00000000c0000100
RDX: ffff954319ef5fd8 RSI: ffff9542f737dee0 RDI: ffff95433fd18b00
RBP: ffff954319ef7c28 R08: ffff954319ef4000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 0000000200000000
R13: 0000000000000000 R14: ffff95433fc18b00 R15: ffffffff9f2d898c
FS:  00007f1e4e196700(0000) GS:ffff95433fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f90bffc5010 CR3: 000000061e6e8000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9f91394c>] __schedule+0x41c/0xa20
[<ffffffff9f2bf040>] ? hrtimer_start_range_ns+0x1d0/0x3c0
[<ffffffff9f913f79>] schedule+0x29/0x70
[<ffffffff9f303886>] futex_wait_queue_me+0xc6/0x130
[<ffffffff9f30456b>] futex_wait+0x17b/0x280
[<ffffffff9f2beba0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff9f303864>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff9f3062c6>] do_futex+0x106/0x5a0
[<ffffffff9f229621>] ? __switch_to+0x151/0x580
[<ffffffff9f3067e0>] SyS_futex+0x80/0x180
[<ffffffff9f92082f>] system_call_fastpath+0x1c/0x21
Code: 1f 44 00 00 65 48 8b 34 25 40 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 40 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #995821
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff9e0da794
Build id: 3.10.0-1160.83.1.el7.x86_64
0x54
2
__schedule
vmlinux 0xffffffff9e7b7920
Build id: 3.10.0-1160.83.1.el7.x86_64
0x320
3
schedule
vmlinux 0xffffffff9e7b7ca9
Build id: 3.10.0-1160.83.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff9e1196d6
Build id: 3.10.0-1160.83.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff9e11a4db
Build id: 3.10.0-1160.83.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff9e0cf230
Build id: 3.10.0-1160.83.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff9e1196b4
Build id: 3.10.0-1160.83.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff9e11c1d6
Build id: 3.10.0-1160.83.1.el7.x86_64
0x106
9
SyS_futex
vmlinux 0xffffffff9e11c620
Build id: 3.10.0-1160.83.1.el7.x86_64
0x80
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 56s! [mysqld:1780]
Modules linked in: nls_utf8 isofs loop binfmt_misc vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr vmw_balloon vmw_vmci joydev sg i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 6 PID: 1780 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-1160.83.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff8ab74877a100 ti: ffff8ab836aa4000 task.ti: ffff8ab836aa4000
RIP: 0010:[<ffffffff9e0da794>]  [<ffffffff9e0da794>] finish_task_switch+0x54/0x1c0
RSP: 0018:ffff8ab836aa7c28  EFLAGS: 00000286
RAX: ffff8ab7ba871080 RBX: ffff8ab836aa7b98 RCX: 00000000c0000100
RDX: ffff8ab836aa5fd8 RSI: ffff8ab74877a100 RDI: ffff8ab87fd9acc0
RBP: ffff8ab836aa7c48 R08: ffff8ab836aa4000 R09: 0000000000000001
R10: 000000000000000a R11: 0000000000000005 R12: 0000000000000002
R13: ffffffff9e0e90cc R14: ffff8ab836aa7bd0 R15: 0000000000000001
FS:  00007fb1ee7f3700(0000) GS:ffff8ab87fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcfb400d000 CR3: 000000011fcfa000 CR4: 00000000000007e0
Call Trace:
[<ffffffff9e7b7920>] __schedule+0x320/0x680
[<ffffffff9e7b7ca9>] schedule+0x29/0x70
[<ffffffff9e1196d6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff9e11a4db>] futex_wait+0x17b/0x290
[<ffffffff9e0cf230>] ? hrtimer_get_res+0x50/0x50
[<ffffffff9e1196b4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff9e11c1d6>] do_futex+0x106/0x4d0
[<ffffffff9e11c620>] SyS_futex+0x80/0x190
Complete report #903809
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff892da8a7
Build id: 3.10.0-1160.95.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff899b7920
Build id: 3.10.0-1160.95.1.el7.x86_64
0x320
3
schedule
vmlinux 0xffffffff899b7ca9
Build id: 3.10.0-1160.95.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff893197e6
Build id: 3.10.0-1160.95.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff8931a5eb
Build id: 3.10.0-1160.95.1.el7.x86_64
0x17b
6
? ttwu_do_wakeup
vmlinux 0xffffffff892dd239
Build id: 3.10.0-1160.95.1.el7.x86_64
0x19
7
? hrtimer_get_res
vmlinux 0xffffffff892cf340
Build id: 3.10.0-1160.95.1.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff893197c4
Build id: 3.10.0-1160.95.1.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff8931c2e6
Build id: 3.10.0-1160.95.1.el7.x86_64
0x106
10
SyS_futex
vmlinux 0xffffffff8931c730
Build id: 3.10.0-1160.95.1.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffff899c539a
Build id: 3.10.0-1160.95.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 460s! [/opt/assure1/bi:3200]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac snd_ens1371 snd_rawmidi snd_ac97_codec btusb btrtl ac97_bus btbcm btintel iosf_mbi ppdev snd_seq crc32_pclmul bluetooth ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw gf128mul rfkill snd_timer pcspkr snd glue_helper ablk_helper vmw_balloon cryptd i2c_piix4 joydev soundcore sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 412s! [VM Thread:4632]
Modules linked in: veth xt_nat xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac snd_ens1371 snd_rawmidi snd_ac97_codec btusb btrtl ac97_bus btbcm btintel iosf_mbi ppdev snd_seq crc32_pclmul bluetooth ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm lrw gf128mul rfkill snd_timer pcspkr snd glue_helper ablk_helper vmw_balloon cryptd i2c_piix4 joydev soundcore sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul
nfit crct10dif_common crc32c_intel drm libata e1000 libnvdimm nvme serio_raw nvme_core drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod synosnap(OE)
CPU: 2 PID: 4632 Comm: VM Thread Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.95.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9a1615e16300 ti: ffff9a1615dfc000 task.ti: ffff9a1615dfc000
RIP: 0010:[<ffffffff892da8a7>]  [<ffffffff892da8a7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff9a1615dffc28  EFLAGS: 00000286
RAX: ffff9a13b08b0000 RBX: ffff9a1615dffbd8 RCX: 00000000c0000100
RDX: ffff9a1615dfdfd8 RSI: ffff9a1615e16300 RDI: ffff9a1841e9acc0
RBP: ffff9a1615dffc48 R08: ffff9a1615dfc000 R09: 0000000000000001
R10: 000000000000000a R11: 0000000000000293 R12: ffff9a1615dffbd8
R13: ffffffff892e91dc R14: ffff9a1615dffbd0 R15: 0000000000000001
FS:  00007f8420178700(0000) GS:ffff9a1841e80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3262e02000 CR3: 0000000071a22000 CR4: 00000000003607e0
Call Trace:
[<ffffffff899b7920>] __schedule+0x320/0x680
[<ffffffff899b7ca9>] schedule+0x29/0x70
[<ffffffff893197e6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff8931a5eb>] futex_wait+0x17b/0x290
[<ffffffff892dd239>] ? ttwu_do_wakeup+0x19/0xf0
[<ffffffff892cf340>] ? hrtimer_get_res+0x50/0x50
[<ffffffff893197c4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff8931c2e6>] do_futex+0x106/0x4d0
[<ffffffff8931c730>] SyS_futex+0x80/0x190
[<ffffffff899c539a>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #881479
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff936d5007
Build id: 3.10.0-1062.12.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff93d805c2
Build id: 3.10.0-1062.12.1.el7.x86_64
0x402
3
schedule
vmlinux 0xffffffff93d80a29
Build id: 3.10.0-1062.12.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff93712086
Build id: 3.10.0-1062.12.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff93712e2b
Build id: 3.10.0-1062.12.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff936c9e30
Build id: 3.10.0-1062.12.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff93712064
Build id: 3.10.0-1062.12.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff93714b76
Build id: 3.10.0-1062.12.1.el7.x86_64
0x106
9
? mntput
vmlinux 0xffffffff9386dea4
Build id: 3.10.0-1062.12.1.el7.x86_64
0x24
10
? __fput
vmlinux 0xffffffff9384cc46
Build id: 3.10.0-1062.12.1.el7.x86_64
0x186
11
SyS_futex
vmlinux 0xffffffff93715090
Build id: 3.10.0-1062.12.1.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff93d8dede
Build id: 3.10.0-1062.12.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [java:2982]
Modules linked in: xt_CHECKSUM tun devlink ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr vmw_vmci sg parport_pc i2c_piix4 parport ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix serio_raw mptspi scsi_transport_spi libata mptscsih mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 2982 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff9afc44f79070 ti: ffff9afc59f7c000 task.ti: ffff9afc59f7c000
RIP: 0010:[<ffffffff936d5007>]  [<ffffffff936d5007>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff9afc59f7fc08  EFLAGS: 00000282
RAX: ffff9afc63e6d230 RBX: ffffffff93882edd RCX: 00000000c0000100
RDX: ffff9afc59f7dfd8 RSI: ffff9afc44f79070 RDI: ffff9afcbfd1ac80
RBP: ffff9afc59f7fc28 R08: ffff9afc59f7c000 R09: 0000000000000001
R10: 000000000000000e R11: 0000000000000000 R12: ffff9afcb13b4340
R13: ffffffff936e4ecc R14: ffff9afc59f7fbb0 R15: 0000000000000001
FS:  00007fae735fd700(0000) GS:ffff9afcbfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fae97e853b8 CR3: 0000000341392000 CR4: 00000000001607e0
Call Trace:
[<ffffffff93d805c2>] __schedule+0x402/0x840
[<ffffffff93d80a29>] schedule+0x29/0x70
[<ffffffff93712086>] futex_wait_queue_me+0xc6/0x130
[<ffffffff93712e2b>] futex_wait+0x17b/0x280
[<ffffffff936c9e30>] ? hrtimer_get_res+0x50/0x50
[<ffffffff93712064>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff93714b76>] do_futex+0x106/0x5a0
[<ffffffff9386dea4>] ? mntput+0x24/0x40
[<ffffffff9384cc46>] ? __fput+0x186/0x260
[<ffffffff93715090>] SyS_futex+0x80/0x190
[<ffffffff93d8dede>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 80 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #857686
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffa48c9667
Build id: 3.10.0-862.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffa4f1291c
Build id: 3.10.0-862.el7.x86_64
0x41c
3
? hrtimer_start_range_ns
vmlinux 0xffffffffa48bed10
Build id: 3.10.0-862.el7.x86_64
0x1d0
4
schedule
vmlinux 0xffffffffa4f12f49
Build id: 3.10.0-862.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffffa4903556
Build id: 3.10.0-862.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffffa490423b
Build id: 3.10.0-862.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffffa48be870
Build id: 3.10.0-862.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffffa4903534
Build id: 3.10.0-862.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffffa4905f96
Build id: 3.10.0-862.el7.x86_64
0x106
10
? __schedule
vmlinux 0xffffffffa4f12b08
Build id: 3.10.0-862.el7.x86_64
0x608
11
? __schedule
vmlinux 0xffffffffa4f12b08
Build id: 3.10.0-862.el7.x86_64
0x608
12
SyS_futex
vmlinux 0xffffffffa49064b0
Build id: 3.10.0-862.el7.x86_64
0x80
13
system_call_fastpath
vmlinux 0xffffffffa4f1f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [java:41789]
Modules linked in: tcp_lp rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event ext4 crc32_pclmul ghash_clmulni_intel mbcache jbd2 ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon snd_ens1371 btusb snd_rawmidi btrtl snd_ac97_codec btbcm btintel ac97_bus snd_seq bluetooth joydev pcspkr snd_seq_device snd_pcm snd_timer rfkill sg snd nfit vmw_vmci libnvdimm soundcore i2c_piix4 shpchp parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif
crct10dif_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi drm mptscsih ata_piix serio_raw libata e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 41789 Comm: java Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff98e3daa4eeb0 ti: ffff98e4f4384000 task.ti: ffff98e4f4384000
RIP: 0010:[<ffffffffa48c9667>]  [<ffffffffa48c9667>] finish_task_switch+0x57/0x170
RSP: 0018:ffff98e4f4387c08  EFLAGS: 00000286
RAX: ffff98e4f164af70 RBX: ffff98e4fd658b80 RCX: 00000000c0000100
RDX: ffff98e4f4385fd8 RSI: ffff98e3daa4eeb0 RDI: ffff98e4fd658b00
RBP: ffff98e4f4387c28 R08: ffff98e4f4384000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000200000000
R13: 0000000000000000 R14: ffff98e4fd658b00 R15: ffffffffa48d865c
FS:  00007fd1409fc700(0000) GS:ffff98e4fd640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd163bfb070 CR3: 0000000020fb4000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa4f1291c>] __schedule+0x41c/0xa20
[<ffffffffa48bed10>] ? hrtimer_start_range_ns+0x1d0/0x3c0
[<ffffffffa4f12f49>] schedule+0x29/0x70
[<ffffffffa4903556>] futex_wait_queue_me+0xc6/0x130
[<ffffffffa490423b>] futex_wait+0x17b/0x280
[<ffffffffa48be870>] ? hrtimer_get_res+0x50/0x50
[<ffffffffa4903534>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffa4905f96>] do_futex+0x106/0x5a0
[<ffffffffa4f12b08>] ? __schedule+0x608/0xa20
[<ffffffffa4f12b08>] ? __schedule+0x608/0xa20
[<ffffffffa49064b0>] SyS_futex+0x80/0x180
[<ffffffffa4f1f7d5>] system_call_fastpath+0x1c/0x21
Code: 1f 44 00 00 65 48 8b 34 25 40 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 40 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #856584
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810b41c6
Build id: 3.10.0-327.el7.x86_64
0x56
2
__schedule
vmlinux 0xffffffff8163a2b8
Build id: 3.10.0-327.el7.x86_64
0x2d8
3
schedule
vmlinux 0xffffffff8163a909
Build id: 3.10.0-327.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810e2464
Build id: 3.10.0-327.el7.x86_64
0xc4
5
futex_wait
vmlinux 0xffffffff810e2fd9
Build id: 3.10.0-327.el7.x86_64
0x179
6
? hrtimer_get_res
vmlinux 0xffffffff810a95c0
Build id: 3.10.0-327.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810e2442
Build id: 3.10.0-327.el7.x86_64
0xa2
8
do_futex
vmlinux 0xffffffff810e506e
Build id: 3.10.0-327.el7.x86_64
0xfe
9
SyS_futex
vmlinux 0xffffffff810e55a0
Build id: 3.10.0-327.el7.x86_64
0x80
10
system_call_fastpath
vmlinux 0xffffffff81645909
Build id: 3.10.0-327.el7.x86_64
0x16
BUG: soft lockup - CPU#0 stuck for 23s! [mysqld:2421]
Modules linked in: tcp_lp bnep bluetooth rfkill fuse vboxsf(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev pcspkr snd_timer snd soundcore i2c_piix4 parport_pc video parport sg vboxguest(OE) nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper ttm ahci ata_piix libahci drm crc32c_intel serio_raw e1000 i2c_core libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 2421 Comm: mysqld Tainted: G           OEL ------------   3.10.0-327.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8800cfa28000 ti: ffff8800c08fc000 task.ti: ffff8800c08fc000
RIP: 0010:[<ffffffff810b41c6>]  [<ffffffff810b41c6>] finish_task_switch+0x56/0x170
RSP: 0018:ffff8800c08ffc50  EFLAGS: 00010246
RAX: ffff880057e53980 RBX: ffff88010ee14780 RCX: 0000000000000000
RDX: ffff8800c08fdfd8 RSI: 00000000e3d2e3d0 RDI: ffff88010ee14780
RBP: ffff8800c08ffc70 R08: ffff8800c08fc000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000057 R12: ffff8800cfa28068
R13: ffff88010ee14800 R14: 0000000100000000 R15: 0000000000157110
FS:  00007f79a2cba700(0000) GS:ffff88010ee00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00001f55e3109000 CR3: 00000001079ef000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8800d7d00640 ffff88010ee14780 ffff880035e3c000 0000000000000000
ffff8800c08ffcd0 ffffffff8163a2b8 ffff8800cfa28000 ffff8800c08fffd8
ffff8800c08fffd8 ffff8800c08fffd8 ffff8800cfa28000 ffff8800cfa28000
Call Trace:
[<ffffffff8163a2b8>] __schedule+0x2d8/0x900
[<ffffffff8163a909>] schedule+0x29/0x70
[<ffffffff810e2464>] futex_wait_queue_me+0xc4/0x120
[<ffffffff810e2fd9>] futex_wait+0x179/0x280
[<ffffffff810a95c0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff810e2442>] ? futex_wait_queue_me+0xa2/0x120
[<ffffffff810e506e>] do_futex+0xfe/0x5b0
[<ffffffff810e55a0>] SyS_futex+0x80/0x180
[<ffffffff81645909>] system_call_fastpath+0x16/0x1b
Code: 0f 1f 44 00 00 65 48 8b 34 25 c0 b7 00 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 e9 75 00 00 00 66 83 03 02 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 b7 00 00 48 8b 98 78 01 00 00 48 85 db 74 1e 
Complete report #803481
Frame # Function Binary Source or offset Line
1
__schedule
vmlinux 0xffffffffadfb7606
Build id: 3.10.0-1160.95.1.el7.x86_64
0x6
2
? schedule
vmlinux 0xffffffffadfb7ca9
Build id: 3.10.0-1160.95.1.el7.x86_64
0x29
3
futex_wait_queue_me
vmlinux 0xffffffffad9197e6
Build id: 3.10.0-1160.95.1.el7.x86_64
0xc6
4
futex_wait
vmlinux 0xffffffffad91a5eb
Build id: 3.10.0-1160.95.1.el7.x86_64
0x17b
5
? hrtimer_get_res
vmlinux 0xffffffffad8cf340
Build id: 3.10.0-1160.95.1.el7.x86_64
0x50
6
? futex_wait_queue_me
vmlinux 0xffffffffad9197c4
Build id: 3.10.0-1160.95.1.el7.x86_64
0xa4
7
do_futex
vmlinux 0xffffffffad91c2e6
Build id: 3.10.0-1160.95.1.el7.x86_64
0x106
8
SyS_futex
vmlinux 0xffffffffad91c730
Build id: 3.10.0-1160.95.1.el7.x86_64
0x80
9
system_call_fastpath
vmlinux 0xffffffffadfc539a
Build id: 3.10.0-1160.95.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 3072s! [Compress Proven:5031]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ip6t_REJECT devlink nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc vfat fat snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device iTCO_wdt snd_pcm iTCO_vendor_support joydev snd_timer pcspkr sg i2c_i801 virtio_rng lpc_ich snd soundcore ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common
virtio_gpu drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm virtio_console drm_panel_orientation_quirks uas usb_storage ahci libahci libata virtio_pci virtio_ring e1000 virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 8 PID: 5031 Comm: Compress Proven Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.95.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown 2/2/2022
task: ffff8f5a841a2100 ti: ffff8f5a8673c000 task.ti: ffff8f5a8673c000
RIP: 0010:[<ffffffffadfb7606>]  [<ffffffffadfb7606>] __schedule+0x6/0x680
RSP: 0018:ffff8f5a8673fca8  EFLAGS: 00000246
RAX: ffff8f5a841a2100 RBX: ffff8f5a8673fd38 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000287 RDI: 0000000000000000
RBP: ffff8f5a8673fcb8 R08: ffff8f5a89a16160 R09: 0000000000000000
R10: 000000000000c1bd R11: ffff8f5a8673ff00 R12: ffffffffadba73d0
R13: ffff8f5a8673fc28 R14: ffff8f5a841a2798 R15: 0000000189a1acc0
FS:  00007ff68bafc700(0000) GS:ffff8f5a89a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9d6662d680 CR3: 000000007df30000 CR4: 00000000000026e0
Call Trace:
[<ffffffffadfb7ca9>] ? schedule+0x29/0x70
[<ffffffffad9197e6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffad91a5eb>] futex_wait+0x17b/0x290
[<ffffffffad8cf340>] ? hrtimer_get_res+0x50/0x50
[<ffffffffad9197c4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffad91c2e6>] do_futex+0x106/0x4d0
[<ffffffffad91c730>] SyS_futex+0x80/0x190
[<ffffffffadfc539a>] system_call_fastpath+0x25/0x2a
Code: 00 00 00 00 00 0f 1f 44 00 00 55 48 bf ff ff ff ff ff ff ff 7f 48 89 e5 e8 a8 fe ff ff 5d c3 cc cc cc cc 66 90 0f 1f 44 00 00 55 <48> c7 c0 c0 ac 01 00 48 89 e5 41 57 41 56 41 55 41 54 53 48 83 
Complete report #746462
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff8d6d0887
Build id: 3.10.0-957.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff8dd6770f
Build id: 3.10.0-957.el7.x86_64
0x3ff
3
? hrtimer_start_range_ns
vmlinux 0xffffffff8d6c5cad
Build id: 3.10.0-957.el7.x86_64
0x1ed
4
schedule
vmlinux 0xffffffff8dd67bc9
Build id: 3.10.0-957.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff8d70ce56
Build id: 3.10.0-957.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff8d70db3b
Build id: 3.10.0-957.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff8d6c57e0
Build id: 3.10.0-957.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff8d70ce34
Build id: 3.10.0-957.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff8d70f886
Build id: 3.10.0-957.el7.x86_64
0x106
10
? recalc_sigpending
vmlinux 0xffffffff8d6ac4ab
Build id: 3.10.0-957.el7.x86_64
0x1b
11
SyS_futex
vmlinux 0xffffffff8d70fda0
Build id: 3.10.0-957.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff8dd74ddb
Build id: 3.10.0-957.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 29s! [java:25052]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 29s! [java:24871]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod ppdev snd_seq_midi snd_seq_midi_event coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common
crc32c_intel serio_raw ata_generic vmwgfx pata_acpi drm_kms_helper e1000 syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi mptscsih drm_panel_orientation_quirks mptbase ata_piix libata floppy
CPU: 0 PID: 24871 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff9c0cf3b82080 ti: ffff9c0bcb084000 task.ti: ffff9c0bcb084000
RIP: 0010:[<ffffffff8d6d0887>]  [<ffffffff8d6d0887>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff9c0bcb087c08  EFLAGS: 00000282
RAX: ffff9c0c73edc100 RBX: ffff9c0cfa120000 RCX: 00000000c0000100
RDX: ffff9c0bcb085fd8 RSI: ffff9c0cf3b82080 RDI: ffff9c0cfa61ab80
RBP: ffff9c0bcb087c28 R08: ffff9c0bcb084000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000200000000
R13: 0000000000000000 R14: ffff9c0cfa61ab80 R15: 0000000000000001
FS:  00007f23dd3ba700(0000) GS:ffff9c0cfa600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f23ba619668 CR3: 0000000135e6e000 CR4: 00000000001607f0
Call Trace:
[<ffffffff8dd6770f>] __schedule+0x3ff/0x890
[<ffffffff8d6c5cad>] ? hrtimer_start_range_ns+0x1ed/0x3c0
[<ffffffff8dd67bc9>] schedule+0x29/0x70
[<ffffffff8d70ce56>] futex_wait_queue_me+0xc6/0x130
[<ffffffff8d70db3b>] futex_wait+0x17b/0x280
[<ffffffff8d6c57e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8d70ce34>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff8d70f886>] do_futex+0x106/0x5a0
[<ffffffff8d6ac4ab>] ? recalc_sigpending+0x1b/0x70
[<ffffffff8d70fda0>] SyS_futex+0x80/0x190
[<ffffffff8dd74ddb>] system_call_fastpath+0x22/0x27
Code: 1f 44 00 00 65 48 8b 34 25 80 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #738304
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffab4d49d7
Build id: 3.10.0-1160.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffabb86c8f
Build id: 3.10.0-1160.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffffabb87169
Build id: 3.10.0-1160.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffab5121f6
Build id: 3.10.0-1160.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffab512f9b
Build id: 3.10.0-1160.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffab4c9860
Build id: 3.10.0-1160.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffffab5121d4
Build id: 3.10.0-1160.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffab514d06
Build id: 3.10.0-1160.el7.x86_64
0x106
9
? sched_clock_cpu
vmlinux 0xffffffffab4de185
Build id: 3.10.0-1160.el7.x86_64
0x85
10
SyS_futex
vmlinux 0xffffffffab515220
Build id: 3.10.0-1160.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffffabb93f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [Wt-640:2267]
scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 2267 Comm: Wt-640 Kdump: loaded Not tainted 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8ac1b339b180 ti: ffff8ac408054000 task.ti: ffff8ac408054000
RIP: 0010:[<ffffffffab4d49d7>]  [<ffffffffab4d49d7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff8ac408057c08  EFLAGS: 00000286
RAX: ffff8ac44c0e1080 RBX: ffff8ac408057bb8 RCX: 00000000c0000102
RDX: ffff8ac408055fd8 RSI: ffff8ac1b339b180 RDI: ffff8ac46f29acc0
RBP: ffff8ac408057c28 R08: ffff8ac408054000 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000000 R12: ffff8ac408057bb8
R13: 0000000300000020 R14: 0000000400000008 R15: ffff8ac1bfd18c00
FS:  00007f020541c700(0000) GS:ffff8ac46f280000(0000) knlGS:0000000b69820000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1c5ead29e5 CR3: 000000042dd56000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffabb86c8f>] __schedule+0x3af/0x860
[<ffffffffabb87169>] schedule+0x29/0x70
[<ffffffffab5121f6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffab512f9b>] futex_wait+0x17b/0x280
[<ffffffffab4c9860>] ? hrtimer_get_res+0x50/0x50
[<ffffffffab5121d4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffab514d06>] do_futex+0x106/0x5a0
[<ffffffffab4de185>] ? sched_clock_cpu+0x85/0xc0
[<ffffffffab515220>] SyS_futex+0x80/0x190
[<ffffffffabb93f92>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #734752
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffbbed49d7
Build id: 3.10.0-1160.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffbc586c8f
Build id: 3.10.0-1160.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffffbc587169
Build id: 3.10.0-1160.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffbbf121f6
Build id: 3.10.0-1160.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffbbf12f9b
Build id: 3.10.0-1160.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffbbec9860
Build id: 3.10.0-1160.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffffbbf121d4
Build id: 3.10.0-1160.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffbbf14d06
Build id: 3.10.0-1160.el7.x86_64
0x106
9
? mntput
vmlinux 0xffffffffbc071bb4
Build id: 3.10.0-1160.el7.x86_64
0x24
10
? __fput
vmlinux 0xffffffffbc04fef5
Build id: 3.10.0-1160.el7.x86_64
0x195
11
SyS_futex
vmlinux 0xffffffffbbf15220
Build id: 3.10.0-1160.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffffbc593f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [python:4474]
Modules linked in: binfmt_misc tcp_lp rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink cbsensor_2_5_1061973(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 snd_rawmidi btusb aesni_intel snd_ac97_codec btrtl btbcm btintel lrw
ac97_bus gf128mul glue_helper vmw_balloon snd_seq ablk_helper cryptd bluetooth snd_seq_device snd_pcm joydev pcspkr snd_timer rfkill sg snd vmw_vmci soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit crct10dif_pclmul crct10dif_common libnvdimm crc32c_intel mptspi scsi_transport_spi libata e1000 mptscsih serio_raw mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 4474 Comm: python Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8dc3d4aa3180 ti: ffff8dc440d14000 task.ti: ffff8dc440d14000
RIP: 0010:[<ffffffffbbed49d7>]  [<ffffffffbbed49d7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff8dc440d17c08  EFLAGS: 00000286
RAX: ffff8dc445386300 RBX: ffff8dc440d17bb8 RCX: 00000000c0000100
RDX: ffff8dc440d15fd8 RSI: ffff8dc3d4aa3180 RDI: ffff8dc47a61acc0
RBP: ffff8dc440d17c28 R08: ffff8dc440d14000 R09: 0000000000000001
R10: 0000000000000013 R11: 0000000000000293 R12: ffff8dc440d17bb8
R13: ffffffffbbee48bc R14: ffff8dc440d17bb0 R15: 0000000000000001
FS:  00007f5d27958700(0000) GS:ffff8dc47a600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f58826a3280 CR3: 000000005ce54000 CR4: 00000000003607f0
Call Trace:
[<ffffffffbc586c8f>] __schedule+0x3af/0x860
[<ffffffffbc587169>] schedule+0x29/0x70
[<ffffffffbbf121f6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffbbf12f9b>] futex_wait+0x17b/0x280
[<ffffffffbbec9860>] ? hrtimer_get_res+0x50/0x50
[<ffffffffbbf121d4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffbbf14d06>] do_futex+0x106/0x5a0
[<ffffffffbc071bb4>] ? mntput+0x24/0x40
[<ffffffffbc04fef5>] ? __fput+0x195/0x230
[<ffffffffbbf15220>] SyS_futex+0x80/0x190
[<ffffffffbc593f92>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #687583
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffaaad49d7
Build id: 3.10.0-1160.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffab186c8f
Build id: 3.10.0-1160.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffffab187169
Build id: 3.10.0-1160.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffaab121f6
Build id: 3.10.0-1160.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffaab12f9b
Build id: 3.10.0-1160.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffaaac9860
Build id: 3.10.0-1160.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffffaab121d4
Build id: 3.10.0-1160.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffaab14d06
Build id: 3.10.0-1160.el7.x86_64
0x106
9
? __seccomp_filter
vmlinux 0xffffffffaab56dfb
Build id: 3.10.0-1160.el7.x86_64
0x5b
10
SyS_futex
vmlinux 0xffffffffaab15220
Build id: 3.10.0-1160.el7.x86_64
0x80
11
tracesys
vmlinux 0xffffffffab194226
Build id: 3.10.0-1160.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [C1 CompilerThre:32347]
Modules linked in: xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr sg vmw_balloon joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs
libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt sd_mod fb_sys_fops ttm crc_t10dif crct10dif_generic drm ahci libahci ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 32347 Comm: C1 CompilerThre Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff980f4d52a100 ti: ffff981082a8c000 task.ti: ffff981082a8c000
RIP: 0010:[<ffffffffaaad49d7>]  [<ffffffffaaad49d7>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff981082a8fc08  EFLAGS: 00000286
RAX: ffff981039f7d280 RBX: ffff981082a8fbb8 RCX: 00000000c0000100
RDX: ffff981082a8dfd8 RSI: ffff980f4d52a100 RDI: ffff9812ffd9acc0
RBP: ffff981082a8fc28 R08: ffff981082a8c000 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000001 R12: ffff981082a8fbb8
R13: ffffffffaaae48bc R14: ffff981082a8fbb0 R15: 0000000000000001
FS:  00007f083ee7c700(0000) GS:ffff9812ffd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5df26e6000 CR3: 00000001f06c6000 CR4: 00000000001607e0
Call Trace:
[<ffffffffab186c8f>] __schedule+0x3af/0x860
[<ffffffffab187169>] schedule+0x29/0x70
[<ffffffffaab121f6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffaab12f9b>] futex_wait+0x17b/0x280
[<ffffffffaaac9860>] ? hrtimer_get_res+0x50/0x50
[<ffffffffaab121d4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffaab14d06>] do_futex+0x106/0x5a0
[<ffffffffaab56dfb>] ? __seccomp_filter+0x5b/0x300
[<ffffffffaab15220>] SyS_futex+0x80/0x190
[<ffffffffab194226>] tracesys+0xa6/0xcc
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:37:crtc-0] flip_done timed out
Complete report #635840
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffa8ad49d7
Build id: 3.10.0-1160.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffa9186c8f
Build id: 3.10.0-1160.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffffa9187169
Build id: 3.10.0-1160.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffa8b121f6
Build id: 3.10.0-1160.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffa8b12f9b
Build id: 3.10.0-1160.el7.x86_64
0x17b
6
? ttwu_do_wakeup
vmlinux 0xffffffffa8ad7229
Build id: 3.10.0-1160.el7.x86_64
0x19
7
? hrtimer_get_res
vmlinux 0xffffffffa8ac9860
Build id: 3.10.0-1160.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffffa8b121d4
Build id: 3.10.0-1160.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffffa8b14d06
Build id: 3.10.0-1160.el7.x86_64
0x106
10
? __seccomp_filter
vmlinux 0xffffffffa8b56dfb
Build id: 3.10.0-1160.el7.x86_64
0x5b
11
? __switch_to
vmlinux 0xffffffffa8a2b981
Build id: 3.10.0-1160.el7.x86_64
0x4b1
12
SyS_futex
vmlinux 0xffffffffa8b15220
Build id: 3.10.0-1160.el7.x86_64
0x80
13
tracesys
vmlinux 0xffffffffa9194226
Build id: 3.10.0-1160.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 63s! [gitaly:43867]
Modules linked in: tcp_lp veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi ppdev snd_ac97_codec ac97_bus aesni_intel
snd_seq lrw gf128mul snd_seq_device glue_helper ablk_helper vmw_balloon snd_pcm cryptd snd_timer joydev pcspkr snd soundcore i2c_piix4 sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ata_generic pata_acpi crct10dif_pclmul mptspi fb_sys_fops crct10dif_common ttm scsi_transport_spi drm ata_piix e1000 mptscsih crc32c_intel serio_raw mptbase nfit libnvdimm libata drm_panel_orientation_quirks fuse
CPU: 1 PID: 43867 Comm: gitaly Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff889597300000 ti: ffff889597114000 task.ti: ffff889597114000
RIP: 0010:[<ffffffffa8ad49d7>]  [<ffffffffa8ad49d7>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff889597117c08  EFLAGS: 00000286
RAX: ffff8895818d0000 RBX: ffffffffa8a37938 RCX: 0000000000000048
RDX: 0000000000000000 RSI: ffff889597300000 RDI: ffff8897af25acc0
RBP: ffff889597117c28 R08: ffff889597114000 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000090 R12: ffff889597117bb8
R13: ffff8897af25ad40 R14: ffff889713ef5d40 R15: 0000000000000001
FS:  00007fd8fffff700(0000) GS:ffff8897af240000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f44aa3e3000 CR3: 000000011c020000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa9186c8f>] __schedule+0x3af/0x860
[<ffffffffa9187169>] schedule+0x29/0x70
[<ffffffffa8b121f6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffa8b12f9b>] futex_wait+0x17b/0x280
[<ffffffffa8ad7229>] ? ttwu_do_wakeup+0x19/0xe0
[<ffffffffa8ac9860>] ? hrtimer_get_res+0x50/0x50
[<ffffffffa8b121d4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffa8b14d06>] do_futex+0x106/0x5a0
[<ffffffffa8b56dfb>] ? __seccomp_filter+0x5b/0x300
[<ffffffffa8a2b981>] ? __switch_to+0x4b1/0x580
[<ffffffffa8b15220>] SyS_futex+0x80/0x190
[<ffffffffa9194226>] tracesys+0xa6/0xcc
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #603708
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810bf104
Build id: 3.10.0-693.el7.x86_64
0x54
2
__schedule
vmlinux 0xffffffff816a8f8d
Build id: 3.10.0-693.el7.x86_64
0x39d
3
schedule
vmlinux 0xffffffff816a94c9
Build id: 3.10.0-693.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810f5126
Build id: 3.10.0-693.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff810f5deb
Build id: 3.10.0-693.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff810b4370
Build id: 3.10.0-693.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810f5104
Build id: 3.10.0-693.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff810f7b26
Build id: 3.10.0-693.el7.x86_64
0x106
9
SyS_futex
vmlinux 0xffffffff810f8040
Build id: 3.10.0-693.el7.x86_64
0x80
10
tracesys
vmlinux 0xffffffff816b51d2
Build id: 3.10.0-693.el7.x86_64
0xdd
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 35s! [vertx-blocked-t:2310]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev ppdev sg vmw_balloon shpchp
vmw_vmci parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata mptspi scsi_transport_spi vmxnet3 serio_raw mptscsih mptbase i2c_core
CPU: 1 PID: 2310 Comm: vertx-blocked-t Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff8803dd78eeb0 ti: ffff8803d5e7c000 task.ti: ffff8803d5e7c000
RIP: 0010:[<ffffffff810bf104>]  [<ffffffff810bf104>] finish_task_switch+0x54/0x160
RSP: 0018:ffff8803d5e7fc58  EFLAGS: 00000286
RAX: ffff88017a26cf10 RBX: ffff88043fc96d40 RCX: 00000000c0000100
RDX: ffff8803d5e7dfd8 RSI: ffff8803dd78eeb0 RDI: ffff88043fc96cc0
RBP: ffff8803d5e7fc78 R08: ffff8803d5e7c000 R09: 0000000000000000
R10: 0000000000000004 R11: 0000000000000005 R12: 0000000200000000
R13: 0000000000000000 R14: ffff88043fd16cc0 R15: ffffffff810cdc7c
FS:  00007fb18072db20(0000) GS:ffff88043fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f67584764f1 CR3: 0000000402347000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88042b5c1900 ffff88043fc96cc0 ffff88041bf4abc0 0000000000000001
ffff8803d5e7fcd8 ffffffff816a8f8d ffff8803dd78eeb0 ffff8803d5e7ffd8
ffff8803d5e7ffd8 ffff8803d5e7ffd8 ffff8803dd78eeb0 ffff8803dd78eeb0
Call Trace:
[<ffffffff816a8f8d>] __schedule+0x39d/0x8b0
[<ffffffff816a94c9>] schedule+0x29/0x70
[<ffffffff810f5126>] futex_wait_queue_me+0xc6/0x130
[<ffffffff810f5deb>] futex_wait+0x17b/0x280
[<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
[<ffffffff810f5104>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff810f7b26>] do_futex+0x106/0x5a0
[<ffffffff810f8040>] SyS_futex+0x80/0x180
[<ffffffff816b51d2>] tracesys+0xdd/0xe2
Code: 8b 36 66 66 66 66 90 65 48 8b 34 25 00 ce 00 00 66 66 66 66 90 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 65 48 8b 04 25 00 ce 00 00 48 8b 98 78 01 00 00 48 85 
Complete report #574292
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffa1ad4bf7
Build id: 3.10.0-1160.76.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffa218c070
Build id: 3.10.0-1160.76.1.el7.x86_64
0x320
3
schedule
vmlinux 0xffffffffa218c3f9
Build id: 3.10.0-1160.76.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffa1b128d6
Build id: 3.10.0-1160.76.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffa1b1369b
Build id: 3.10.0-1160.76.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffa1ac9bd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffffa1b128b4
Build id: 3.10.0-1160.76.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffa1b15356
Build id: 3.10.0-1160.76.1.el7.x86_64
0x106
9
SyS_futex
vmlinux 0xffffffffa1b157a0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x80
10
system_call_fastpath
vmlinux 0xffffffffa2199f92
Build id: 3.10.0-1160.76.1.el7.x86_64
0x25
11
? system_call_after_swapgs
vmlinux 0xffffffffa2199ed5
Build id: 3.10.0-1160.76.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [Timer-4:3001]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vboxsf(OE) sunrpc snd_intel8x0 snd_ac97_codec nfit ac97_bus joydev snd_seq ppdev libnvdimm snd_seq_device snd_pcm snd_timer vboxguest(OE) iosf_mbi snd pcspkr soundcore sg i2c_piix4 parport_pc video parport ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic crct10dif_common ata_generic pata_acpi serio_raw ahci ata_piix libahci vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata drm crc32c_intel e1000 drm_panel_orientation_quirks
dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 3001 Comm: Timer-4 Kdump: loaded Tainted: G        W  OE  ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8f6f0e8c2100 ti: ffff8f6ebfc3c000 task.ti: ffff8f6ebfc3c000
RIP: 0010:[<ffffffffa1ad4bf7>]  [<ffffffffa1ad4bf7>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff8f6ebfc3fc28  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8f6f1fc9acc0 RCX: 00000000c0000100
RDX: ffff8f6ebfc3dfd8 RSI: ffff8f6f0e8c2100 RDI: ffff8f6f1fc9acc0
RBP: ffff8f6ebfc3fc48 R08: ffff8f6ebfc3c000 R09: 0000000000000001
R10: 0000000000000013 R11: 0000000000000000 R12: ffff8f6ebfc3fba8
R13: ffff8f6f1fc9acc0 R14: ffff8f6ebfc3fbd0 R15: 0000000000000001
FS:  00007f29baef0700(0000) GS:ffff8f6f1fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2a58326300 CR3: 000000040e834000 CR4: 00000000000206e0
Call Trace:
[<ffffffffa218c070>] __schedule+0x320/0x680
[<ffffffffa218c3f9>] schedule+0x29/0x70
[<ffffffffa1b128d6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffa1b1369b>] futex_wait+0x17b/0x280
[<ffffffffa1ac9bd0>] ? hrtimer_get_res+0x50/0x50
[<ffffffffa1b128b4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffa1b15356>] do_futex+0x106/0x4d0
[<ffffffffa1b157a0>] SyS_futex+0x80/0x190
[<ffffffffa2199f92>] system_call_fastpath+0x25/0x2a
[<ffffffffa2199ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df e8 36 29 04 00 66 90 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #495874
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff83ed4bf7
Build id: 3.10.0-1160.71.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff8458c070
Build id: 3.10.0-1160.71.1.el7.x86_64
0x320
3
schedule
vmlinux 0xffffffff8458c3f9
Build id: 3.10.0-1160.71.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff83f12956
Build id: 3.10.0-1160.71.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff83f1371b
Build id: 3.10.0-1160.71.1.el7.x86_64
0x17b
6
? __do_fault.isra.61
vmlinux 0xffffffff83fee7fa
Build id: 3.10.0-1160.71.1.el7.x86_64
0x8a
7
? hrtimer_get_res
vmlinux 0xffffffff83ec9bd0
Build id: 3.10.0-1160.71.1.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff83f12934
Build id: 3.10.0-1160.71.1.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff83f153d6
Build id: 3.10.0-1160.71.1.el7.x86_64
0x106
10
? __switch_to
vmlinux 0xffffffff83e2b621
Build id: 3.10.0-1160.71.1.el7.x86_64
0x151
11
SyS_futex
vmlinux 0xffffffff83f15820
Build id: 3.10.0-1160.71.1.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff84599f92
Build id: 3.10.0-1160.71.1.el7.x86_64
0x25
13
? system_call_after_swapgs
vmlinux 0xffffffff84599ed5
Build id: 3.10.0-1160.71.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 211s! [cpptools-srv:31399]
Modules linked in: ppp_deflate bsd_comp ppp_async crc_ccitt ppp_generic slhc tcp_lp nls_utf8 isofs vboxsf(OE) nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter binfmt_misc overlay(T) snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device joydev snd_pcm vboxguest(OE) ppdev snd_timer pcspkr iosf_mbi snd parport_pc video parport i2c_piix4 soundcore sg nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi crc32c_intel
vmwgfx drm_kms_helper serio_raw ahci syscopyarea sysfillrect sysimgblt fb_sys_fops libahci ttm e1000 drm ata_piix drm_panel_orientation_quirks libata dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 31399 Comm: cpptools-srv Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff96f188c91080 ti: ffff96f18aec4000 task.ti: ffff96f18aec4000
RIP: 0010:[<ffffffff83ed4bf7>]  [<ffffffff83ed4bf7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff96f18aec7c28  EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffffffff83e6d3be RCX: 00000000c0000100
RDX: ffff96f18aec5fd8 RSI: ffff96f188c91080 RDI: ffff96f216c1acc0
RBP: ffff96f18aec7c48 R08: ffff96f18aec4000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000005 R12: 0000000000000000
R13: ffffffff83ee4e4c R14: ffff96f18aec7bd0 R15: 0000000000000001
FS:  00007fed4cc85700(0000) GS:ffff96f216c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000025ed00e37000 CR3: 0000000124ec6000 CR4: 00000000000206f0
Call Trace:
[<ffffffff8458c070>] __schedule+0x320/0x680
[<ffffffff8458c3f9>] schedule+0x29/0x70
[<ffffffff83f12956>] futex_wait_queue_me+0xc6/0x130
[<ffffffff83f1371b>] futex_wait+0x17b/0x280
[<ffffffff83fee7fa>] ? __do_fault.isra.61+0x8a/0x100
[<ffffffff83ec9bd0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff83f12934>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff83f153d6>] do_futex+0x106/0x4d0
[<ffffffff83e2b621>] ? __switch_to+0x151/0x580
[<ffffffff83f15820>] SyS_futex+0x80/0x190
[<ffffffff84599f92>] system_call_fastpath+0x25/0x2a
[<ffffffff84599ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df e8 b6 29 04 00 66 90 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #475258
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffb40d5404
Build id: 3.10.0-1127.10.1.el7.x86_64
0x54
2
__schedule
vmlinux 0xffffffffb4785942
Build id: 3.10.0-1127.10.1.el7.x86_64
0x402
3
schedule
vmlinux 0xffffffffb4785da9
Build id: 3.10.0-1127.10.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffb4112bd6
Build id: 3.10.0-1127.10.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffb411397b
Build id: 3.10.0-1127.10.1.el7.x86_64
0x17b
6
? kmem_cache_alloc
vmlinux 0xffffffffb42286c5
Build id: 3.10.0-1127.10.1.el7.x86_64
0x35
7
? hrtimer_get_res
vmlinux 0xffffffffb40ca2d0
Build id: 3.10.0-1127.10.1.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffffb4112bb4
Build id: 3.10.0-1127.10.1.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffffb41156c6
Build id: 3.10.0-1127.10.1.el7.x86_64
0x106
10
SyS_futex
vmlinux 0xffffffffb4115be0
Build id: 3.10.0-1127.10.1.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffffb4792ed2
Build id: 3.10.0-1127.10.1.el7.x86_64
0x25
12
? system_call_after_swapgs
vmlinux 0xffffffffb4792e15
Build id: 3.10.0-1127.10.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [Multifile Text :14360]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables bridge stp llc ip_set nfnetlink vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg i2c_piix4 vmw_vmci parport_pc parport xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm
drm nfit sd_mod crc_t10dif crct10dif_generic ata_piix libnvdimm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmw_pvscsi vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 14360 Comm: Multifile Text  Kdump: loaded Tainted: G             L ------------   3.10.0-1127.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9a2dc9ea62a0 ti: ffff9a2cfefa8000 task.ti: ffff9a2cfefa8000
RIP: 0010:[<ffffffffb40d5404>]  [<ffffffffb40d5404>] finish_task_switch+0x54/0x1c0
RSP: 0000:ffff9a2cfefabc08  EFLAGS: 00000286
RAX: ffff9a2dc1b8a0e0 RBX: ffff9a2cfefabd90 RCX: 00000000c0000100
RDX: ffff9a2cfefa9fd8 RSI: ffff9a2dc9ea62a0 RDI: ffff9a2effc9acc0
RBP: ffff9a2cfefabc28 R08: ffff9a2cfefa8000 R09: 0000000000000001
R10: 0000000000000068 R11: 0000000000000000 R12: 0000000000000000
R13: ffffffffb40e539c R14: ffff9a2cfefabbb0 R15: 0000000000000001
FS:  00007f1d1bafc700(0000) GS:ffff9a2effc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1cf40cd00c CR3: 0000000101920000 CR4: 00000000000607e0
Call Trace:
[<ffffffffb4785942>] __schedule+0x402/0x840
[<ffffffffb4785da9>] schedule+0x29/0x70
[<ffffffffb4112bd6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffb411397b>] futex_wait+0x17b/0x280
[<ffffffffb42286c5>] ? kmem_cache_alloc+0x35/0x1f0
[<ffffffffb40ca2d0>] ? hrtimer_get_res+0x50/0x50
[<ffffffffb4112bb4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffb41156c6>] do_futex+0x106/0x5a0
[<ffffffffb4115be0>] SyS_futex+0x80/0x190
[<ffffffffb4792ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffb4792e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 8b 36 66 66 66 66 90 65 48 8b 34 25 c0 0e 01 00 66 66 66 66 90 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 65 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 
Complete report #471060
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff86ed4c97
Build id: 3.10.0-1160.15.2.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff875878ff
Build id: 3.10.0-1160.15.2.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffff87587dd9
Build id: 3.10.0-1160.15.2.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff86f12436
Build id: 3.10.0-1160.15.2.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff86f131db
Build id: 3.10.0-1160.15.2.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff86ec9a50
Build id: 3.10.0-1160.15.2.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff86f12414
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff86f14f46
Build id: 3.10.0-1160.15.2.el7.x86_64
0x106
9
SyS_futex
vmlinux 0xffffffff86f15460
Build id: 3.10.0-1160.15.2.el7.x86_64
0x80
10
system_call_fastpath
vmlinux 0xffffffff87594f92
Build id: 3.10.0-1160.15.2.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 23s! [expiration-thre:127110]
Modules linked in: bmhook(OE) tmhook(OE) dsa_filter(POE) dsa_filter_hook(OE) tcp_lp rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 ppdev sb_edac vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport
vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul nfit crct10dif_common crc32c_intel mptspi libata scsi_transport_spi libnvdimm mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: dsa_filter_hook]
CPU: 15 PID: 127110 Comm: expiration-thre Kdump: loaded Tainted: P           OEL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff89c31c0bd280 ti: ffff89b97cf8c000 task.ti: ffff89b97cf8c000
RIP: 0010:[<ffffffff86ed4c97>]  [<ffffffff86ed4c97>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff89b97cf8fc08  EFLAGS: 00000286
RAX: ffff89bc8783a100 RBX: ffff89b97cf8fbb8 RCX: 00000000c0000100
RDX: ffff89b97cf8dfd8 RSI: ffff89c31c0bd280 RDI: ffff89c50d9dacc0
RBP: ffff89b97cf8fc28 R08: ffff89b97cf8c000 R09: 0000000000000001
R10: 0000000000000021 R11: 0000000000000000 R12: ffff89b97cf8fbb8
R13: ffffffff86ee4afc R14: ffff89b97cf8fbb0 R15: 0000000000000001
FS:  00007f27de7d8700(0000) GS:ffff89c50d9c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3c84c40008 CR3: 00000009fa11a000 CR4: 00000000003607e0
Call Trace:
[<ffffffff875878ff>] __schedule+0x3af/0x860
[<ffffffff87587dd9>] schedule+0x29/0x70
[<ffffffff86f12436>] futex_wait_queue_me+0xc6/0x130
[<ffffffff86f131db>] futex_wait+0x17b/0x280
[<ffffffff86ec9a50>] ? hrtimer_get_res+0x50/0x50
[<ffffffff86f12414>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff86f14f46>] do_futex+0x106/0x5a0
[<ffffffff86f15460>] SyS_futex+0x80/0x190
[<ffffffff87594f92>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #401424
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff868d0897
Build id: 3.10.0-957.1.3.el7.x86_64
0x67
2
__schedule
vmlinux 0xffffffff86f6778f
Build id: 3.10.0-957.1.3.el7.x86_64
0x3ff
3
? hrtimer_start_range_ns
vmlinux 0xffffffff868c5cad
Build id: 3.10.0-957.1.3.el7.x86_64
0x1ed
4
schedule
vmlinux 0xffffffff86f67c49
Build id: 3.10.0-957.1.3.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff8690cf36
Build id: 3.10.0-957.1.3.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff8690dc1b
Build id: 3.10.0-957.1.3.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff868c57e0
Build id: 3.10.0-957.1.3.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff8690cf14
Build id: 3.10.0-957.1.3.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff8690f966
Build id: 3.10.0-957.1.3.el7.x86_64
0x106
10
? __fput
vmlinux 0xffffffff86a436e6
Build id: 3.10.0-957.1.3.el7.x86_64
0x186
11
SyS_futex
vmlinux 0xffffffff8690fe80
Build id: 3.10.0-957.1.3.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff86f74ddb
Build id: 3.10.0-957.1.3.el7.x86_64
0x22
13
? system_call_after_swapgs
vmlinux 0xffffffff86f74d21
Build id: 3.10.0-957.1.3.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 42s! [sql.csv:24396]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink vmw_vsock_vmci_transport vsock ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw joydev pcspkr gf128mul glue_helper ppdev ablk_helper cryptd sg parport_pc vmw_balloon parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi
sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ata_piix libata vmw_pvscsi drm_panel_orientation_quirks nfit libnvdimm floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 24396 Comm: sql.csv Kdump: loaded Tainted: G             L ------------   3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff92e145559040 ti: ffff92e145430000 task.ti: ffff92e145430000
RIP: 0010:[<ffffffff868d0897>]  [<ffffffff868d0897>] finish_task_switch+0x67/0x1c0
RSP: 0018:ffff92e145433c08  EFLAGS: 00000282
RAX: ffff92e145559040 RBX: 0202000000000000 RCX: 00000000c0000100
RDX: ffff92e145431fd8 RSI: ffff92e145559040 RDI: ffff92e2fefdab80
RBP: ffff92e145433c28 R08: ffff92e145430000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: ffffffff868e060c R14: ffff92e145433bb0 R15: 0000000000000001
FS:  00007f4a8fdf7700(0000) GS:ffff92e2fefc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007faa42e62000 CR3: 00000001e97b2000 CR4: 00000000000407e0
Call Trace:
[<ffffffff86f6778f>] __schedule+0x3ff/0x890
[<ffffffff868c5cad>] ? hrtimer_start_range_ns+0x1ed/0x3c0
[<ffffffff86f67c49>] schedule+0x29/0x70
[<ffffffff8690cf36>] futex_wait_queue_me+0xc6/0x130
[<ffffffff8690dc1b>] futex_wait+0x17b/0x280
[<ffffffff868c57e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8690cf14>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff8690f966>] do_futex+0x106/0x5a0
[<ffffffff86a436e6>] ? __fput+0x186/0x260
[<ffffffff8690fe80>] SyS_futex+0x80/0x190
[<ffffffff86f74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff86f74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 65 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 <48> 85 db 74 22 0f 1f 40 00 48 8b 43 10 48 89 df 65 8b 35 6e d7 
Complete report #328201
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff886d09f7
Build id: 3.10.0-957.21.3.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff88d68a7a
Build id: 3.10.0-957.21.3.el7.x86_64
0x42a
3
? hrtimer_start_range_ns
vmlinux 0xffffffff886c5e1d
Build id: 3.10.0-957.21.3.el7.x86_64
0x1ed
4
schedule
vmlinux 0xffffffff88d68ed9
Build id: 3.10.0-957.21.3.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff8870d1a6
Build id: 3.10.0-957.21.3.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff8870de8b
Build id: 3.10.0-957.21.3.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff886c5950
Build id: 3.10.0-957.21.3.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff8870d184
Build id: 3.10.0-957.21.3.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff8870fbd6
Build id: 3.10.0-957.21.3.el7.x86_64
0x106
10
? __check_object_size
vmlinux 0xffffffff8883e7f8
Build id: 3.10.0-957.21.3.el7.x86_64
0x198
11
SyS_futex
vmlinux 0xffffffff887100f0
Build id: 3.10.0-957.21.3.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff88d75ddb
Build id: 3.10.0-957.21.3.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 54s! [Api Webserver:30772]
Modules linked in: nf_conntrack_netlink nfnetlink xt_CHECKSUM iptable_mangle xt_addrtype ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack br_netfilter ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc sr_mod cdrom ppdev snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer ahci sg libahci snd soundcore nfit libnvdimm parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
crct10dif_pclmul crct10dif_common crc32c_intel serio_raw e1000 ata_generic floppy pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi libata scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 30772 Comm: Api Webserver Tainted: G               ------------ T 3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff881175da5140 ti: ffff881184958000 task.ti: ffff881184958000
RIP: 0010:[<ffffffff886d09f7>]  [<ffffffff886d09f7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff88118495bc08  EFLAGS: 00000282
RAX: ffff8811926dd140 RBX: ffff881192d55600 RCX: 00000000c0000100
RDX: ffff881184959fd8 RSI: ffff881175da5140 RDI: ffff881195e1ab80
RBP: ffff88118495bc28 R08: ffff881184958000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000001 R12: 00000b0188d6897c
R13: ffffffff886e084c R14: ffff88118495bbb0 R15: 0000000000000001
FS:  00007f2af8738700(0000) GS:ffff881195e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2b1cc4cc28 CR3: 0000000027ace000 CR4: 00000000001607f0
Call Trace:
[<ffffffff88d68a7a>] __schedule+0x42a/0x860
[<ffffffff886c5e1d>] ? hrtimer_start_range_ns+0x1ed/0x3c0
[<ffffffff88d68ed9>] schedule+0x29/0x70
[<ffffffff8870d1a6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff8870de8b>] futex_wait+0x17b/0x280
[<ffffffff886c5950>] ? hrtimer_get_res+0x50/0x50
[<ffffffff8870d184>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff8870fbd6>] do_futex+0x106/0x5a0
[<ffffffff8883e7f8>] ? __check_object_size+0x198/0x250
[<ffffffff887100f0>] SyS_futex+0x80/0x190
[<ffffffff88d75ddb>] system_call_fastpath+0x22/0x27
Code: 1f 44 00 00 65 48 8b 34 25 80 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #327637
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810bf107
Build id: 3.10.0-693.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff816a8f8d
Build id: 3.10.0-693.el7.x86_64
0x39d
3
schedule
vmlinux 0xffffffff816a94c9
Build id: 3.10.0-693.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810f5126
Build id: 3.10.0-693.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff810f5deb
Build id: 3.10.0-693.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff810b4370
Build id: 3.10.0-693.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810f5104
Build id: 3.10.0-693.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff810f7b26
Build id: 3.10.0-693.el7.x86_64
0x106
9
? do_futex
vmlinux 0xffffffff810f7a3e
Build id: 3.10.0-693.el7.x86_64
0x1e
10
SyS_futex
vmlinux 0xffffffff810f8040
Build id: 3.10.0-693.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 27s! [java:27299]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter arc4 md4 nls_utf8 cifs dns_resolver vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev sg pcspkr parport_pc parport vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod vmwgfx crc_t10dif crct10dif_generic sr_mod cdrom drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_generic ttm pata_acpi mptsas drm scsi_transport_sas
ata_piix mptscsih crct10dif_pclmul crct10dif_common libata mptbase crc32c_intel serio_raw vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 27299 Comm: java Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff880bf8af6eb0 ti: ffff880b9974c000 task.ti: ffff880b9974c000
RIP: 0010:[<ffffffff810bf107>]  [<ffffffff810bf107>] finish_task_switch+0x57/0x160
RSP: 0000:ffff880b9974fc58  EFLAGS: 00000286
RAX: ffff880bf8af0fd0 RBX: ffff880c0f296d40 RCX: 00000000c0000100
RDX: ffff880b9974dfd8 RSI: ffff880bf8af6eb0 RDI: ffff880c0f296cc0
RBP: ffff880b9974fc78 R08: ffff880b9974c000 R09: 0000000000000000
R10: 000000000000bb94 R11: 0000000000000001 R12: ffff880c0f296d40
R13: 0000544b00000001 R14: 0000544c0f516cc0 R15: ffffffff810cdc7c
FS:  00007f07e0c3b700(0000) GS:ffff880c0f280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7e21edd008 CR3: 0000000bfb3ba000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880bf7b7f6c0 ffff880c0f296cc0 ffff880035ebea40 0000000000000002
ffff880b9974fcd8 ffffffff816a8f8d ffff880bf8af6eb0 ffff880b9974ffd8
ffff880b9974ffd8 ffff880b9974ffd8 ffff880bf8af6eb0 ffff880bf8af6eb0
Call Trace:
[<ffffffff816a8f8d>] __schedule+0x39d/0x8b0
[<ffffffff816a94c9>] schedule+0x29/0x70
[<ffffffff810f5126>] futex_wait_queue_me+0xc6/0x130
[<ffffffff810f5deb>] futex_wait+0x17b/0x280
[<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
[<ffffffff810f5104>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff810f7b26>] do_futex+0x106/0x5a0
[<ffffffff810f7a3e>] ? do_futex+0x1e/0x5a0
[<ffffffff810f8040>] SyS_futex+0x80/0x180
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 1f 44 00 00 65 48 8b 34 25 00 ce 00 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 00 ce 00 00 48 8b 98 78 01 00 00 48 85 db 74 1c 
Complete report #316373
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff91ad4c97
Build id: 3.10.0-1160.15.2.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff921878ff
Build id: 3.10.0-1160.15.2.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffff92187dd9
Build id: 3.10.0-1160.15.2.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff91b12436
Build id: 3.10.0-1160.15.2.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff91b131db
Build id: 3.10.0-1160.15.2.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff91ac9a50
Build id: 3.10.0-1160.15.2.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff91b12414
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff91b14f46
Build id: 3.10.0-1160.15.2.el7.x86_64
0x106
9
? __seccomp_filter
vmlinux 0xffffffff91b56ffb
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5b
10
? __switch_to
vmlinux 0xffffffff91a2b621
Build id: 3.10.0-1160.15.2.el7.x86_64
0x151
11
SyS_futex
vmlinux 0xffffffff91b15460
Build id: 3.10.0-1160.15.2.el7.x86_64
0x80
12
tracesys
vmlinux 0xffffffff92195226
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 27s! [java:3044]
Modules linked in: iptable_raw xt_mac xt_mark xt_multiport xt_limit xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg vmw_balloon pcspkr joydev parport_pc parport vmw_vmci i2c_piix4 sch_fq_codel ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common drm_kms_helper crc32c_intel syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm serio_raw mptspi ata_piix drm scsi_transport_spi mptscsih vmxnet3 libata mptbase nfit libnvdimm drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 3044 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9ad6cce06300 ti: ffff9ad690048000 task.ti: ffff9ad690048000
RIP: 0010:[<ffffffff91ad4c97>]  [<ffffffff91ad4c97>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff9ad69004bc08  EFLAGS: 00000286
RAX: ffff9ad43b795280 RBX: ffff9ad69004bbb8 RCX: 00000000c0000100
RDX: ffff9ad690049fd8 RSI: ffff9ad6cce06300 RDI: ffff9ad6ed65acc0
RBP: ffff9ad69004bc28 R08: ffff9ad690048000 R09: 0000000000000001
R10: 0000000000000021 R11: 0000000000000293 R12: ffff9ad69004bbb8
R13: ffffffff91ae4afc R14: ffff9ad69004bbb0 R15: 0000000000000001
FS:  00007fb880b7c700(0000) GS:ffff9ad6ed640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fad06620000 CR3: 0000000172baa000 CR4: 00000000001607e0
Call Trace:
[<ffffffff921878ff>] __schedule+0x3af/0x860
[<ffffffff92187dd9>] schedule+0x29/0x70
[<ffffffff91b12436>] futex_wait_queue_me+0xc6/0x130
[<ffffffff91b131db>] futex_wait+0x17b/0x280
[<ffffffff91ac9a50>] ? hrtimer_get_res+0x50/0x50
[<ffffffff91b12414>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff91b14f46>] do_futex+0x106/0x5a0
[<ffffffff91b56ffb>] ? __seccomp_filter+0x5b/0x300
[<ffffffff91a2b621>] ? __switch_to+0x151/0x580
[<ffffffff91b15460>] SyS_futex+0x80/0x190
[<ffffffff92195226>] tracesys+0xa6/0xcc
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #271889
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810bf107
Build id: 3.10.0-693.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff816a8f8d
Build id: 3.10.0-693.el7.x86_64
0x39d
3
schedule
vmlinux 0xffffffff816a94c9
Build id: 3.10.0-693.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810f5126
Build id: 3.10.0-693.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff810f5deb
Build id: 3.10.0-693.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff810b4370
Build id: 3.10.0-693.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810f5104
Build id: 3.10.0-693.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff810f7b26
Build id: 3.10.0-693.el7.x86_64
0x106
9
? __do_page_fault
vmlinux 0xffffffff816affd1
Build id: 3.10.0-693.el7.x86_64
0x171
10
SyS_futex
vmlinux 0xffffffff810f8040
Build id: 3.10.0-693.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 43s! [production_moni:1195]
Modules linked in: zram isofs vmw_vsock_vmci_transport vsock xfs libcrc32c sb_edac edac_core coretemp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci shpchp i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common mptspi scsi_transport_spi mptscsih crc32c_intel mptbase serio_raw vmxnet3 ata_piix libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 1195 Comm: production_moni Tainted: G             L ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff880079698fd0 ti: ffff880100404000 task.ti: ffff880100404000
RIP: 0010:[<ffffffff810bf107>]  [<ffffffff810bf107>] finish_task_switch+0x57/0x160
RSP: 0018:ffff880100407c58  EFLAGS: 00000286
RAX: ffff880056185ee0 RBX: ffff8801bfd96d40 RCX: 00000000c0000100
RDX: ffff880100405fd8 RSI: ffff880079698fd0 RDI: ffff8801bfd96cc0
RBP: ffff880100407c78 R08: ffff880100404000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: ffffffff816ab9ee
R13: ffff880100407bc8 R14: ffffffff810cae7b R15: ffff880100407c20
FS:  00007fbf04ecd700(0000) GS:ffff8801bfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000037f69a9da000 CR3: 00000000064de000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
 ffff880019e09f40 ffff8801bfd96cc0 ffff8801b1309900 0000000000000003
 ffff880100407cd8 ffffffff816a8f8d ffff880079698fd0 ffff880100407fd8
 ffff880100407fd8 ffff880100407fd8 ffff880079698fd0 ffff880079698fd0
Call Trace:
 [<ffffffff816a8f8d>] __schedule+0x39d/0x8b0
 [<ffffffff816a94c9>] schedule+0x29/0x70
 [<ffffffff810f5126>] futex_wait_queue_me+0xc6/0x130
 [<ffffffff810f5deb>] futex_wait+0x17b/0x280
 [<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
 [<ffffffff810f5104>] ? futex_wait_queue_me+0xa4/0x130
 [<ffffffff810f7b26>] do_futex+0x106/0x5a0
 [<ffffffff816affd1>] ? __do_page_fault+0x171/0x450
 [<ffffffff810f8040>] SyS_futex+0x80/0x180
 [<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 1f 44 00 00 65 48 8b 34 25 00 ce 00 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 00 ce 00 00 48 8b 98 78 01 00 00 48 85 db 74 1c 
Complete report #270769
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810bf107
Build id: 3.10.0-693.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff816a8f8d
Build id: 3.10.0-693.el7.x86_64
0x39d
3
schedule
vmlinux 0xffffffff816a94c9
Build id: 3.10.0-693.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810f5126
Build id: 3.10.0-693.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff810f5deb
Build id: 3.10.0-693.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff810b4370
Build id: 3.10.0-693.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810f5104
Build id: 3.10.0-693.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff810f7b26
Build id: 3.10.0-693.el7.x86_64
0x106
9
? __switch_to
vmlinux 0xffffffff810295da
Build id: 3.10.0-693.el7.x86_64
0x15a
10
SyS_futex
vmlinux 0xffffffff810f8040
Build id: 3.10.0-693.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 28s! [java:4345]
Modules linked in: ipt_REJECT nf_reject_ipv4 binfmt_misc ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) dm_mirror dm_region_hash dm_log dm_mod edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg ppdev joydev virtio_balloon parport_pc parport i2c_piix4 pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common virtio_pci i2c_core crc32c_intel virtio_ring serio_raw floppy virtio
CPU: 3 PID: 4345 Comm: java Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: RDO OpenStack Compute, BIOS 1.11.0-2.el7 04/01/2014
task: ffff8808252eeeb0 ti: ffff880022dbc000 task.ti: ffff880022dbc000
RIP: 0010:[<ffffffff810bf107>]  [<ffffffff810bf107>] finish_task_switch+0x57/0x160
RSP: 0000:ffff880022dbfc58  EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff880c0f2d6d40 RCX: 00000000c0000100
RDX: ffff880022dbdfd8 RSI: ffff8808252eeeb0 RDI: ffff880c0f2d6cc0
RBP: ffff880022dbfc78 R08: ffff880022dbc000 R09: 0000000000000000
R10: 000000000000b917 R11: 0000000000000001 R12: 0000000200000000
R13: 0000000000000000 R14: ffffffff810cae7b R15: ffffffff810cdc7c
FS:  00007ff6ab79c700(0000) GS:ffff880c0f2c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0010bb000 CR3: 0000000bd9ae6000 CR4: 00000000003406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880036986400 ffff880c0f2d6cc0 ffff880691a7b200 0000000000000003
ffff880022dbfcd8 ffffffff816a8f8d ffff8808252eeeb0 ffff880022dbffd8
ffff880022dbffd8 ffff880022dbffd8 ffff8808252eeeb0 ffff8808252eeeb0
Call Trace:
[<ffffffff816a8f8d>] __schedule+0x39d/0x8b0
[<ffffffff816a94c9>] schedule+0x29/0x70
[<ffffffff810f5126>] futex_wait_queue_me+0xc6/0x130
[<ffffffff810f5deb>] futex_wait+0x17b/0x280
[<ffffffff810b4370>] ? hrtimer_get_res+0x50/0x50
[<ffffffff810f5104>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff810f7b26>] do_futex+0x106/0x5a0
[<ffffffff810295da>] ? __switch_to+0x15a/0x510
[<ffffffff810f8040>] SyS_futex+0x80/0x180
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 1f 44 00 00 65 48 8b 34 25 00 ce 00 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df ff 14 25 50 31 a1 81 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 00 ce 00 00 48 8b 98 78 01 00 00 48 85 db 74 1c 
Complete report #255137
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff928d5404
Build id: 3.10.0-1127.el7.x86_64
0x54
2
__schedule
vmlinux 0xffffffff92f85922
Build id: 3.10.0-1127.el7.x86_64
0x402
3
schedule
vmlinux 0xffffffff92f85d89
Build id: 3.10.0-1127.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff92912c16
Build id: 3.10.0-1127.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff929139bb
Build id: 3.10.0-1127.el7.x86_64
0x17b
6
? ttwu_do_wakeup
vmlinux 0xffffffff928d7c79
Build id: 3.10.0-1127.el7.x86_64
0x19
7
? hrtimer_get_res
vmlinux 0xffffffff928ca2d0
Build id: 3.10.0-1127.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff92912bf4
Build id: 3.10.0-1127.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff92915706
Build id: 3.10.0-1127.el7.x86_64
0x106
10
? __check_object_size
vmlinux 0xffffffff92a49cc8
Build id: 3.10.0-1127.el7.x86_64
0x198
11
SyS_futex
vmlinux 0xffffffff92915c20
Build id: 3.10.0-1127.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff92f92ed2
Build id: 3.10.0-1127.el7.x86_64
0x25
13
? system_call_after_swapgs
vmlinux 0xffffffff92f92e15
Build id: 3.10.0-1127.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [alertmanager:1952]
Modules linked in: tcp_lp btrfs raid6_pq xor vfat fat tcp_diag inet_diag ext4 mbcache jbd2 fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc uas coretemp usb_storage crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon
sg pcspkr joydev vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi ata_piix scsi_transport_spi mptscsih libata mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1952 Comm: alertmanager Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
task: ffff9c85b8551070 ti: ffff9c85b8570000 task.ti: ffff9c85b8570000
RIP: 0010:[<ffffffff928d5404>]  [<ffffffff928d5404>] finish_task_switch+0x54/0x1c0
RSP: 0018:ffff9c85b8573c08  EFLAGS: 00000286
RAX: ffff9c85b94eb150 RBX: ffff9c85b8573bb8 RCX: 00000000c0000100
RDX: ffff9c85b8571fd8 RSI: ffff9c85b8551070 RDI: ffff9c85bfc1acc0
RBP: ffff9c85b8573c28 R08: ffff9c85b8570000 R09: 0000000000000001
R10: 000000000000001a R11: 00000000000003ff R12: ffff9c85b8573bb8
R13: ffffffff928e539c R14: ffff9c85b8573bb0 R15: 0000000000000001
FS:  00007f29ac9b0700(0000) GS:ffff9c85bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ba0af3100 CR3: 0000000135f7e000 CR4: 00000000000007f0
Call Trace:
[<ffffffff92f85922>] __schedule+0x402/0x840
[<ffffffff92f85d89>] schedule+0x29/0x70
[<ffffffff92912c16>] futex_wait_queue_me+0xc6/0x130
[<ffffffff929139bb>] futex_wait+0x17b/0x280
[<ffffffff928d7c79>] ? ttwu_do_wakeup+0x19/0xe0
[<ffffffff928ca2d0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff92912bf4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff92915706>] do_futex+0x106/0x5a0
[<ffffffff92a49cc8>] ? __check_object_size+0x198/0x250
[<ffffffff92915c20>] SyS_futex+0x80/0x190
[<ffffffff92f92ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff92f92e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 8b 36 66 66 66 66 90 65 48 8b 34 25 c0 0e 01 00 66 66 66 66 90 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 65 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 
Complete report #252741
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffbd2d5407
Build id: 3.10.0-1127.19.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffbd985942
Build id: 3.10.0-1127.19.1.el7.x86_64
0x402
3
schedule
vmlinux 0xffffffffbd985da9
Build id: 3.10.0-1127.19.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffbd312c26
Build id: 3.10.0-1127.19.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffbd3139cb
Build id: 3.10.0-1127.19.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffbd2ca2d0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffffbd312c04
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffbd315716
Build id: 3.10.0-1127.19.1.el7.x86_64
0x106
9
? do_munmap
vmlinux 0xffffffffbd3fb8a7
Build id: 3.10.0-1127.19.1.el7.x86_64
0x327
10
SyS_futex
vmlinux 0xffffffffbd315c30
Build id: 3.10.0-1127.19.1.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffffbd992ed2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [tnslsnr:1088]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter ipmi_devintf ipmi_msghandler vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr joydev vmw_balloon parport_pc i2c_piix4 parport vmw_vmci binfmt_misc ip_tables xfs libcrc32c ata_generic pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci nfit drm
libnvdimm ata_piix libahci e1000e crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw ptp pps_core vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1088 Comm: tnslsnr Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8c8f2cefe2a0 ti: ffff8c8b353f8000 task.ti: ffff8c8b353f8000
RIP: 0010:[<ffffffffbd2d5407>]  [<ffffffffbd2d5407>] finish_task_switch+0x57/0x1c0
RSP: 0000:ffff8c8b353fbc08  EFLAGS: 00000286
RAX: ffff8c8b357b41c0 RBX: ffff8c8b353fbb78 RCX: 00000000c0000100
RDX: ffff8c8b353f9fd8 RSI: ffff8c8f2cefe2a0 RDI: ffff8c8f3fc1acc0
RBP: ffff8c8b353fbc28 R08: ffff8c8b353f8000 R09: 0000000000000001
R10: 0000000000000039 R11: 0000000000000005 R12: 0000000000000002
R13: 0000000000000020 R14: 0000000000000000 R15: ffff8c8c7fd10000
FS:  00007fd2ee867700(0000) GS:ffff8c8f3fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c29aeb6c8 CR3: 000000042e360000 CR4: 00000000003607f0
Call Trace:
[<ffffffffbd985942>] __schedule+0x402/0x840
[<ffffffffbd985da9>] schedule+0x29/0x70
[<ffffffffbd312c26>] futex_wait_queue_me+0xc6/0x130
[<ffffffffbd3139cb>] futex_wait+0x17b/0x280
[<ffffffffbd2ca2d0>] ? hrtimer_get_res+0x50/0x50
[<ffffffffbd312c04>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffbd315716>] do_futex+0x106/0x5a0
[<ffffffffbd3fb8a7>] ? do_munmap+0x327/0x480
[<ffffffffbd315c30>] SyS_futex+0x80/0x190
[<ffffffffbd992ed2>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #198972
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff878d4ac7
Build id: 3.10.0-1160.25.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff87f88a77
Build id: 3.10.0-1160.25.1.el7.x86_64
0x3c7
3
schedule
vmlinux 0xffffffff87f88f49
Build id: 3.10.0-1160.25.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff879124d6
Build id: 3.10.0-1160.25.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff8791327b
Build id: 3.10.0-1160.25.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff878c99e0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff879124b4
Build id: 3.10.0-1160.25.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff87914fe6
Build id: 3.10.0-1160.25.1.el7.x86_64
0x106
9
? __seccomp_filter
vmlinux 0xffffffff879570ab
Build id: 3.10.0-1160.25.1.el7.x86_64
0x5b
10
? __switch_to
vmlinux 0xffffffff8782b621
Build id: 3.10.0-1160.25.1.el7.x86_64
0x151
11
SyS_futex
vmlinux 0xffffffff87915500
Build id: 3.10.0-1160.25.1.el7.x86_64
0x80
12
tracesys
vmlinux 0xffffffff87f96226
Build id: 3.10.0-1160.25.1.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [java:24407]
Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_REDIRECT nf_nat_redirect vxlan ip6_udp_tunnel udp_tunnel xt_mark xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM tun devlink rpcsec_gss_krb5 cmac auth_rpcgss nfsv4 arc4 md4 nfs lockd nls_utf8 grace cifs fscache ccm dns_resolver ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash
dm_log dm_mod nfit libnvdimm ppdev snd_intel8x0 iosf_mbi crc32_pclmul snd_ac97_codec ac97_bus ghash_clmulni_intel snd_seq snd_seq_device snd_pcm joydev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg snd_timer parport_pc snd parport i2c_piix4 pcspkr video soundcore ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul e1000 crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks fuse
CPU: 2 PID: 24407 Comm: java Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8ae24bd63180 ti: ffff8ae1d7060000 task.ti: ffff8ae1d7060000
RIP: 0010:[<ffffffff878d4ac7>]  [<ffffffff878d4ac7>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff8ae1d7063c08  EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff8ae1d7063bb8 RCX: 00000000c0000100
RDX: ffff8ae1d7061fd8 RSI: ffff8ae24bd63180 RDI: ffff8ae398c9acc0
RBP: ffff8ae1d7063c28 R08: ffff8ae1d7060000 R09: 0000000000000001
R10: 0000000000000002 R11: ffff8ae1e536bf00 R12: ffff8ae1d7063bb8
R13: ffffffff878e4bcc R14: ffff8ae1d7063bb0 R15: 0000000000000001
FS:  00007f68f21e1700(0000) GS:ffff8ae398c80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd3d20e1000 CR3: 00000005c5b80000 CR4: 00000000000606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff87f88a77>] __schedule+0x3c7/0x870
[<ffffffff87f88f49>] schedule+0x29/0x70
[<ffffffff879124d6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff8791327b>] futex_wait+0x17b/0x280
[<ffffffff878c99e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff879124b4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff87914fe6>] do_futex+0x106/0x5a0
[<ffffffff879570ab>] ? __seccomp_filter+0x5b/0x300
[<ffffffff8782b621>] ? __switch_to+0x151/0x580
[<ffffffff87915500>] SyS_futex+0x80/0x190
[<ffffffff87f96226>] tracesys+0xa6/0xcc
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df e8 c6 27 04 00 66 90 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #128598
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff9c8cc7f7
Build id: 3.10.0-862.14.4.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff9cf1890c
Build id: 3.10.0-862.14.4.el7.x86_64
0x41c
3
? hrtimer_start_range_ns
vmlinux 0xffffffff9c8c1e60
Build id: 3.10.0-862.14.4.el7.x86_64
0x1d0
4
schedule
vmlinux 0xffffffff9cf18f39
Build id: 3.10.0-862.14.4.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff9c9067f6
Build id: 3.10.0-862.14.4.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff9c9074db
Build id: 3.10.0-862.14.4.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff9c8c19c0
Build id: 3.10.0-862.14.4.el7.x86_64
0x60
8
? futex_wait_queue_me
vmlinux 0xffffffff9c9067d4
Build id: 3.10.0-862.14.4.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff9c909226
Build id: 3.10.0-862.14.4.el7.x86_64
0x106
10
SyS_futex
vmlinux 0xffffffff9c909740
Build id: 3.10.0-862.14.4.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffff9cf2579b
Build id: 3.10.0-862.14.4.el7.x86_64
0x22
12
? system_call_after_swapgs
vmlinux 0xffffffff9cf256e1
Build id: 3.10.0-862.14.4.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 118s! [ftdc:3412]
<EOI> 
Modules linked in:
xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxsf(OE) sunrpc joydev ppdev vmwgfx iosf_mbi snd_intel8x0 snd_ac97_codec ttm ac97_bus drm_kms_helper pcspkr snd_seq syscopyarea sysfillrect snd_seq_device sysimgblt fb_sys_fops snd_pcm drm video sg vboxguest(OE) snd_timer parport_pc i2c_piix4 snd parport soundcore i2c_core ip_tables xfs libcrc32c sd_mod
sr_mod crc_t10dif cdrom crct10dif_generic crct10dif_common ata_generic pata_acpi ahci libahci ata_piix libata e1000 crc32c_intel serio_raw dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 3412 Comm: ftdc Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9f530282bf40 ti: ffff9f51515c0000 task.ti: ffff9f51515c0000
RIP: 0010:[<ffffffff9c8cc7f7>]  [<ffffffff9c8cc7f7>] finish_task_switch+0x57/0x170
RSP: 0018:ffff9f51515c3c08  EFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff9f5310e18bc0 RCX: 00000000c0000100
RDX: ffff9f51515c1fd8 RSI: ffff9f530282bf40 RDI: ffff9f5310e98b40
RBP: ffff9f51515c3c28 R08: ffff9f51515c0000 R09: 0000000000000001
R10: ffff9f525433bfb8 R11: 0000000000000000 R12: ffffffff9c86a16e
R13: ffff9f51515c3b78 R14: 0000000000000000 R15: ffffffff9c8db7ec
FS:  00007fc476de5700(0000) GS:ffff9f5310e80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f48ae576028 CR3: 00000000d7724000 CR4: 00000000000206e0
Call Trace:
[<ffffffff9cf1890c>] __schedule+0x41c/0xa20
[<ffffffff9c8c1e60>] ? hrtimer_start_range_ns+0x1d0/0x3c0
[<ffffffff9cf18f39>] schedule+0x29/0x70
[<ffffffff9c9067f6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff9c9074db>] futex_wait+0x17b/0x280
[<ffffffff9c8c19c0>] ? hrtimer_get_res+0x60/0x60
[<ffffffff9c9067d4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff9c909226>] do_futex+0x106/0x5a0
[<ffffffff9c909740>] SyS_futex+0x80/0x180
[<ffffffff9cf2579b>] system_call_fastpath+0x22/0x27
[<ffffffff9cf256e1>] ? system_call_after_swapgs+0xae/0x146
Code: 1f 44 00 00 65 48 8b 34 25 80 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df ff 14 25 d0 07 43 9d fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #122548
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffaaad4c27
Build id: 3.10.0-1160.21.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffab18899f
Build id: 3.10.0-1160.21.1.el7.x86_64
0x3af
3
schedule
vmlinux 0xffffffffab188e79
Build id: 3.10.0-1160.21.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffaab123c6
Build id: 3.10.0-1160.21.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffaab1316b
Build id: 3.10.0-1160.21.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffaaac99e0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffffaab123a4
Build id: 3.10.0-1160.21.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffaab14ed6
Build id: 3.10.0-1160.21.1.el7.x86_64
0x106
9
? __check_object_size
vmlinux 0xffffffffaac4a7ff
Build id: 3.10.0-1160.21.1.el7.x86_64
0x14f
10
SyS_futex
vmlinux 0xffffffffaab153f0
Build id: 3.10.0-1160.21.1.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffffab195f92
Build id: 3.10.0-1160.21.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 35s! [mysqld:7774]
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 31s! [HangDetector:777]
Modules linked in: ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_ipv4 nf_log_common xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_nat_ftp bridge xt_REDIRECT stp nf_nat_redirect llc nf_conntrack_ftp xt_LOG xt_limit xt_multiport ebtable_filter ebtables devlink ip6table_filter ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 xt_nat iptable_nat nf_nat_ipv4 nf_nat tun nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter xt_conntrack nf_conntrack vmw_vsock_vmci_transport vsock sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul vmw_balloon glue_helper ablk_helper joydev cryptd pcspkr parport_pc sg parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod
cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix libata crc32c_intel serio_raw vmw_pvscsi vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 777 Comm: HangDetector Kdump: loaded Tainted: G             L ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8936f27a5280 ti: ffff8936eb7ec000 task.ti: ffff8936eb7ec000
RIP: 0010:[<ffffffffaaad4c27>]  [<ffffffffaaad4c27>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff8936eb7efc08  EFLAGS: 00000286
RAX: ffff8936e5f9d280 RBX: ffff8936eb7efbb8 RCX: 00000000c0000100
RDX: ffff8936eb7edfd8 RSI: ffff8936f27a5280 RDI: ffff8939bfd1acc0
RBP: ffff8936eb7efc28 R08: ffff8936eb7ec000 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000100 R12: ffff8936eb7efbb8
R13: ffffffffaaae4a8c R14: ffff8936eb7efbb0 R15: 0000000000000001
FS:  00007f2165f87700(0000) GS:ffff8939bfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000217f8a8 CR3: 0000000169ba4000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffab18899f>] __schedule+0x3af/0x860
[<ffffffffab188e79>] schedule+0x29/0x70
[<ffffffffaab123c6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffaab1316b>] futex_wait+0x17b/0x280
[<ffffffffaaac99e0>] ? hrtimer_get_res+0x50/0x50
[<ffffffffaab123a4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffaab14ed6>] do_futex+0x106/0x5a0
[<ffffffffaac4a7ff>] ? __check_object_size+0x14f/0x250
[<ffffffffaab153f0>] SyS_futex+0x80/0x190
[<ffffffffab195f92>] system_call_fastpath+0x25/0x2a
Complete report #64653
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810c28a7
Build id: 3.10.0-693.21.1.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff816b3b0c
Build id: 3.10.0-693.21.1.el7.x86_64
0x47c
3
? hrtimer_start_range_ns
vmlinux 0xffffffff810b7eb0
Build id: 3.10.0-693.21.1.el7.x86_64
0x1d0
4
schedule
vmlinux 0xffffffff816b40e9
Build id: 3.10.0-693.21.1.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff810f8d36
Build id: 3.10.0-693.21.1.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff810f9a1b
Build id: 3.10.0-693.21.1.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff810b7a10
Build id: 3.10.0-693.21.1.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff810f8d14
Build id: 3.10.0-693.21.1.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff810fb756
Build id: 3.10.0-693.21.1.el7.x86_64
0x106
10
SyS_futex
vmlinux 0xffffffff810fbc70
Build id: 3.10.0-693.21.1.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [mysqld:1607]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper
 syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul mptspi crct10dif_common crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1607 Comm: mysqld Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff881f0832af70 ti: ffff881efd7c4000 task.ti: ffff881efd7c4000
RIP: 0010:[<ffffffff810c28a7>]  [<ffffffff810c28a7>] finish_task_switch+0x57/0x170
RSP: 0018:ffff881efd7c7c08  EFLAGS: 00000282
RAX: ffff88024d420000 RBX: ffff881f8fd98ac0 RCX: 00000000c0000100
RDX: ffff881efd7c5fd8 RSI: ffff881f0832af70 RDI: ffff881f8fc18a40
RBP: ffff881efd7c7c28 R08: ffff881efd7c4000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: 0000000200000000
R13: 0000000000000000 R14: ffff881f8fd98a40 R15: ffffffff810d16ac
FS:  00007f2d0299f700(0000) GS:ffff881f8fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000556ebc460678 CR3: 0000001efcbc8000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 [<ffffffff816b3b0c>] __schedule+0x47c/0xa30
 [<ffffffff810b7eb0>] ? hrtimer_start_range_ns+0x1d0/0x3c0
 [<ffffffff816b40e9>] schedule+0x29/0x70
 [<ffffffff810f8d36>] futex_wait_queue_me+0xc6/0x130
 [<ffffffff810f9a1b>] futex_wait+0x17b/0x280
 [<ffffffff810b7a10>] ? hrtimer_get_res+0x50/0x50
 [<ffffffff810f8d14>] ? futex_wait_queue_me+0xa4/0x130
 [<ffffffff810fb756>] do_futex+0x106/0x5a0
 [<ffffffff810fbc70>] SyS_futex+0x80/0x180
 [<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 1f 44 00 00 65 48 8b 34 25 00 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 00 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #15603
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff91ac9664
Build id: 3.10.0-862.el7.x86_64
0x54
2
__schedule
vmlinux 0xffffffff9211291c
Build id: 3.10.0-862.el7.x86_64
0x41c
3
? hrtimer_start_range_ns
vmlinux 0xffffffff91abed10
Build id: 3.10.0-862.el7.x86_64
0x1d0
4
schedule
vmlinux 0xffffffff92112f49
Build id: 3.10.0-862.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff91b03556
Build id: 3.10.0-862.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff91b0423b
Build id: 3.10.0-862.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff91abe870
Build id: 3.10.0-862.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff91b03534
Build id: 3.10.0-862.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff91b05f96
Build id: 3.10.0-862.el7.x86_64
0x106
10
? update_curr
vmlinux 0xffffffff91ad816c
Build id: 3.10.0-862.el7.x86_64
0x14c
11
SyS_futex
vmlinux 0xffffffff91b064b0
Build id: 3.10.0-862.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff9211f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
13
? system_call_after_swapgs
vmlinux 0xffffffff9211f721
Build id: 3.10.0-862.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 25s! [java:7443]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 tun ipt_MASQUERADE nf_nat_masquerade_ipv4 ebtable_filter iptable_nat ebtables nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype xt_conntrack ip6table_filter nf_nat ip6_tables nf_conntrack iptable_filter br_netfilter bridge stp llc devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev sg pcspkr floppy parport_pc parport i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi drm scsi_transport_spi crct10dif_pclmul
crct10dif_common mptscsih e1000 crc32c_intel libata serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 7443 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9fef4d6aaf70 ti: ffff9fee8db9c000 task.ti: ffff9fee8db9c000
RIP: 0010:[<ffffffff91ac9664>]  [<ffffffff91ac9664>] finish_task_switch+0x54/0x170
RSP: 0000:ffff9fee8db9fc08  EFLAGS: 00000286
RAX: ffff9fe8b4521fa0 RBX: ffff9fef5ee58b80 RCX: 00000000c0000100
RDX: ffff9fee8db9dfd8 RSI: ffff9fef4d6aaf70 RDI: ffff9fef5ee58b00
RBP: ffff9fee8db9fc28 R08: ffff9fee8db9c000 R09: 0000000000000000
R10: 000000000000b804 R11: 0000000000000001 R12: 0000000200000000
R13: 0000000000000000 R14: ffff9fef5ee58b00 R15: ffffffff91ad865c
FS:  00007f25a4817700(0000) GS:ffff9fef5ee40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f584eaefcb8 CR3: 000000081a2c8000 CR4: 00000000000007e0
Call Trace:
[<ffffffff9211291c>] __schedule+0x41c/0xa20
[<ffffffff91abed10>] ? hrtimer_start_range_ns+0x1d0/0x3c0
[<ffffffff92112f49>] schedule+0x29/0x70
[<ffffffff91b03556>] futex_wait_queue_me+0xc6/0x130
[<ffffffff91b0423b>] futex_wait+0x17b/0x280
[<ffffffff91abe870>] ? hrtimer_get_res+0x50/0x50
[<ffffffff91b03534>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff91b05f96>] do_futex+0x106/0x5a0
[<ffffffff91ad816c>] ? update_curr+0x14c/0x1e0
[<ffffffff91b064b0>] SyS_futex+0x80/0x180
[<ffffffff9211f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff9211f721>] ? system_call_after_swapgs+0xae/0x146
Code: 8b 36 66 66 66 66 90 65 48 8b 34 25 40 0e 01 00 66 66 66 66 90 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 65 48 8b 04 25 40 0e 01 00 48 8b 98 78 01 00 00 48 85 
Complete report #13342
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff92cc9664
Build id: 3.10.0-862.2.3.el7.x86_64
0x54
2
__schedule
vmlinux 0xffffffff9331294c
Build id: 3.10.0-862.2.3.el7.x86_64
0x41c
3
? hrtimer_start_range_ns
vmlinux 0xffffffff92cbed10
Build id: 3.10.0-862.2.3.el7.x86_64
0x1d0
4
schedule
vmlinux 0xffffffff93312f79
Build id: 3.10.0-862.2.3.el7.x86_64
0x29
5
futex_wait_queue_me
vmlinux 0xffffffff92d03556
Build id: 3.10.0-862.2.3.el7.x86_64
0xc6
6
futex_wait
vmlinux 0xffffffff92d0423b
Build id: 3.10.0-862.2.3.el7.x86_64
0x17b
7
? hrtimer_get_res
vmlinux 0xffffffff92cbe870
Build id: 3.10.0-862.2.3.el7.x86_64
0x50
8
? futex_wait_queue_me
vmlinux 0xffffffff92d03534
Build id: 3.10.0-862.2.3.el7.x86_64
0xa4
9
do_futex
vmlinux 0xffffffff92d05f96
Build id: 3.10.0-862.2.3.el7.x86_64
0x106
10
SyS_futex
vmlinux 0xffffffff92d064b0
Build id: 3.10.0-862.2.3.el7.x86_64
0x80
11
system_call_fastpath
vmlinux 0xffffffff9331f795
Build id: 3.10.0-862.2.3.el7.x86_64
0x1c
12
? system_call_after_swapgs
vmlinux 0xffffffff9331f6e1
Build id: 3.10.0-862.2.3.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [mysqld:7088]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev nfit pcspkr joydev sg vmw_balloon libnvdimm shpchp parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c
sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul crct10dif_common drm libata crc32c_intel serio_raw e1000 mptspi scsi_transport_spi mptscsih mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 7088 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff98e5b9b48000 ti: ffff98e58d97c000 task.ti: ffff98e58d97c000
RIP: 0010:[<ffffffff92cc9664>]  [<ffffffff92cc9664>] finish_task_switch+0x54/0x170
RSP: 0000:ffff98e58d97fc08  EFLAGS: 00000286
RAX: ffff98e6936d2f70 RBX: ffff98e6bfc98b80 RCX: 00000000c0000100
RDX: ffff98e58d97dfd8 RSI: ffff98e5b9b48000 RDI: ffff98e6bfc98b00
RBP: ffff98e58d97fc28 R08: ffff98e58d97c000 R09: 0000000000000001
R10: 000000000000bc55 R11: 0000000000000001 R12: 0000000200000000
R13: 0000000000000000 R14: ffff98e6bfd18b00 R15: ffffffff92cd865c
FS:  00007fb20c390700(0000) GS:ffff98e6bfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fe3cc98d038 CR3: 0000000188ece000 CR4: 00000000000607e0
Call Trace:
[<ffffffff9331294c>] __schedule+0x41c/0xa20
[<ffffffff92cbed10>] ? hrtimer_start_range_ns+0x1d0/0x3c0
[<ffffffff93312f79>] schedule+0x29/0x70
[<ffffffff92d03556>] futex_wait_queue_me+0xc6/0x130
[<ffffffff92d0423b>] futex_wait+0x17b/0x280
[<ffffffff92cbe870>] ? hrtimer_get_res+0x50/0x50
[<ffffffff92d03534>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff92d05f96>] do_futex+0x106/0x5a0
[<ffffffff92d064b0>] SyS_futex+0x80/0x180
[<ffffffff9331f795>] system_call_fastpath+0x1c/0x21
[<ffffffff9331f6e1>] ? system_call_after_swapgs+0xae/0x146
Code: 8b 36 66 66 66 66 90 65 48 8b 34 25 40 0e 01 00 66 66 66 66 90 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 65 48 8b 04 25 40 0e 01 00 48 8b 98 78 01 00 00 48 85 
Complete report #10623
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffffb48cc7f7
Build id: 3.10.0-862.14.4.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffffb4f1890c
Build id: 3.10.0-862.14.4.el7.x86_64
0x41c
3
schedule
vmlinux 0xffffffffb4f18f39
Build id: 3.10.0-862.14.4.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffffb49067f6
Build id: 3.10.0-862.14.4.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffffb49074db
Build id: 3.10.0-862.14.4.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffffb48c19c0
Build id: 3.10.0-862.14.4.el7.x86_64
0x60
7
? futex_wait_queue_me
vmlinux 0xffffffffb49067d4
Build id: 3.10.0-862.14.4.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffffb4909226
Build id: 3.10.0-862.14.4.el7.x86_64
0x106
9
SyS_futex
vmlinux 0xffffffffb4909740
Build id: 3.10.0-862.14.4.el7.x86_64
0x80
10
system_call_fastpath
vmlinux 0xffffffffb4f2579b
Build id: 3.10.0-862.14.4.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 30s! [abrtd:697]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 43s! [java:1851]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_seq_midi snd_seq_midi_event lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer ppdev snd vmw_balloon soundcore pcspkr nfit sg joydev libnvdimm parport_pc parport vmw_vmci shpchp i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi serio_raw scsi_transport_spi e1000 mptscsih ata_piix libata i2c_core mptbase dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1851 Comm: java Tainted: G        W    L ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9eb7346c0000 ti: ffff9eb73a698000 task.ti: ffff9eb73a698000
RIP: 0010:[<ffffffffb48cc7f7>]  [<ffffffffb48cc7f7>] finish_task_switch+0x57/0x170
RSP: 0000:ffff9eb73a69bc08  EFLAGS: 00000282
RAX: ffff9eb6bab6dee0 RBX: ffffffffb4f18af8 RCX: 00000000c0000100
RDX: ffff9eb73a699fd8 RSI: ffff9eb7346c0000 RDI: ffff9eb76d618b40
RBP: ffff9eb73a69bc28 R08: ffff9eb73a698000 R09: 0000000000000001
R10: 000000000000ba95 R11: 0000000000000001 R12: ffffffffb4f18af8
R13: ffffffffb4f18b04 R14: ffffffffb4f18af8 R15: ffffffffb4f18b04
FS:  00007fd63b61d700(0000) GS:ffff9eb76d600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd2ea2ba1f0 CR3: 00000003f97b6000 CR4: 00000000003607f0
Call Trace:
[<ffffffffb4f1890c>] __schedule+0x41c/0xa20
[<ffffffffb4f18f39>] schedule+0x29/0x70
[<ffffffffb49067f6>] futex_wait_queue_me+0xc6/0x130
[<ffffffffb49074db>] futex_wait+0x17b/0x280
[<ffffffffb48c19c0>] ? hrtimer_get_res+0x60/0x60
[<ffffffffb49067d4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffffb4909226>] do_futex+0x106/0x5a0
[<ffffffffb4909740>] SyS_futex+0x80/0x180
[<ffffffffb4f2579b>] system_call_fastpath+0x22/0x27
Code: 1f 44 00 00 65 48 8b 34 25 80 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #10433
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff94cd5407
Build id: 3.10.0-1127.8.2.el7.x86_64
0x57
2
__schedule
vmlinux 0xffffffff95385942
Build id: 3.10.0-1127.8.2.el7.x86_64
0x402
3
schedule
vmlinux 0xffffffff95385da9
Build id: 3.10.0-1127.8.2.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff94d12bd6
Build id: 3.10.0-1127.8.2.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff94d1397b
Build id: 3.10.0-1127.8.2.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff94cca2d0
Build id: 3.10.0-1127.8.2.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff94d12bb4
Build id: 3.10.0-1127.8.2.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff94d156c6
Build id: 3.10.0-1127.8.2.el7.x86_64
0x106
9
? put_prev_entity
vmlinux 0xffffffff94ce69fb
Build id: 3.10.0-1127.8.2.el7.x86_64
0x8b
10
? __switch_to
vmlinux 0xffffffff94c2b621
Build id: 3.10.0-1127.8.2.el7.x86_64
0x151
11
SyS_futex
vmlinux 0xffffffff94d15be0
Build id: 3.10.0-1127.8.2.el7.x86_64
0x80
12
system_call_fastpath
vmlinux 0xffffffff95392ed2
Build id: 3.10.0-1127.8.2.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [AsyncReporter{S:88555]
Modules linked in: veth dummy vport_vxlan vxlan ip6_udp_tunnel udp_tunnel openvswitch xt_nat xt_statistic ipt_REJECT nf_reject_ipv4 xt_NFLOG xt_physdev xt_state nfnetlink_log ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs ip_set_hash_ip xt_set ip_set nf_tables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c
sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libahci drm libata nfit serio_raw vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 88555 Comm: AsyncReporter{S Kdump: loaded Tainted: G             L ------------ T 3.10.0-1127.8.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff96b8283ea0e0 ti: ffff9698a9b84000 task.ti: ffff9698a9b84000
RIP: 0010:[<ffffffff94cd5407>]  [<ffffffff94cd5407>] finish_task_switch+0x57/0x1c0
RSP: 0018:ffff9698a9b87c08  EFLAGS: 00000286
RAX: ffff96b806cc5230 RBX: ffff9698a9b87bb8 RCX: 00000000c0000100
RDX: ffff9698a9b85fd8 RSI: ffff96b8283ea0e0 RDI: ffff96b83d71acc0
RBP: ffff9698a9b87c28 R08: ffff9698a9b84000 R09: 0000000000000001
R10: 0000000000000003 R11: 0000000000000000 R12: ffff9698a9b87bb8
R13: ffffffff94ce0628 R14: ffff9698a9b87b70 R15: ffff96b83867e900
FS:  00007f03ce7fc700(0000) GS:ffff96b83d700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8a1e146008 CR3: 0000001bb865e000 CR4: 00000000001607e0
Call Trace:
[<ffffffff95385942>] __schedule+0x402/0x840
[<ffffffff95385da9>] schedule+0x29/0x70
[<ffffffff94d12bd6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff94d1397b>] futex_wait+0x17b/0x280
[<ffffffff94cca2d0>] ? hrtimer_get_res+0x50/0x50
[<ffffffff94d12bb4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff94d156c6>] do_futex+0x106/0x5a0
[<ffffffff94ce69fb>] ? put_prev_entity+0x8b/0x400
[<ffffffff94c2b621>] ? __switch_to+0x151/0x580
[<ffffffff94d15be0>] SyS_futex+0x80/0x190
[<ffffffff95392ed2>] system_call_fastpath+0x25/0x2a
Code: 1f 44 00 00 65 48 8b 34 25 c0 0e 01 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 0f 1f 40 00 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0e 01 00 48 8b 98 78 01 00 00 48 85 db 74 22 
Complete report #8501
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff810bfde6
Build id: 3.10.0-514.26.2.el7.x86_64
0x56
2
__schedule
vmlinux 0xffffffff8168c1f0
Build id: 3.10.0-514.26.2.el7.x86_64
0x3b0
3
schedule
vmlinux 0xffffffff8168c7f9
Build id: 3.10.0-514.26.2.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff810f5464
Build id: 3.10.0-514.26.2.el7.x86_64
0xc4
5
futex_wait
vmlinux 0xffffffff810f5fd9
Build id: 3.10.0-514.26.2.el7.x86_64
0x179
6
? hrtimer_get_res
vmlinux 0xffffffff810b4640
Build id: 3.10.0-514.26.2.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff810f5442
Build id: 3.10.0-514.26.2.el7.x86_64
0xa2
8
do_futex
vmlinux 0xffffffff810f80de
Build id: 3.10.0-514.26.2.el7.x86_64
0xfe
9
SyS_futex
vmlinux 0xffffffff810f8610
Build id: 3.10.0-514.26.2.el7.x86_64
0x80
10
system_call_fastpath
vmlinux 0xffffffff81697809
Build id: 3.10.0-514.26.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [java:11661]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd nfit ppdev vmw_balloon sg pcspkr libnvdimm i2c_piix4 vmw_vmci parport_pc shpchp parport xfs dm_thin_pool dm_persistent_data dm_bufio dm_bio_prison libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic
ata_generic pata_acpi crct10dif_pclmul vmwgfx crct10dif_common crc32c_intel drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi ata_piix drm mptscsih libata mptbase vmxnet3 i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 11661 Comm: java Tainted: G             L ------------   3.10.0-514.26.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
task: ffff88031f2ede20 ti: ffff8804273e4000 task.ti: ffff8804273e4000
RIP: 0010:[<ffffffff810bfde6>]  [<ffffffff810bfde6>] finish_task_switch+0x56/0x180
RSP: 0018:ffff8804273e7c50  EFLAGS: 00000206
RAX: ffff88017a1e0fb0 RBX: ffff88043fc16c40 RCX: 00000000c0000100
RDX: ffff8804273e5fd8 RSI: ffff88031f2ede20 RDI: ffff88043fc16c40
RBP: ffff8804273e7c70 R08: ffff8804273e4000 R09: 0000000000000000
R10: 0000000000000006 R11: 0000000000000000 R12: ffff88031f2ede88
R13: 0000000200000000 R14: 0000000000000000 R15: ffff88043fcd6c40
FS:  00007f0660967700(0000) GS:ffff88043fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f075402b00a CR3: 00000001c67ba000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88042c39e400 ffff88043fc16c40 ffff88042c39e400 0000000000000000
ffff8804273e7cd0 ffffffff8168c1f0 ffff88031f2ede20 ffff8804273e7fd8
ffff8804273e7fd8 ffff8804273e7fd8 ffff88031f2ede20 ffff88031f2ede20
Call Trace:
[<ffffffff8168c1f0>] __schedule+0x3b0/0x990
[<ffffffff8168c7f9>] schedule+0x29/0x70
[<ffffffff810f5464>] futex_wait_queue_me+0xc4/0x120
[<ffffffff810f5fd9>] futex_wait+0x179/0x280
[<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[<ffffffff810f5442>] ? futex_wait_queue_me+0xa2/0x120
[<ffffffff810f80de>] do_futex+0xfe/0x5b0
[<ffffffff810f8610>] SyS_futex+0x80/0x180
[<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Code: 0f 1f 44 00 00 65 48 8b 34 25 c0 cd 00 00 0f 1f 44 00 00 41 c7 45 28 00 00 00 00 0f 1f 44 00 00 66 83 03 02 fb 66 0f 1f 44 00 00 <65> 48 8b 04 25 c0 cd 00 00 48 8b 98 78 01 00 00 48 85 db 74 1e 
Complete report #5386
Frame # Function Binary Source or offset Line
1
finish_task_switch
vmlinux 0xffffffff83cd5134
Build id: 3.10.0-1062.18.1.el7.x86_64
0x54
2
__schedule
vmlinux 0xffffffff84380d92
Build id: 3.10.0-1062.18.1.el7.x86_64
0x402
3
schedule
vmlinux 0xffffffff843811f9
Build id: 3.10.0-1062.18.1.el7.x86_64
0x29
4
futex_wait_queue_me
vmlinux 0xffffffff83d122c6
Build id: 3.10.0-1062.18.1.el7.x86_64
0xc6
5
futex_wait
vmlinux 0xffffffff83d1306b
Build id: 3.10.0-1062.18.1.el7.x86_64
0x17b
6
? hrtimer_get_res
vmlinux 0xffffffff83cc9f60
Build id: 3.10.0-1062.18.1.el7.x86_64
0x50
7
? futex_wait_queue_me
vmlinux 0xffffffff83d122a4
Build id: 3.10.0-1062.18.1.el7.x86_64
0xa4
8
do_futex
vmlinux 0xffffffff83d14db6
Build id: 3.10.0-1062.18.1.el7.x86_64
0x106
9
SyS_futex
vmlinux 0xffffffff83d152d0
Build id: 3.10.0-1062.18.1.el7.x86_64
0x80
10
system_call_fastpath
vmlinux 0xffffffff8438dede
Build id: 3.10.0-1062.18.1.el7.x86_64
0x25
11
? system_call_after_swapgs
vmlinux 0xffffffff8438de21
Build id: 3.10.0-1062.18.1.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [hz._hzInstance_:2789]
hrtimer: interrupt took 1627301229 ns
Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm dns_resolver binfmt_misc fuse devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod
crc_t10dif crct10dif_generic vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ata_piix libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2789 Comm: hz._hzInstance_ Kdump: loaded Not tainted 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff898dcdc362a0 ti: ffff898dcdd78000 task.ti: ffff898dcdd78000
RIP: 0010:[<ffffffff83cd5134>]  [<ffffffff83cd5134>] finish_task_switch+0x54/0x1c0
RSP: 0000:ffff898dcdd7bc08  EFLAGS: 00000282
RAX: ffff898d57911070 RBX: ffff898dcdd7bbb8 RCX: 00000000c0000100
RDX: ffff898dcdd79fd8 RSI: ffff898dcdc362a0 RDI: ffff898df56dac80
RBP: ffff898dcdd7bc28 R08: ffff898dcdd78000 R09: 0000000000000001
R10: 000000000000002c R11: 0000000000000000 R12: ffff898dcdd7bbb8
R13: ffffffff83ce4ffc R14: ffff898dcdd7bbb0 R15: 0000000000000001
FS:  00007f8d30f59700(0000) GS:ffff898df56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f73ac3b0bf0 CR3: 000000020dd96000 CR4: 00000000000007e0
Call Trace:
[<ffffffff84380d92>] __schedule+0x402/0x840
[<ffffffff843811f9>] schedule+0x29/0x70
[<ffffffff83d122c6>] futex_wait_queue_me+0xc6/0x130
[<ffffffff83d1306b>] futex_wait+0x17b/0x280
[<ffffffff83cc9f60>] ? hrtimer_get_res+0x50/0x50
[<ffffffff83d122a4>] ? futex_wait_queue_me+0xa4/0x130
[<ffffffff83d14db6>] do_futex+0x106/0x5a0
[<ffffffff83d152d0>] SyS_futex+0x80/0x190
[<ffffffff8438dede>] system_call_fastpath+0x25/0x2a
[<ffffffff8438de21>] ? system_call_after_swapgs+0xae/0x146
Code: 8b 36 66 66 66 66 90 65 48 8b 34 25 80 0e 01 00 66 66 66 66 90 41 c7 45 28 00 00 00 00 48 89 df c6 07 00 66 66 66 90 fb 66 66 90 <66> 66 90 65 48 8b 04 25 80 0e 01 00 48 8b 98 78 01 00 00 48 85