Info

Permalink
Function
native_queued_spin_lock_slowpath
First occurrence
2021-01-09
Last occurrence
2026-08-12
State
NEW
Components
kernel
Quality
-23

Statistics

Operating system Count
CentOS 7 70
Architecture Count
x86_64 70
Related packages Count
kernel 70
 0:3.10.0-1160.el7 40
 0:3.10.0-693.21.1.el7 11
 0:3.10.0-1127.el7 5
 0:3.10.0-1160.76.1.el7 4
 0:3.10.0-1127.10.1.el7 2
 0:3.10.0-862.el7 2
 0:3.10.0-1062.el7 2
 0:3.10.0-1062.9.1.el7 1
 0:3.10.0-693.el7 1
 0:3.10.0-1160.25.1.el7 1
 0:3.10.0-957.12.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #659979
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffa5d17ecd
Build id: 3.10.0-1127.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffa637a004
Build id: 3.10.0-1127.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffa63886f3
Build id: 3.10.0-1127.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffffa62375e0
Build id: 3.10.0-1127.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffa62ad0d3
Build id: 3.10.0-1127.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffffa5f0b155
Build id: 3.10.0-1127.el7.x86_64
0x75
7
inet_recvmsg
vmlinux 0xffffffffa62dc520
Build id: 3.10.0-1127.el7.x86_64
0x80
8
sock_recvmsg
vmlinux 0xffffffffa62334f5
Build id: 3.10.0-1127.el7.x86_64
0xc5
9
? __pollwait
vmlinux 0xffffffffa5e63380
Build id: 3.10.0-1127.el7.x86_64
0xf0
10
? __schedule
vmlinux 0xffffffffa6385824
Build id: 3.10.0-1127.el7.x86_64
0x304
11
SYSC_recvfrom
vmlinux 0xffffffffa6233668
Build id: 3.10.0-1127.el7.x86_64
0xe8
12
SyS_recvfrom
vmlinux 0xffffffffa623561e
Build id: 3.10.0-1127.el7.x86_64
0xe
13
system_call_fastpath
vmlinux 0xffffffffa6392ed2
Build id: 3.10.0-1127.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [Socket Thread:85162]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libnvdimm e1000 mptspi libata scsi_transport_spi
mptscsih serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 85162 Comm: Socket Thread Kdump: loaded Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff94aab225d230 ti: ffff94aab2e04000 task.ti: ffff94aab2e04000
RIP: 0010:[<ffffffffa5d17ecd>]  [<ffffffffa5d17ecd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff94aab2e07ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff94aaeab217b0
RBP: ffff94aab2e07ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 000000000000094f R11: 0000000000000000 R12: 0000000000000000
R13: ffff94aab225d230 R14: ffff94aab2e07fd8 R15: 00007f97da96a900
FS:  00007f97da96b700(0000) GS:ffff94ab29680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd21bbf4000 CR3: 0000000522e52000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa637a004>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa63886f3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffffa62375e0>] lock_sock_nested+0x20/0x50
[<ffffffffa62ad0d3>] tcp_recvmsg+0x73/0xb30
[<ffffffffa5f0b155>] ? sock_has_perm+0x75/0x90
[<ffffffffa62dc520>] inet_recvmsg+0x80/0xb0
[<ffffffffa62334f5>] sock_recvmsg+0xc5/0x100
[<ffffffffa5e63380>] ? __pollwait+0xf0/0xf0
[<ffffffffa6385824>] ? __schedule+0x304/0x840
[<ffffffffa6233668>] SYSC_recvfrom+0xe8/0x160
[<ffffffffa623561e>] SyS_recvfrom+0xe/0x10
[<ffffffffa6392ed2>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #609501
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff93f17a4d
Build id: 3.10.0-1160.76.1.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffff94580d83
Build id: 3.10.0-1160.76.1.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff9458ed63
Build id: 3.10.0-1160.76.1.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff9443d5d0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff944b3aa3
Build id: 3.10.0-1160.76.1.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffff9410c685
Build id: 3.10.0-1160.76.1.el7.x86_64
0x75
7
inet_recvmsg
vmlinux 0xffffffff944e2ff0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x80
8
sock_recvmsg
vmlinux 0xffffffff944394e5
Build id: 3.10.0-1160.76.1.el7.x86_64
0xc5
9
? timerqueue_del
vmlinux 0xffffffff941911d4
Build id: 3.10.0-1160.76.1.el7.x86_64
0x24
10
SYSC_recvfrom
vmlinux 0xffffffff94439658
Build id: 3.10.0-1160.76.1.el7.x86_64
0xe8
11
? wake_up_state
vmlinux 0xffffffff93edb2e0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x20
12
SyS_recvfrom
vmlinux 0xffffffff9443b60e
Build id: 3.10.0-1160.76.1.el7.x86_64
0xe
13
system_call_fastpath
vmlinux 0xffffffff94599f92
Build id: 3.10.0-1160.76.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 65s! [sunloginclient:1194]
Modules linked in: tcp_diag udp_diag inet_diag veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock overlay(T) sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus crc32_pclmul snd_seq ppdev snd_seq_device ghash_clmulni_intel snd_pcm vmw_balloon aesni_intel lrw snd_timer gf128mul glue_helper ablk_helper cryptd snd soundcore joydev sg pcspkr vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit ata_piix mptspi scsi_transport_spi libnvdimm crct10dif_pclmul mptscsih crct10dif_common libata crc32c_intel e1000 mptbase serio_raw drm_panel_orientation_quirks fuse
CPU: 8 PID: 1194 Comm: sunloginclient Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9c7bc8918000 ti: ffff9c7bc7794000 task.ti: ffff9c7bc7794000
RIP: 0010:[<ffffffff93f17a4d>]  [<ffffffff93f17a4d>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff9c7bc7797ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffffffff945856d6 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9c7bc2936530
RBP: ffff9c7bc7797ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000020 R11: 0000000000000000 R12: 0000000000000202
R13: 0000000000000202 R14: ffff9c7bc7797ba4 R15: 0000000000000079
FS:  00007fba020dd700(0000) GS:ffff9c7bdd800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00084d000 CR3: 000000080c8a2000 CR4: 0000000000340fe0
Call Trace:
[<ffffffff94580d83>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9458ed63>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff9443d5d0>] lock_sock_nested+0x20/0x50
[<ffffffff944b3aa3>] tcp_recvmsg+0x73/0xb30
[<ffffffff9410c685>] ? sock_has_perm+0x75/0x90
[<ffffffff944e2ff0>] inet_recvmsg+0x80/0xb0
[<ffffffff944394e5>] sock_recvmsg+0xc5/0x100
[<ffffffff941911d4>] ? timerqueue_del+0x24/0x70
[<ffffffff94439658>] SYSC_recvfrom+0xe8/0x160
[<ffffffff93edb2e0>] ? wake_up_state+0x20/0x20
[<ffffffff9443b60e>] SyS_recvfrom+0xe/0x10
[<ffffffff94599f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #173235
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff93f177af
Build id: 3.10.0-1160.25.1.el7.x86_64
0x1f
2
queued_spin_lock_slowpath
vmlinux 0xffffffff9457d155
Build id: 3.10.0-1160.25.1.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff9458b8b3
Build id: 3.10.0-1160.25.1.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff9443a5e0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff944aff23
Build id: 3.10.0-1160.25.1.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffff9410c2a5
Build id: 3.10.0-1160.25.1.el7.x86_64
0x75
7
inet_recvmsg
vmlinux 0xffffffff944df430
Build id: 3.10.0-1160.25.1.el7.x86_64
0x80
8
sock_recvmsg
vmlinux 0xffffffff944364f5
Build id: 3.10.0-1160.25.1.el7.x86_64
0xc5
9
? ttwu_do_activate
vmlinux 0xffffffff93ed757f
Build id: 3.10.0-1160.25.1.el7.x86_64
0x6f
10
? wake_up_q
vmlinux 0xffffffff93edadab
Build id: 3.10.0-1160.25.1.el7.x86_64
0x5b
11
SYSC_recvfrom
vmlinux 0xffffffff94436668
Build id: 3.10.0-1160.25.1.el7.x86_64
0xe8
12
? do_vfs_ioctl
vmlinux 0xffffffff94063590
Build id: 3.10.0-1160.25.1.el7.x86_64
0x3a0
13
SyS_recvfrom
vmlinux 0xffffffff9443861e
Build id: 3.10.0-1160.25.1.el7.x86_64
0xe
14
system_call_fastpath
vmlinux 0xffffffff94595f92
Build id: 3.10.0-1160.25.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [Socket Thread:10080]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event vfat fat xfs iosf_mbi libcrc32c kvm_intel kvm snd_ens1371 snd_rawmidi snd_ac97_codec irqbypass ac97_bus crc32_pclmul snd_seq ghash_clmulni_intel snd_seq_device ppdev snd_pcm aesni_intel lrw gf128mul
snd_timer glue_helper vmw_balloon ablk_helper cryptd snd soundcore pcspkr joydev vmw_vmci sg i2c_piix4 tpm_crb parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit ata_piix libnvdimm serio_raw libata mptspi scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 10080 Comm: Socket Thread Kdump: loaded Tainted: G             L ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.16221537.B64.2005150253 05/15/2020
task: ffff8ce382502100 ti: ffff8ce22ec78000 task.ti: ffff8ce22ec78000
RIP: 0010:[<ffffffff93f177af>]  [<ffffffff93f177af>] native_queued_spin_lock_slowpath+0x1f/0x200
RSP: 0018:ffff8ce22ec7bba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8ce3b3fe6530
RBP: ffff8ce22ec7bba0 R08: 0000000000000040 R09: 0000000000000000
R10: 000000000000097c R11: 0000000000000000 R12: ffff8ce22ec7bf18
R13: ffff8ce382502100 R14: ffff8ce22ec7bfd8 R15: 00007f7a9df62740
FS:  00007f7a9df63700(0000) GS:ffff8ce3b5640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7a883dbff0 CR3: 00000002024fe000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9457d155>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9458b8b3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff9443a5e0>] lock_sock_nested+0x20/0x50
[<ffffffff944aff23>] tcp_recvmsg+0x73/0xb30
[<ffffffff9410c2a5>] ? sock_has_perm+0x75/0x90
[<ffffffff944df430>] inet_recvmsg+0x80/0xb0
[<ffffffff944364f5>] sock_recvmsg+0xc5/0x100
[<ffffffff93ed757f>] ? ttwu_do_activate+0x6f/0x80
[<ffffffff93edadab>] ? wake_up_q+0x5b/0x80
[<ffffffff94436668>] SYSC_recvfrom+0xe8/0x160
[<ffffffff94063590>] ? do_vfs_ioctl+0x3a0/0x5b0
[<ffffffff9443861e>] SyS_recvfrom+0xe/0x10
[<ffffffff94595f92>] system_call_fastpath+0x25/0x2a
Code: ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 8b 17 <85> d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 1f 80 
Complete report #995542
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffacd1598d
Build id: 3.10.0-1062.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffad373365
Build id: 3.10.0-1062.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffad381b13
Build id: 3.10.0-1062.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffffad2315e0
Build id: 3.10.0-1062.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffad2a7783
Build id: 3.10.0-1062.el7.x86_64
0x73
6
inet_recvmsg
vmlinux 0xffffffffad2d6ae0
Build id: 3.10.0-1062.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffffad22d4f5
Build id: 3.10.0-1062.el7.x86_64
0xc5
8
? lru_cache_add
vmlinux 0xffffffffacdcb46e
Build id: 3.10.0-1062.el7.x86_64
0xe
9
? page_add_new_anon_rmap
vmlinux 0xffffffffacdfd348
Build id: 3.10.0-1062.el7.x86_64
0xb8
10
SYSC_recvfrom
vmlinux 0xffffffffad22d668
Build id: 3.10.0-1062.el7.x86_64
0xe8
11
? ktime_get_ts64
vmlinux 0xffffffffacd047e2
Build id: 3.10.0-1062.el7.x86_64
0x52
12
SyS_recvfrom
vmlinux 0xffffffffad22f61e
Build id: 3.10.0-1062.el7.x86_64
0xe
13
system_call_fastpath
vmlinux 0xffffffffad38bede
Build id: 3.10.0-1062.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [reporter-urepor:60541]
Modules linked in: nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc ext4 snd_seq_midi snd_seq_midi_event mbcache jbd2 snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel snd_timer lrw gf128mul snd glue_helper ablk_helper cryptd joydev soundcore pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix nfit drm crct10dif_pclmul crct10dif_common libata mptspi crc32c_intel libnvdimm scsi_transport_spi e1000 mptscsih mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 14 PID: 60541 Comm: reporter-urepor Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff95bc7af362a0 ti: ffff95be9f36c000 task.ti: ffff95be9f36c000
RIP: 0010:[<ffffffffacd1598d>]  [<ffffffffacd1598d>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff95be9f36fba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff95bb61392ef0
RBP: ffff95be9f36fba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: ffff95be9f36ff18
R13: ffff95bc7af362a0 R14: ffff95be9f36ffd8 R15: 00007fff52b46ce0
FS:  00007f91a62da8c0(0000) GS:ffff95bfbd980000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fba38640000 CR3: 0000000895e24000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffad373365>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffad381b13>] _raw_spin_lock_bh+0x33/0x40
[<ffffffffad2315e0>] lock_sock_nested+0x20/0x50
[<ffffffffad2a7783>] tcp_recvmsg+0x73/0xb30
[<ffffffffad2d6ae0>] inet_recvmsg+0x80/0xb0
[<ffffffffad22d4f5>] sock_recvmsg+0xc5/0x100
[<ffffffffacdcb46e>] ? lru_cache_add+0xe/0x10
[<ffffffffacdfd348>] ? page_add_new_anon_rmap+0xb8/0x170
[<ffffffffad22d668>] SYSC_recvfrom+0xe8/0x160
[<ffffffffacd047e2>] ? ktime_get_ts64+0x52/0xf0
[<ffffffffad22f61e>] SyS_recvfrom+0xe/0x10
[<ffffffffad38bede>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #985941
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff9a9174cd
Build id: 3.10.0-1160.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffff9af7b37b
Build id: 3.10.0-1160.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff9af89ad3
Build id: 3.10.0-1160.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff9ae385e0
Build id: 3.10.0-1160.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff9aeae293
Build id: 3.10.0-1160.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffff9ab0be65
Build id: 3.10.0-1160.el7.x86_64
0x75
7
inet_recvmsg
vmlinux 0xffffffff9aedd730
Build id: 3.10.0-1160.el7.x86_64
0x80
8
sock_recvmsg
vmlinux 0xffffffff9ae344f5
Build id: 3.10.0-1160.el7.x86_64
0xc5
9
? sock_sendmsg
vmlinux 0xffffffff9ae343a6
Build id: 3.10.0-1160.el7.x86_64
0xb6
10
SYSC_recvfrom
vmlinux 0xffffffff9ae34668
Build id: 3.10.0-1160.el7.x86_64
0xe8
11
? poll_select_copy_remaining
vmlinux 0xffffffff9aa64134
Build id: 3.10.0-1160.el7.x86_64
0x144
12
SyS_recvfrom
vmlinux 0xffffffff9ae3661e
Build id: 3.10.0-1160.el7.x86_64
0xe
13
system_call_fastpath
vmlinux 0xffffffff9af93f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 23s! [connection:17020]
Modules linked in: nls_utf8 isofs tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper vmw_balloon joydev cryptd pcspkr sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt nfit fb_sys_fops ttm libnvdimm sd_mod drm crc_t10dif ahci crct10dif_generic libahci ata_piix libata crct10dif_pclmul crct10dif_common drm_panel_orientation_quirks crc32c_intel vmxnet3 serio_raw vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 15 PID: 17020 Comm: connection Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff93b04eb88000 ti: ffff93c8e4460000 task.ti: ffff93c8e4460000
RIP: 0010:[<ffffffff9a9174cd>]  [<ffffffff9a9174cd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff93c8e4463ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff93b04ff86370
RBP: ffff93c8e4463ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 00000000000005c7 R11: 0000000000000000 R12: 000000019ae3d101
R13: ffff93b000000001 R14: 0000000000000001 R15: 0000000000000000
FS:  00007f352e8fd700(0000) GS:ffff93ce7f5c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3831b85000 CR3: 000000017b7de000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9af7b37b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9af89ad3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff9ae385e0>] lock_sock_nested+0x20/0x50
[<ffffffff9aeae293>] tcp_recvmsg+0x73/0xb30
[<ffffffff9ab0be65>] ? sock_has_perm+0x75/0x90
[<ffffffff9aedd730>] inet_recvmsg+0x80/0xb0
[<ffffffff9ae344f5>] sock_recvmsg+0xc5/0x100
[<ffffffff9ae343a6>] ? sock_sendmsg+0xb6/0xf0
[<ffffffff9ae34668>] SYSC_recvfrom+0xe8/0x160
[<ffffffff9aa64134>] ? poll_select_copy_remaining+0x144/0x180
[<ffffffff9ae3661e>] SyS_recvfrom+0xe/0x10
[<ffffffff9af93f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #783782
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff929174cd
Build id: 3.10.0-1160.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffff92f7b37b
Build id: 3.10.0-1160.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff92f89ad3
Build id: 3.10.0-1160.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff92e385e0
Build id: 3.10.0-1160.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff92eae293
Build id: 3.10.0-1160.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffff92b0be65
Build id: 3.10.0-1160.el7.x86_64
0x75
7
inet_recvmsg
vmlinux 0xffffffff92edd730
Build id: 3.10.0-1160.el7.x86_64
0x80
8
sock_recvmsg
vmlinux 0xffffffff92e344f5
Build id: 3.10.0-1160.el7.x86_64
0xc5
9
SYSC_recvfrom
vmlinux 0xffffffff92e34668
Build id: 3.10.0-1160.el7.x86_64
0xe8
10
? ktime_get_ts64
vmlinux 0xffffffff92906362
Build id: 3.10.0-1160.el7.x86_64
0x52
11
SyS_recvfrom
vmlinux 0xffffffff92e3661e
Build id: 3.10.0-1160.el7.x86_64
0xe
12
system_call_fastpath
vmlinux 0xffffffff92f93f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lorax-composer:84043]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr joydev vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm nfit serio_raw drm ahci libahci vmxnet3 libnvdimm ata_piix vmw_pvscsi libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 84043 Comm: lorax-composer Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8dc4f32ca100 ti: ffff8dc2df6e8000 task.ti: ffff8dc2df6e8000
RIP: 0010:[<ffffffff929174cd>]  [<ffffffff929174cd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff8dc2df6ebba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8dc4b6118ff0
RBP: ffff8dc2df6ebba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000077 R11: 0000000000000000 R12: ffff8dc2df6ebf18
R13: ffff8dc4f32ca100 R14: ffff8dc2df6ebfd8 R15: 00007fffab98c9f0
FS:  00007f283338c740(0000) GS:ffff8dc52d6c0000(0000) knlGS:0000000000000000
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/1:3:82039]
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558cc45906b0 CR3: 00000001cc9d6000 CR4: 00000000003607e0
Call Trace:
[<ffffffff92f7b37b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff92f89ad3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff92e385e0>] lock_sock_nested+0x20/0x50
[<ffffffff92eae293>] tcp_recvmsg+0x73/0xb30
[<ffffffff92b0be65>] ? sock_has_perm+0x75/0x90
[<ffffffff92edd730>] inet_recvmsg+0x80/0xb0
[<ffffffff92e344f5>] sock_recvmsg+0xc5/0x100
[<ffffffff92e34668>] SYSC_recvfrom+0xe8/0x160
[<ffffffff92906362>] ? ktime_get_ts64+0x52/0xf0
[<ffffffff92e3661e>] SyS_recvfrom+0xe/0x10
[<ffffffff92f93f92>] system_call_fastpath+0x25/0x2a
Complete report #780275
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff9db174cd
Build id: 3.10.0-1160.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffff9e17b37b
Build id: 3.10.0-1160.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff9e189ad3
Build id: 3.10.0-1160.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff9e0385e0
Build id: 3.10.0-1160.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff9e0ae293
Build id: 3.10.0-1160.el7.x86_64
0x73
6
inet_recvmsg
vmlinux 0xffffffff9e0dd730
Build id: 3.10.0-1160.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffff9e0344f5
Build id: 3.10.0-1160.el7.x86_64
0xc5
8
? futex_wait
vmlinux 0xffffffff9db12fb5
Build id: 3.10.0-1160.el7.x86_64
0x195
9
SYSC_recvfrom
vmlinux 0xffffffff9e034668
Build id: 3.10.0-1160.el7.x86_64
0xe8
10
? ktime_get_ts64
vmlinux 0xffffffff9db06362
Build id: 3.10.0-1160.el7.x86_64
0x52
11
SyS_recvfrom
vmlinux 0xffffffff9e03661e
Build id: 3.10.0-1160.el7.x86_64
0xe
12
system_call_fastpath
vmlinux 0xffffffff9e193f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [java:39557]
Modules linked in: tcp_diag inet_diag veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter ext4 mbcache jbd2 overlay(T) vmw_vsock_vmci_transport vsock sunrpc kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx ata_piix nfit drm_kms_helper libata syscopyarea sysfillrect
libnvdimm sysimgblt fb_sys_fops ttm mptspi e1000 crct10dif_pclmul crct10dif_common scsi_transport_spi mptscsih mptbase crc32c_intel drm serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 39557 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8f8847a14200 ti: ffff8f895bb44000 task.ti: ffff8f895bb44000
RIP: 0010:[<ffffffff9db174cd>]  [<ffffffff9db174cd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff8f895bb47ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8f88408ae370
RBP: ffff8f895bb47ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000040 R11: 0000000000000246 R12: ffff8f895bb47f18
R13: ffff8f8847a14200 R14: ffff8f895bb47fd8 R15: 00007f526d0a16f8
FS:  00007f526d0b2700(0000) GS:ffff8f89756c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000017dff0000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9e17b37b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff9e189ad3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff9e0385e0>] lock_sock_nested+0x20/0x50
[<ffffffff9e0ae293>] tcp_recvmsg+0x73/0xb30
[<ffffffff9e0dd730>] inet_recvmsg+0x80/0xb0
[<ffffffff9e0344f5>] sock_recvmsg+0xc5/0x100
[<ffffffff9db12fb5>] ? futex_wait+0x195/0x280
[<ffffffff9e034668>] SYSC_recvfrom+0xe8/0x160
[<ffffffff9db06362>] ? ktime_get_ts64+0x52/0xf0
[<ffffffff9e03661e>] SyS_recvfrom+0xe/0x10
[<ffffffff9e193f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #769037
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffb73174cd
Build id: 3.10.0-1160.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffb797b37b
Build id: 3.10.0-1160.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffb7989ad3
Build id: 3.10.0-1160.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffffb78385e0
Build id: 3.10.0-1160.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffb78ae293
Build id: 3.10.0-1160.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffffb750be65
Build id: 3.10.0-1160.el7.x86_64
0x75
7
? try_to_wake_up
vmlinux 0xffffffffb72da830
Build id: 3.10.0-1160.el7.x86_64
0x190
8
inet_recvmsg
vmlinux 0xffffffffb78dd730
Build id: 3.10.0-1160.el7.x86_64
0x80
9
sock_recvmsg
vmlinux 0xffffffffb78344f5
Build id: 3.10.0-1160.el7.x86_64
0xc5
10
? __wake_up_common_lock
vmlinux 0xffffffffb72d3101
Build id: 3.10.0-1160.el7.x86_64
0x91
11
? task_rq_unlock
vmlinux 0xffffffffb72d2ba0
Build id: 3.10.0-1160.el7.x86_64
0x20
12
SYSC_recvfrom
vmlinux 0xffffffffb7834668
Build id: 3.10.0-1160.el7.x86_64
0xe8
13
? ktime_get_ts64
vmlinux 0xffffffffb7306362
Build id: 3.10.0-1160.el7.x86_64
0x52
14
SyS_recvfrom
vmlinux 0xffffffffb783661e
Build id: 3.10.0-1160.el7.x86_64
0xe
15
system_call_fastpath
vmlinux 0xffffffffb7993f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [curl:61942]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) tcp_lp nls_utf8 isofs rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev btusb
btrtl btbcm btintel vmw_balloon aesni_intel bluetooth lrw gf128mul glue_helper ablk_helper cryptd snd_timer joydev pcspkr snd rfkill sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix nfit mptspi libnvdimm scsi_transport_spi serio_raw mptscsih drm e1000 libata mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 61942 Comm: curl Tainted: G        W    L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8daad1e74200 ti: ffff8daa8d388000 task.ti: ffff8daa8d388000
RIP: 0010:[<ffffffffb73174cd>]  [<ffffffffb73174cd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff8daa8d38bba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8daaed9de530
RBP: ffff8daa8d38bba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000968 R11: 0000000000000000 R12: ffff8daa8d38bf18
R13: ffff8daad1e74200 R14: ffff8daa8d38bfd8 R15: 00007ffcabeb1850
FS:  00007fdc9ffeb840(0000) GS:ffff8daaf2c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5ffefdbff0 CR3: 000000006d138000 CR4: 00000000003607f0
Call Trace:
[<ffffffffb797b37b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffb7989ad3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffffb78385e0>] lock_sock_nested+0x20/0x50
[<ffffffffb78ae293>] tcp_recvmsg+0x73/0xb30
[<ffffffffb750be65>] ? sock_has_perm+0x75/0x90
[<ffffffffb72da830>] ? try_to_wake_up+0x190/0x390
[<ffffffffb78dd730>] inet_recvmsg+0x80/0xb0
[<ffffffffb78344f5>] sock_recvmsg+0xc5/0x100
[<ffffffffb72d3101>] ? __wake_up_common_lock+0x91/0xc0
[<ffffffffb72d2ba0>] ? task_rq_unlock+0x20/0x20
[<ffffffffb7834668>] SYSC_recvfrom+0xe8/0x160
[<ffffffffb7306362>] ? ktime_get_ts64+0x52/0xf0
[<ffffffffb783661e>] SyS_recvfrom+0xe/0x10
[<ffffffffb7993f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #636102
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff8b11598d
Build id: 3.10.0-1062.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffff8b773365
Build id: 3.10.0-1062.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff8b781b13
Build id: 3.10.0-1062.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff8b6315e0
Build id: 3.10.0-1062.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff8b6a7783
Build id: 3.10.0-1062.el7.x86_64
0x73
6
inet_recvmsg
vmlinux 0xffffffff8b6d6ae0
Build id: 3.10.0-1062.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffff8b62d4f5
Build id: 3.10.0-1062.el7.x86_64
0xc5
8
? sock_sendmsg
vmlinux 0xffffffff8b62d3a6
Build id: 3.10.0-1062.el7.x86_64
0xb6
9
SYSC_recvfrom
vmlinux 0xffffffff8b62d668
Build id: 3.10.0-1062.el7.x86_64
0xe8
10
? __schedule
vmlinux 0xffffffff8b77ec28
Build id: 3.10.0-1062.el7.x86_64
0x448
11
SyS_recvfrom
vmlinux 0xffffffff8b62f61e
Build id: 3.10.0-1062.el7.x86_64
0xe
12
system_call_fastpath
vmlinux 0xffffffff8b78bede
Build id: 3.10.0-1062.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [reporter-urepor:57148]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper pcspkr vmw_vmci ablk_helper cryptd sg i2c_piix4 joydev parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom vmwgfx crc_t10dif crct10dif_generic ata_generic pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit libnvdimm libata mptspi e1000 scsi_transport_spi mptscsih mptbase crct10dif_pclmul
crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 57148 Comm: reporter-urepor Kdump: loaded Tainted: G             L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff8d4742235230 ti: ffff8d4755134000 task.ti: ffff8d4755134000
RIP: 0010:[<ffffffff8b11598d>]  [<ffffffff8b11598d>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff8d4755137ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff8d3f7531b640 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8d3f7531b6b0
RBP: ffff8d4755137ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: ffff8d3f7531b640
R13: 000000009e98b2a4 R14: 000000000000001a R15: ffffffff8b631e68
FS:  00007f2234f4b8c0(0000) GS:ffff8d475d700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005652000d9bc8 CR3: 00000007d8288000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8b773365>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff8b781b13>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff8b6315e0>] lock_sock_nested+0x20/0x50
[<ffffffff8b6a7783>] tcp_recvmsg+0x73/0xb30
[<ffffffff8b6d6ae0>] inet_recvmsg+0x80/0xb0
[<ffffffff8b62d4f5>] sock_recvmsg+0xc5/0x100
[<ffffffff8b62d3a6>] ? sock_sendmsg+0xb6/0xf0
[<ffffffff8b62d668>] SYSC_recvfrom+0xe8/0x160
[<ffffffff8b77ec28>] ? __schedule+0x448/0x9c0
[<ffffffff8b62f61e>] SyS_recvfrom+0xe/0x10
[<ffffffff8b78bede>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #582062
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffa8b174cd
Build id: 3.10.0-1160.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffa917b37b
Build id: 3.10.0-1160.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffa9189ad3
Build id: 3.10.0-1160.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffffa90385e0
Build id: 3.10.0-1160.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffa90ae293
Build id: 3.10.0-1160.el7.x86_64
0x73
6
inet_recvmsg
vmlinux 0xffffffffa90dd730
Build id: 3.10.0-1160.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffffa90344f5
Build id: 3.10.0-1160.el7.x86_64
0xc5
8
SYSC_recvfrom
vmlinux 0xffffffffa9034668
Build id: 3.10.0-1160.el7.x86_64
0xe8
9
? __schedule
vmlinux 0xffffffffa9186c8f
Build id: 3.10.0-1160.el7.x86_64
0x3af
10
SyS_recvfrom
vmlinux 0xffffffffa903661e
Build id: 3.10.0-1160.el7.x86_64
0xe
11
system_call_fastpath
vmlinux 0xffffffffa9193f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [reporter-urepor:70129]
Modules linked in: tcp_lp veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi ppdev snd_ac97_codec ac97_bus aesni_intel
snd_seq lrw gf128mul snd_seq_device glue_helper ablk_helper vmw_balloon snd_pcm cryptd snd_timer joydev pcspkr snd soundcore i2c_piix4 sg vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ata_generic pata_acpi crct10dif_pclmul mptspi fb_sys_fops crct10dif_common ttm scsi_transport_spi drm ata_piix e1000 mptscsih crc32c_intel serio_raw mptbase nfit libnvdimm libata drm_panel_orientation_quirks fuse
CPU: 1 PID: 70129 Comm: reporter-urepor Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8895818d4200 ti: ffff8896a4934000 task.ti: ffff8896a4934000
RIP: 0010:[<ffffffffa8b174cd>]  [<ffffffffa8b174cd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff8896a4937ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff8896a4937e40 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff88958cfccdf0
RBP: ffff8896a4937ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: ffff8896a4937b40
R13: ffff8895818d4200 R14: ffff8896a4937fd8 R15: 00007fff67748b90
FS:  00007fbd95a3c940(0000) GS:ffff8897af240000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcaa15fb000 CR3: 000000000a84c000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa917b37b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa9189ad3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffffa90385e0>] lock_sock_nested+0x20/0x50
[<ffffffffa90ae293>] tcp_recvmsg+0x73/0xb30
[<ffffffffa90dd730>] inet_recvmsg+0x80/0xb0
[<ffffffffa90344f5>] sock_recvmsg+0xc5/0x100
[<ffffffffa9034668>] SYSC_recvfrom+0xe8/0x160
[<ffffffffa9186c8f>] ? __schedule+0x3af/0x860
[<ffffffffa903661e>] SyS_recvfrom+0xe/0x10
[<ffffffffa9193f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #579198
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff89717a4d
Build id: 3.10.0-1160.76.1.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffff89d80d83
Build id: 3.10.0-1160.76.1.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff89d8ed63
Build id: 3.10.0-1160.76.1.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff89c3d5d0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff89cb3aa3
Build id: 3.10.0-1160.76.1.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffff8990c685
Build id: 3.10.0-1160.76.1.el7.x86_64
0x75
7
inet_recvmsg
vmlinux 0xffffffff89ce2ff0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x80
8
sock_recvmsg
vmlinux 0xffffffff89c394e5
Build id: 3.10.0-1160.76.1.el7.x86_64
0xc5
9
SYSC_recvfrom
vmlinux 0xffffffff89c39658
Build id: 3.10.0-1160.76.1.el7.x86_64
0xe8
10
? wake_up_state
vmlinux 0xffffffff896db2e0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x20
11
SyS_recvfrom
vmlinux 0xffffffff89c3b60e
Build id: 3.10.0-1160.76.1.el7.x86_64
0xe
12
system_call_fastpath
vmlinux 0xffffffff89d99f92
Build id: 3.10.0-1160.76.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [sunloginclient:1074]
Modules linked in: veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ppdev vmw_balloon sg joydev snd_seq_midi snd_seq_midi_event parport_pc parport snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif vmwgfx crct10dif_generic sr_mod cdrom
drm_kms_helper ata_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm pata_acpi crct10dif_pclmul crct10dif_common drm serio_raw crc32c_intel mptspi ata_piix scsi_transport_spi mptscsih nfit libata libnvdimm mptbase e1000 drm_panel_orientation_quirks fuse
CPU: 3 PID: 1074 Comm: sunloginclient Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8f3eda383180 ti: ffff8f3edbf60000 task.ti: ffff8f3edbf60000
RIP: 0010:[<ffffffff89717a4d>]  [<ffffffff89717a4d>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff8f3edbf63ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffffffff89d856d6 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8f3ec7950070
RBP: ffff8f3edbf63ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000020 R11: 0000000000000000 R12: 0000000000000202
R13: 0000000000000018 R14: ffff8f3edbf63b18 R15: 0000000000000202
FS:  00007f015ede4700(0000) GS:ffff8f3edd6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efd28097000 CR3: 000000080d45a000 CR4: 0000000000340fe0
Call Trace:
[<ffffffff89d80d83>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff89d8ed63>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff89c3d5d0>] lock_sock_nested+0x20/0x50
[<ffffffff89cb3aa3>] tcp_recvmsg+0x73/0xb30
[<ffffffff8990c685>] ? sock_has_perm+0x75/0x90
[<ffffffff89ce2ff0>] inet_recvmsg+0x80/0xb0
[<ffffffff89c394e5>] sock_recvmsg+0xc5/0x100
[<ffffffff89c39658>] SYSC_recvfrom+0xe8/0x160
[<ffffffff896db2e0>] ? wake_up_state+0x20/0x20
[<ffffffff89c3b60e>] SyS_recvfrom+0xe/0x10
[<ffffffff89d99f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #541508
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffbb1174cd
Build id: 3.10.0-1160.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffbb77b37b
Build id: 3.10.0-1160.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffbb789ad3
Build id: 3.10.0-1160.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffffbb6385e0
Build id: 3.10.0-1160.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffbb6ae293
Build id: 3.10.0-1160.el7.x86_64
0x73
6
inet_recvmsg
vmlinux 0xffffffffbb6dd730
Build id: 3.10.0-1160.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffffbb6344f5
Build id: 3.10.0-1160.el7.x86_64
0xc5
8
? do_sync_write
vmlinux 0xffffffffbb24d063
Build id: 3.10.0-1160.el7.x86_64
0x93
9
SYSC_recvfrom
vmlinux 0xffffffffbb634668
Build id: 3.10.0-1160.el7.x86_64
0xe8
10
? ktime_get_ts64
vmlinux 0xffffffffbb106362
Build id: 3.10.0-1160.el7.x86_64
0x52
11
SyS_recvfrom
vmlinux 0xffffffffbb63661e
Build id: 3.10.0-1160.el7.x86_64
0xe
12
system_call_fastpath
vmlinux 0xffffffffbb793f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [urlgrabber-ext-:33553]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM tun devlink xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack bonding ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel vmw_balloon cryptd pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c
sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit crct10dif_pclmul crct10dif_common libnvdimm ata_piix crc32c_intel serio_raw libata mptspi scsi_transport_spi e1000 mptscsih drm_panel_orientation_quirks mptbase dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 33553 Comm: urlgrabber-ext- Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff952b730c1080 ti: ffff952a40b2c000 task.ti: ffff952a40b2c000
RIP: 0010:[<ffffffffbb1174cd>]  [<ffffffffbb1174cd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff952a40b2fba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff952a77396cf0
RBP: ffff952a40b2fba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: ffff952a40b2ff18
R13: ffff952b730c1080 R14: ffff952a40b2ffd8 R15: 00007ffdad8ee440
FS:  00007f5295d07740(0000) GS:ffff952bf56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001b76028 CR3: 00000001b1f54000 CR4: 00000000003607e0
Call Trace:
[<ffffffffbb77b37b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffbb789ad3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffffbb6385e0>] lock_sock_nested+0x20/0x50
[<ffffffffbb6ae293>] tcp_recvmsg+0x73/0xb30
[<ffffffffbb6dd730>] inet_recvmsg+0x80/0xb0
[<ffffffffbb6344f5>] sock_recvmsg+0xc5/0x100
[<ffffffffbb24d063>] ? do_sync_write+0x93/0xe0
[<ffffffffbb634668>] SYSC_recvfrom+0xe8/0x160
[<ffffffffbb106362>] ? ktime_get_ts64+0x52/0xf0
[<ffffffffbb63661e>] SyS_recvfrom+0xe/0x10
[<ffffffffbb793f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #435904
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffb9d174cd
Build id: 3.10.0-1160.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffba37b37b
Build id: 3.10.0-1160.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffba389ad3
Build id: 3.10.0-1160.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffffba2385e0
Build id: 3.10.0-1160.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffba2ae293
Build id: 3.10.0-1160.el7.x86_64
0x73
6
? sock_has_perm
vmlinux 0xffffffffb9f0be65
Build id: 3.10.0-1160.el7.x86_64
0x75
7
inet_recvmsg
vmlinux 0xffffffffba2dd730
Build id: 3.10.0-1160.el7.x86_64
0x80
8
sock_recvmsg
vmlinux 0xffffffffba2344f5
Build id: 3.10.0-1160.el7.x86_64
0xc5
9
? do_sync_write
vmlinux 0xffffffffb9e4d063
Build id: 3.10.0-1160.el7.x86_64
0x93
10
SYSC_recvfrom
vmlinux 0xffffffffba234668
Build id: 3.10.0-1160.el7.x86_64
0xe8
11
? ktime_get_ts64
vmlinux 0xffffffffb9d06362
Build id: 3.10.0-1160.el7.x86_64
0x52
12
SyS_recvfrom
vmlinux 0xffffffffba23661e
Build id: 3.10.0-1160.el7.x86_64
0xe
13
system_call_fastpath
vmlinux 0xffffffffba393f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 35s! [urlgrabber-ext-:3695]
Modules linked in: tcp_lp rfcomm xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ext4 mbcache jbd2 snd_ens1371 snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel aesni_intel lrw gf128mul ac97_bus glue_helper snd_seq snd_seq_device snd_pcm ablk_helper cryptd btusb btrtl btbcm btintel bluetooth snd_timer snd sg ppdev pcspkr soundcore rfkill joydev vmw_balloon parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm libata mptspi scsi_transport_spi serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks fuse
CPU: 0 PID: 3695 Comm: urlgrabber-ext- Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff974ebb791080 ti: ffff974ed525c000 task.ti: ffff974ed525c000
RIP: 0010:[<ffffffffb9d174cd>]  [<ffffffffb9d174cd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff974ed525fba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff974ef291aef0
RBP: ffff974ed525fba0 R08: 0000000000000040 R09: 0000000000000000
R10: 00000000000005b2 R11: 0000000000000000 R12: ffff974ed525ff18
R13: ffff974ebb791080 R14: ffff974ed525ffd8 R15: 00007ffe5eade990
FS:  00007f420dbed740(0000) GS:ffff974efb600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f973a41d000 CR3: 000000002f72e000 CR4: 00000000007607f0
PKRU: 55555554
Call Trace:
[<ffffffffba37b37b>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffba389ad3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffffba2385e0>] lock_sock_nested+0x20/0x50
[<ffffffffba2ae293>] tcp_recvmsg+0x73/0xb30
[<ffffffffb9f0be65>] ? sock_has_perm+0x75/0x90
[<ffffffffba2dd730>] inet_recvmsg+0x80/0xb0
[<ffffffffba2344f5>] sock_recvmsg+0xc5/0x100
[<ffffffffb9e4d063>] ? do_sync_write+0x93/0xe0
[<ffffffffba234668>] SYSC_recvfrom+0xe8/0x160
[<ffffffffb9d06362>] ? ktime_get_ts64+0x52/0xf0
[<ffffffffba23661e>] SyS_recvfrom+0xe/0x10
[<ffffffffba393f92>] system_call_fastpath+0x25/0x2a
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #372493
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff810fdeaf
Build id: 3.10.0-693.21.1.el7.x86_64
0x1f
2
queued_spin_lock_slowpath
vmlinux 0xffffffff816a8ff4
Build id: 3.10.0-693.21.1.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff816b6893
Build id: 3.10.0-693.21.1.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff815760e0
Build id: 3.10.0-693.21.1.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff815e2311
Build id: 3.10.0-693.21.1.el7.x86_64
0x91
6
inet_recvmsg
vmlinux 0xffffffff81610b60
Build id: 3.10.0-693.21.1.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffff81573a75
Build id: 3.10.0-693.21.1.el7.x86_64
0xc5
8
? xfs_iunlock
xfs 0xffffffffc029c13c
Build id: 3.10.0-693.21.1.el7.x86_64
0x11c
9
? xfs_filemap_fault
xfs 0xffffffffc028e54e
Build id: 3.10.0-693.21.1.el7.x86_64
0x6e
10
SYSC_recvfrom
vmlinux 0xffffffff81573be8
Build id: 3.10.0-693.21.1.el7.x86_64
0xe8
11
? ktime_get_ts64
vmlinux 0xffffffff810ecec2
Build id: 3.10.0-693.21.1.el7.x86_64
0x52
12
SyS_recvfrom
vmlinux 0xffffffff815743ce
Build id: 3.10.0-693.21.1.el7.x86_64
0xe
13
system_call_fastpath
vmlinux 0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [reporter-urepor:62242]
Modules linked in: nls_utf8 isofs fuse vmw_vsock_vmci_transport vsock ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter snd_seq_midi snd_seq_midi_event sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper snd_ens1371 snd_rawmidi snd_ac97_codec ablk_helper cryptd ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore ppdev vmw_balloon joydev sg pcspkr i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata e1000 mptspi scsi_transport_spi serio_raw mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 62242 Comm: reporter-urepor Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff880426614f10 ti: ffff8801d33ec000 task.ti: ffff8801d33ec000
RIP: 0010:[<ffffffff810fdeaf>]  [<ffffffff810fdeaf>] native_queued_spin_lock_slowpath+0x1f/0x200
RSP: 0018:ffff8801d33efba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 00000000000f423f RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff88041d7b2730
RBP: ffff8801d33efba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 01ff88041d7b26c0
R13: ffff880426614f10 R14: ffff8801d33effd8 R15: 0000000000f52880
FS:  00007fd15a2ec880(0000) GS:ffff88042d680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6ced76f968 CR3: 00000003336c6000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff816a8ff4>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff816b6893>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff815760e0>] lock_sock_nested+0x20/0x50
[<ffffffff815e2311>] tcp_recvmsg+0x91/0xb50
[<ffffffff81610b60>] inet_recvmsg+0x80/0xb0
[<ffffffff81573a75>] sock_recvmsg+0xc5/0x100
[<ffffffffc029c13c>] ? xfs_iunlock+0x11c/0x130 [xfs]
[<ffffffffc028e54e>] ? xfs_filemap_fault+0x6e/0xf0 [xfs]
[<ffffffff81573be8>] SYSC_recvfrom+0xe8/0x160
[<ffffffff810ecec2>] ? ktime_get_ts64+0x52/0xf0
[<ffffffff815743ce>] SyS_recvfrom+0xe/0x10
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 8b 17 <85> d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 1f 80 
Complete report #245645
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffffb7d1214d
Build id: 3.10.0-957.12.1.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffb835d1ab
Build id: 3.10.0-957.12.1.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffb836b6a3
Build id: 3.10.0-957.12.1.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffffb821e5d0
Build id: 3.10.0-957.12.1.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffb8292573
Build id: 3.10.0-957.12.1.el7.x86_64
0x73
6
inet_recvmsg
vmlinux 0xffffffffb82c1220
Build id: 3.10.0-957.12.1.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffffb821a4f5
Build id: 3.10.0-957.12.1.el7.x86_64
0xc5
8
? do_read_fault.isra.61
vmlinux 0xffffffffb7de4cc9
Build id: 3.10.0-957.12.1.el7.x86_64
0x139
9
SYSC_recvfrom
vmlinux 0xffffffffb821a668
Build id: 3.10.0-957.12.1.el7.x86_64
0xe8
10
? ktime_get_ts64
vmlinux 0xffffffffb7d01092
Build id: 3.10.0-957.12.1.el7.x86_64
0x52
11
SyS_recvfrom
vmlinux 0xffffffffb821c61e
Build id: 3.10.0-957.12.1.el7.x86_64
0xe
12
system_call_fastpath
vmlinux 0xffffffffb8375d9b
Build id: 3.10.0-957.12.1.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [reporter-urepor:75694]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink binfmt_misc ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi snd_ens1371 kvm_intel kvm snd_rawmidi snd_ac97_codec irqbypass ac97_bus ppdev crc32_pclmul snd_seq ghash_clmulni_intel snd_seq_device snd_pcm aesni_intel
lrw gf128mul snd_timer glue_helper vmw_balloon ablk_helper cryptd snd soundcore pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi ata_piix serio_raw scsi_transport_spi e1000 drm_panel_orientation_quirks mptscsih mptbase libata nfit libnvdimm
CPU: 1 PID: 75694 Comm: reporter-urepor Kdump: loaded Tainted: G             L ------------   3.10.0-957.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9fca1ec01040 ti: ffff9fca1f860000 task.ti: ffff9fca1f860000
RIP: 0010:[<ffffffffb7d1214d>]  [<ffffffffb7d1214d>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff9fca1f863ba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9fcbe5e4b6b0
RBP: ffff9fca1f863ba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: ffff9fca1f863f18
R13: ffff9fca1ec01040 R14: ffff9fca1f863fd8 R15: 000000000117f260
FS:  00007f1fb5e6c8c0(0000) GS:ffff9fcbf5640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0369d42310 CR3: 00000001e4f3a000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb835d1ab>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffb836b6a3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffffb821e5d0>] lock_sock_nested+0x20/0x50
[<ffffffffb8292573>] tcp_recvmsg+0x73/0xb30
[<ffffffffb82c1220>] inet_recvmsg+0x80/0xb0
[<ffffffffb821a4f5>] sock_recvmsg+0xc5/0x100
[<ffffffffb7de4cc9>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffffb821a668>] SYSC_recvfrom+0xe8/0x160
[<ffffffffb7d01092>] ? ktime_get_ts64+0x52/0xf0
[<ffffffffb821c61e>] SyS_recvfrom+0xe/0x10
[<ffffffffb8375d9b>] system_call_fastpath+0x22/0x27
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #229625
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff9fb086fd
Build id: 3.10.0-862.el7.x86_64
0x1d
2
queued_spin_lock_slowpath
vmlinux 0xffffffffa010842a
Build id: 3.10.0-862.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffffa01156c3
Build id: 3.10.0-862.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff9ffd0170
Build id: 3.10.0-862.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffffa003ffe3
Build id: 3.10.0-862.el7.x86_64
0x73
6
inet_recvmsg
vmlinux 0xffffffffa006e750
Build id: 3.10.0-862.el7.x86_64
0x80
7
sock_recvmsg
vmlinux 0xffffffff9ffcc4e5
Build id: 3.10.0-862.el7.x86_64
0xc5
8
? do_read_fault.isra.60
vmlinux 0xffffffff9fbc0c31
Build id: 3.10.0-862.el7.x86_64
0x121
9
SYSC_recvfrom
vmlinux 0xffffffff9ffcc658
Build id: 3.10.0-862.el7.x86_64
0xe8
10
? ktime_get_ts64
vmlinux 0xffffffff9faf76e2
Build id: 3.10.0-862.el7.x86_64
0x52
11
SyS_recvfrom
vmlinux 0xffffffff9ffce45e
Build id: 3.10.0-862.el7.x86_64
0xe
12
system_call_fastpath
vmlinux 0xffffffffa011f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [reporter-urepor:61705]
Modules linked in: esn_cfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul
syscopyarea crct10dif_common sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 61705 Comm: reporter-urepor Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
task: ffff887093bdaf70 ti: ffff8870613e8000 task.ti: ffff8870613e8000
RIP: 0010:[<ffffffff9fb086fd>]  [<ffffffff9fb086fd>] native_queued_spin_lock_slowpath+0x1d/0x200
RSP: 0018:ffff8870613ebba0  EFLAGS: 00000293
RAX: 0000000000000001 RBX: 00000000000f423f RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8870b943f4b0
RBP: ffff8870613ebba0 R08: 0000000000000040 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 01ffffff9ffd0928
R13: ffff887093bdaf70 R14: ffff8870613ebfd8 R15: 000000000232a380
FS:  00007f35c6cde8c0(0000) GS:ffff8870bc6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8a170b8000 CR3: 0000000021792000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa010842a>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa01156c3>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff9ffd0170>] lock_sock_nested+0x20/0x50
[<ffffffffa003ffe3>] tcp_recvmsg+0x73/0xb30
[<ffffffffa006e750>] inet_recvmsg+0x80/0xb0
[<ffffffff9ffcc4e5>] sock_recvmsg+0xc5/0x100
[<ffffffff9fbc0c31>] ? do_read_fault.isra.60+0x121/0x1a0
[<ffffffff9ffcc658>] SYSC_recvfrom+0xe8/0x160
[<ffffffff9faf76e2>] ? ktime_get_ts64+0x52/0xf0
[<ffffffff9ffce45e>] SyS_recvfrom+0xe/0x10
[<ffffffffa011f7d5>] system_call_fastpath+0x1c/0x21
Code: 47 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 b9 01 00 00 00 8b 17 85 d2 74 0d 83 fa 03 74 08 f3 90 <8b> 17 85 d2 75 f3 89 d0 f0 0f b1 0f 39 c2 75 e3 5d 66 90 c3 0f 
Complete report #29884
Frame # Function Binary Source or offset Line
1
native_queued_spin_lock_slowpath
vmlinux 0xffffffff810fa232
Build id: 3.10.0-693.el7.x86_64
0x22
2
queued_spin_lock_slowpath
vmlinux 0xffffffff8169e5ff
Build id: 3.10.0-693.el7.x86_64
0xb
3
_raw_spin_lock_bh
vmlinux 0xffffffff816abb93
Build id: 3.10.0-693.el7.x86_64
0x33
4
lock_sock_nested
vmlinux 0xffffffff8156cfd0
Build id: 3.10.0-693.el7.x86_64
0x20
5
tcp_recvmsg
vmlinux 0xffffffff815d8681
Build id: 3.10.0-693.el7.x86_64
0x91
6
inet_recvmsg
vmlinux 0xffffffff81606a3a
Build id: 3.10.0-693.el7.x86_64
0x7a
7
sock_recvmsg
vmlinux 0xffffffff8156a88f
Build id: 3.10.0-693.el7.x86_64
0xbf
8
SYSC_recvfrom
vmlinux 0xffffffff8156aa08
Build id: 3.10.0-693.el7.x86_64
0xe8
9
? ktime_get_ts64
vmlinux 0xffffffff810e939c
Build id: 3.10.0-693.el7.x86_64
0x4c
10
SyS_recvfrom
vmlinux 0xffffffff8156b2fe
Build id: 3.10.0-693.el7.x86_64
0xe
11
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [java:23254]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus coretemp iosf_mbi crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel cryptd snd_pcm snd_timer snd ppdev soundcore vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom e1000 ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul ata_piix drm mptspi crct10dif_common crc32c_intel scsi_transport_spi libata mptscsih serio_raw i2c_core mptbase floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 23254 Comm: java Not tainted 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff88003489cf10 ti: ffff880026828000 task.ti: ffff880026828000
RIP: 0010:[<ffffffff810fa232>]  [<ffffffff810fa232>] native_queued_spin_lock_slowpath+0x22/0x1e0
RSP: 0018:ffff88002682bbd0  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 000000000098967f RCX: ffff88002682bfd8
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8800b3636ef0
RBP: ffff88002682bbd0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 01ff8800b3636e80
R13: ffff88003489cf10 R14: ffff88002682bfd8 R15: 00007f209fbea310
FS:  00007f209fbfb700(0000) GS:ffff88013a680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa000d61068 CR3: 0000000008d24000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88002682bbe0 ffffffff8169e5ff ffff88002682bbf0 ffffffff816abb93
ffff88002682bc10 ffffffff8156cfd0 0000000000000000 ffff880094a35280
ffff88002682bcb0 ffffffff815d8681 ffff88002682bfd8 000000003489cf10
Call Trace:
[<ffffffff8169e5ff>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffff816abb93>] _raw_spin_lock_bh+0x33/0x40
[<ffffffff8156cfd0>] lock_sock_nested+0x20/0x50
[<ffffffff815d8681>] tcp_recvmsg+0x91/0xb50
[<ffffffff81606a3a>] inet_recvmsg+0x7a/0xa0
[<ffffffff8156a88f>] sock_recvmsg+0xbf/0x100
[<ffffffff8156aa08>] SYSC_recvfrom+0xe8/0x160
[<ffffffff810e939c>] ? ktime_get_ts64+0x4c/0xf0
[<ffffffff8156b2fe>] SyS_recvfrom+0xe/0x10
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 66 90 ba 01 00 00 00 8b 07 85 c0 75 0a f0 0f b1 17 85 c0 75 f2 5d c3 f3 90 <eb> ec 0f 1f 40 00 81 fe 00 01 00 00 66 90 0f 84 5c 01 00 00 41