Info

Permalink
Function
_raw_spin_unlock_irqrestore
First occurrence
2020-09-23
Last occurrence
2020-10-05
State
NEW
Components
kernel
Quality
-55

Statistics

Operating system Count
CentOS 7 54
Architecture Count
x86_64 54
Related packages Count
kernel 54
 0:3.10.0-1127.19.1.el7 8
 0:3.10.0-1160.45.1.el7 6
 0:3.10.0-1160.25.1.el7 4
 0:3.10.0-957.el7 4
 0:3.10.0-1127.8.2.el7 3
 0:3.10.0-1062.9.1.el7 3
 0:3.10.0-1160.31.1.el7 3
 0:3.10.0-1160.21.1.el7 2
 0:3.10.0-1160.el7 2
 0:3.10.0-693.21.1.el7 2
 0:3.10.0-862.el7 2
 0:3.10.0-1127.el7 2
 0:3.10.0-1160.11.1.el7 2
 0:3.10.0-1062.el7 2
 0:3.10.0-1062.18.1.el7 1
 0:3.10.0-1160.42.2.el7 1
 0:3.10.0-1160.53.1.el7 1
 0:3.10.0-1160.6.1.el7 1
 0:3.10.0-862.3.2.el7 1
 0:3.10.0-1062.1.2.el7 1
 0:3.10.0-957.21.3.el7 1
 0:3.10.0-1160.36.2.el7 1
 0:3.10.0-1160.59.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #948305
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff965883a5
Build id: 3.10.0-1127.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc0181d55
Build id: 3.10.0-1127.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc0180db1
Build id: 3.10.0-1127.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc0180e3d
Build id: 3.10.0-1127.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc065ba75
Build id: 3.10.0-1127.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc065bba7
Build id: 3.10.0-1127.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc065d17c
Build id: 3.10.0-1127.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc065dd95
Build id: 3.10.0-1127.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc065b360
Build id: 3.10.0-1127.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc064d099
Build id: 3.10.0-1127.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffff95ec7780
Build id: 3.10.0-1127.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffff964333a6
Build id: 3.10.0-1127.el7.x86_64 		0xb6
13
SYSC_sendto
vmlinux 0xffffffff96433ad1
Build id: 3.10.0-1127.el7.x86_64 		0x121
14
? SYSC_statfs
vmlinux 0xffffffff9608448a
Build id: 3.10.0-1127.el7.x86_64 		0x3a
15
? system_call_after_swapgs
vmlinux 0xffffffff96592e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
16
? system_call_after_swapgs
vmlinux 0xffffffff96592e09
Build id: 3.10.0-1127.el7.x86_64 		0x96
17
? system_call_after_swapgs
vmlinux 0xffffffff96592e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffff96592e09
Build id: 3.10.0-1127.el7.x86_64 		0x96
19
? system_call_after_swapgs
vmlinux 0xffffffff96592e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
20
? system_call_after_swapgs
vmlinux 0xffffffff96592e09
Build id: 3.10.0-1127.el7.x86_64 		0x96
21
? system_call_after_swapgs
vmlinux 0xffffffff96592e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
22
SyS_sendto
vmlinux 0xffffffff964355ee
Build id: 3.10.0-1127.el7.x86_64 		0xe
23
system_call_fastpath
vmlinux 0xffffffff96592ed2
Build id: 3.10.0-1127.el7.x86_64 		0x25
24
? system_call_after_swapgs
vmlinux 0xffffffff96592e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 27s! [vmtoolsd:737]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter cmac arc4 md4 nls_utf8 cifs ccm dns_resolver overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi kvm_intel kvm snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq ppdev irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd snd_seq_device snd_pcm joydev pcspkr snd_timer snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables
xfs libcrc32c sr_mod sd_mod cdrom vmwgfx crc_t10dif crct10dif_generic ata_generic pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix nfit libnvdimm mptspi libata scsi_transport_spi serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 737 Comm: vmtoolsd Kdump: loaded Tainted: G             L ------------ T 3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff954d34648000 ti: ffff954d34644000 task.ti: ffff954d34644000
RIP: 0010:[<ffffffff965883a5>]  [<ffffffff965883a5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff954d34647af8  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff954dbffd8000 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff954d34647af8 R08: 0000000000000001 R09: 000000004c759a63
R10: 0000000000000000 R11: 0000000000000000 R12: ffff954dbffd8018
R13: 0000000000000246 R14: ffff954dbffd8000 R15: 0000000000000002
FS:  00007f7e7ce2b880(0000) GS:ffff954db9600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7e7ce49000 CR3: 00000000b46c6000 CR4: 00000000000607f0
Call Trace:
[<ffffffffc0181d55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc0180db1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc0180e3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc065ba75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc065bba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc065d17c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc065dd95>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc065b360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc064d099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffff95ec7780>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff964333a6>] sock_sendmsg+0xb6/0xf0
[<ffffffff96433ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffff9608448a>] ? SYSC_statfs+0x3a/0x60
[<ffffffff96592e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff96592e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff96592e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff96592e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff96592e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff96592e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff96592e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff964355ee>] SyS_sendto+0xe/0x10
[<ffffffff96592ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff96592e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #1179436
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffab18ea05
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc0692d55
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc0691db1
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc0691e3d
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc07ffa85
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc07ffbb7
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc080118c
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0801da5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc07ff360
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc07f6099
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffffaaac6f50
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffffab0393a6
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xb6
13
? call_rcu_sched
vmlinux 0xffffffffaab59a9d
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x1d
14
SYSC_sendto
vmlinux 0xffffffffab039ad1
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x121
15
? putname
vmlinux 0xffffffffaac6034d
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x3d
16
? system_call_after_swapgs
vmlinux 0xffffffffab199ed5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffab199ec9
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x96
18
? system_call_after_swapgs
vmlinux 0xffffffffab199ed5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa2
19
? system_call_after_swapgs
vmlinux 0xffffffffab199ec9
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x96
20
? system_call_after_swapgs
vmlinux 0xffffffffab199ed5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa2
21
? system_call_after_swapgs
vmlinux 0xffffffffab199ec9
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x96
22
? system_call_after_swapgs
vmlinux 0xffffffffab199ed5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa2
23
SyS_sendto
vmlinux 0xffffffffab03b5ee
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xe
24
system_call_fastpath
vmlinux 0xffffffffab199f92
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x25
25
? system_call_after_swapgs
vmlinux 0xffffffffab199ed5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 39s! [vmtoolsd:701]
Modules linked in: devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit libnvdimm ata_piix crct10dif_pclmul crct10dif_common mptspi crc32c_intel libata scsi_transport_spi serio_raw mptscsih e1000 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 701 Comm: vmtoolsd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8e18f25a5280 ti: ffff8e18f410c000 task.ti: ffff8e18f410c000
RIP: 0010:[<ffffffffab18ea05>]  [<ffffffffab18ea05>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8e18f410faf8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffffffffffffffff RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8e18f410faf8 R08: 0000000000000001 R09: 000000006973bef6
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000010
R13: 0000000000000000 R14: ffff8e18fffd6000 R15: 0000000000000000
FS:  00007f516400c780(0000) GS:ffff8e18ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5164024000 CR3: 00000000350c6000 CR4: 00000000000407f0
Call Trace:
[<ffffffffc0692d55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc0691db1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc0691e3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc07ffa85>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc07ffbb7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc080118c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0801da5>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc07ff360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc07f6099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffffaaac6f50>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffab0393a6>] sock_sendmsg+0xb6/0xf0
[<ffffffffaab59a9d>] ? call_rcu_sched+0x1d/0x20
[<ffffffffab039ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffffaac6034d>] ? putname+0x3d/0x60
[<ffffffffab199ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffab199ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffab199ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffab199ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffab199ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffab199ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffab199ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffab03b5ee>] SyS_sendto+0xe/0x10
[<ffffffffab199f92>] system_call_fastpath+0x25/0x2a
[<ffffffffab199ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #298432
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa03893e5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc0359d55
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc0358db1
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc0358e3d
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc055ba75
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc055bba7
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc055d17c
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc055dd95
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc055b360
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc054d099
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffff9fcc6f60
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffffa02343a6
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb6
13
? free_pages_and_swap_cache
vmlinux 0xffffffff9fe0978d
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xad
14
SYSC_sendto
vmlinux 0xffffffffa0234ad1
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x121
15
? do_munmap
vmlinux 0xffffffff9fdfb937
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x327
16
? system_call_after_swapgs
vmlinux 0xffffffffa0393ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffa0393ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
18
? system_call_after_swapgs
vmlinux 0xffffffffa0393ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
19
? system_call_after_swapgs
vmlinux 0xffffffffa0393ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
20
? system_call_after_swapgs
vmlinux 0xffffffffa0393ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
21
? system_call_after_swapgs
vmlinux 0xffffffffa0393ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
22
? system_call_after_swapgs
vmlinux 0xffffffffa0393ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
23
SyS_sendto
vmlinux 0xffffffffa02365ee
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xe
24
system_call_fastpath
vmlinux 0xffffffffa0393f92
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x25
25
? system_call_after_swapgs
vmlinux 0xffffffffa0393ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [vmtoolsd:853]
Modules linked in: iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper ahci syscopyarea sysfillrect libahci sysimgblt fb_sys_fops ata_piix ttm nfit libnvdimm drm serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 853 Comm: vmtoolsd Kdump: loaded Tainted: G        W      ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
task: ffff8d7835872100 ti: ffff8d795fbec000 task.ti: ffff8d795fbec000
RIP: 0010:[<ffffffffa03893e5>]  [<ffffffffa03893e5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8d795fbefaf8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: 00000000000001c1 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8d795fbefaf8 R08: 0000000000000001 R09: 00000000012d289d
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8d7c3ffcf008
R13: ffff8d7c2bc7b4e6 R14: ffff8d7c3ffcf000 R15: 0000000000000000
FS:  00007fc19206e780(0000) GS:ffff8d7c2d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc19207b000 CR3: 00000000364f2000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc0359d55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc0358db1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc0358e3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc055ba75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc055bba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc055d17c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc055dd95>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc055b360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc054d099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffff9fcc6f60>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffa02343a6>] sock_sendmsg+0xb6/0xf0
[<ffffffff9fe0978d>] ? free_pages_and_swap_cache+0xad/0xd0
[<ffffffffa0234ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffff9fdfb937>] ? do_munmap+0x327/0x480
[<ffffffffa0393ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa0393ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa0393ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa0393ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa0393ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa0393ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa0393ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa02365ee>] SyS_sendto+0xe/0x10
[<ffffffffa0393f92>] system_call_fastpath+0x25/0x2a
[<ffffffffa0393ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #297699
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffba7883c5
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc04cbd55
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc04cadb1
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc04cae3d
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0567a75
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0567ba7
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc056917c
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0569d95
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0567360
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc0644099
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffffba0c7780
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffffba6333a6
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xb6
13
SYSC_sendto
vmlinux 0xffffffffba633ad1
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x121
14
? system_call_after_swapgs
vmlinux 0xffffffffba792e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
15
? system_call_after_swapgs
vmlinux 0xffffffffba792e09
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x96
16
? system_call_after_swapgs
vmlinux 0xffffffffba792e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffba792e09
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x96
18
? system_call_after_swapgs
vmlinux 0xffffffffba792e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
19
? system_call_after_swapgs
vmlinux 0xffffffffba792e09
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x96
20
? system_call_after_swapgs
vmlinux 0xffffffffba792e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
21
SyS_sendto
vmlinux 0xffffffffba6355ee
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xe
22
system_call_fastpath
vmlinux 0xffffffffba792ed2
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x25
23
? system_call_after_swapgs
vmlinux 0xffffffffba792e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 434s! [pool:1011]
Modules linked in: nfnetlink_queue nfnetlink_log nfnetlink bluetooth rfkill vmsecmod(OE) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr joydev sg vmw_balloon vmxnet3 parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit libata libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel mptspi scsi_transport_spi mptscsih serio_raw mptbase drm_panel_orientation_quirks
CPU: 1 PID: 1011 Comm: pool Kdump: loaded Tainted: G           OE  ------------   3.10.0-1127.8.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8d4fb3c1d230 ti: ffff8d4e39e34000 task.ti: ffff8d4e39e34000
RIP: 0010:[<ffffffffba7883c5>]  [<ffffffffba7883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8d4e39e37af8  EFLAGS: 00000282
RAX: 0000000000000038 RBX: ffffffffc0567b6a RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff8d4e39e37af8 R08: 0000000000000001 R09: 00000000707ec3c3
R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000400 R14: 0000000000000400 R15: 0000000000000400
FS:  00007fc17ca78700(0000) GS:ffff8d4fb5640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f13b32373ce CR3: 0000000233884000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc04cbd55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc04cadb1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc04cae3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc0567a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc0567ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc056917c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0569d95>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc0567360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0644099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffffba0c7780>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffba6333a6>] sock_sendmsg+0xb6/0xf0
[<ffffffffba633ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffffba792e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffba792e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffba792e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffba792e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffba792e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffba792e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffba792e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffba6355ee>] SyS_sendto+0xe/0x10
[<ffffffffba792ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffba792e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #282924
Frame # Function Binary Source or offset Line
1
vmci_datagram_send
vmw_vmci 0xffffffffc0445e3d
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x1d
2
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc063ba85
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa5
3
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc063bbb7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa7
4
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc063d18c
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2c
5
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc063dda5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x65
6
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc063b360
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x20
7
vsock_stream_sendmsg
vsock 0xffffffffc0632099
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2e9
8
? wake_up_atomic_t
vmlinux 0xffffffff874c6f50
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x30
9
sock_sendmsg
vmlinux 0xffffffff87a363a6
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xb6
10
? free_pages_and_swap_cache
vmlinux 0xffffffff87609f8d
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xad
11
SYSC_sendto
vmlinux 0xffffffff87a36ad1
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x121
12
? do_munmap
vmlinux 0xffffffff875fc0c7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x327
13
? system_call_after_swapgs
vmlinux 0xffffffff87b95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2
14
? system_call_after_swapgs
vmlinux 0xffffffff87b95ec9
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x96
15
? system_call_after_swapgs
vmlinux 0xffffffff87b95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2
16
? system_call_after_swapgs
vmlinux 0xffffffff87b95ec9
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x96
17
? system_call_after_swapgs
vmlinux 0xffffffff87b95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffff87b95ec9
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x96
19
? system_call_after_swapgs
vmlinux 0xffffffff87b95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2
20
SyS_sendto
vmlinux 0xffffffff87a385ee
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xe
21
system_call_fastpath
vmlinux 0xffffffff87b95f92
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x25
22
? system_call_after_swapgs
vmlinux 0xffffffff87b95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 40s! [kworker/1:1:64301]
[<ffffffffc0445e3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc063ba85>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc063bbb7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc063d18c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc063dda5>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc063b360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0632099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffff874c6f50>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff87a363a6>] sock_sendmsg+0xb6/0xf0
[<ffffffff87609f8d>] ? free_pages_and_swap_cache+0xad/0xd0
[<ffffffff87a36ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffff875fc0c7>] ? do_munmap+0x327/0x480
[<ffffffff87b95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff87b95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff87b95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff87b95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff87b95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff87b95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff87b95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff87a385ee>] SyS_sendto+0xe/0x10
[<ffffffff87b95f92>] system_call_fastpath+0x25/0x2a
[<ffffffff87b95ed5>] ? system_call_after_swapgs+0xa2/0x13a
Complete report #199035
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8bd8b495
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc04fdd55
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc04fcdb1
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc04fce3d
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0565a75
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0565ba7
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc056717c
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0567d95
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0565360
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc055c099
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffff8b6c6e90
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffff8bc363a6
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xb6
13
? free_pages_and_swap_cache
vmlinux 0xffffffff8b80972d
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xad
14
SYSC_sendto
vmlinux 0xffffffff8bc36ad1
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x121
15
? do_munmap
vmlinux 0xffffffff8b7fb877
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x327
16
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ed5
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ec9
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x96
18
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ed5
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xa2
19
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ec9
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x96
20
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ed5
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xa2
21
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ec9
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x96
22
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ed5
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xa2
23
SyS_sendto
vmlinux 0xffffffff8bc385ee
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xe
24
system_call_fastpath
vmlinux 0xffffffff8bd95f92
Build id: 3.10.0-1160.21.1.el7.x86_64 		0x25
25
? system_call_after_swapgs
vmlinux 0xffffffff8bd95ed5
Build id: 3.10.0-1160.21.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 24s! [vmtoolsd:1050]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink xt_nat xt_mark ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock nbd(OE) sb_edac coretemp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd parport_pc pcspkr joydev i2c_piix4 parport sg vmw_vmci nfsd nfs_acl lockd grace auth_rpcgss sunrpc
ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix nfit serio_raw mptspi libata libnvdimm e1000 scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy rapidrecovery_vss(POE) dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 6 PID: 1050 Comm: vmtoolsd Kdump: loaded Tainted: P           OE  ------------   3.10.0-1160.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9a2f749e9080 ti: ffff9a2f72f58000 task.ti: ffff9a2f72f58000
RIP: 0010:[<ffffffff8bd8b495>]  [<ffffffff8bd8b495>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9a2f72f5baf8  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8b91acc5 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff9a2f72f5baf8 R08: 0000000000000001 R09: 00000000661eb519
R10: 0000000000000000 R11: 0000000000000000 R12: ffff9a2f71ae2000
R13: ffff9a2f72f5bb00 R14: 000000000000068d R15: ffff9a2f71ae2000
FS:  00007f325d06a780(0000) GS:ffff9a2f7fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f325d080000 CR3: 0000000034776000 CR4: 00000000000407e0
Call Trace:
[<ffffffffc04fdd55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc04fcdb1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc04fce3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc0565a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc0565ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc056717c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0567d95>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc0565360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc055c099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffff8b6c6e90>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff8bc363a6>] sock_sendmsg+0xb6/0xf0
[<ffffffff8b80972d>] ? free_pages_and_swap_cache+0xad/0xd0
[<ffffffff8bc36ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffff8b7fb877>] ? do_munmap+0x327/0x480
[<ffffffff8bd95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff8bd95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff8bd95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff8bd95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff8bd95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff8bd95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff8bd95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff8bc385ee>] SyS_sendto+0xe/0x10
[<ffffffff8bd95f92>] system_call_fastpath+0x25/0x2a
[<ffffffff8bd95ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #35931
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff82f893e5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc0584d55
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc0583db1
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc0583e3d
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0636a75
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0636ba7
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc063817c
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0638d95
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0636360
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc061a099
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffff828c6f60
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffff82e343a6
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb6
13
? xfs_file_buffered_aio_write
xfs 0xffffffffc04c5928
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x298
14
? call_rcu_sched
vmlinux 0xffffffff8295985d
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x1d
15
SYSC_sendto
vmlinux 0xffffffff82e34ad1
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x121
16
? __sb_end_write
vmlinux 0xffffffff82a50bf1
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x31
17
? vfs_write
vmlinux 0xffffffff82a4ddf2
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x182
18
? system_call_after_swapgs
vmlinux 0xffffffff82f93ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
19
? system_call_after_swapgs
vmlinux 0xffffffff82f93ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
20
? system_call_after_swapgs
vmlinux 0xffffffff82f93ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
21
? system_call_after_swapgs
vmlinux 0xffffffff82f93ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
22
? system_call_after_swapgs
vmlinux 0xffffffff82f93ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
23
? system_call_after_swapgs
vmlinux 0xffffffff82f93ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
24
? system_call_after_swapgs
vmlinux 0xffffffff82f93ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
25
SyS_sendto
vmlinux 0xffffffff82e365ee
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xe
26
system_call_fastpath
vmlinux 0xffffffff82f93f92
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x25
27
? system_call_after_swapgs
vmlinux 0xffffffff82f93ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [vmtoolsd:945]
Modules linked in: tcp_diag udp_diag inet_diag nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm crc32_pclmul ghash_clmulni_intel aesni_intel snd_timer lrw gf128mul snd glue_helper ablk_helper ppdev sg vmw_balloon cryptd joydev vmw_vmci i2c_piix4 pcspkr soundcore
parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata mptspi serio_raw scsi_transport_spi nfit mptscsih e1000 mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 6 PID: 945 Comm: vmtoolsd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff96c5722a0000 ti: ffff96c56b698000 task.ti: ffff96c56b698000
RIP: 0010:[<ffffffff82f893e5>]  [<ffffffff82f893e5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff96c56b69baf8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff96c56b69bac0 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff96c56b69baf8 R08: 0000000000000001 R09: 000000005189896a
R10: 0000000000000000 R11: 0000000000000000 R12: ffff96c2553c6f08
R13: ffff96c5722a0000 R14: 0000000000000192 R15: ffffffff82a88b7f
FS:  00007f8bee88e780(0000) GS:ffff96c56f580000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8bee899000 CR3: 0000000036418000 CR4: 00000000000007e0
Call Trace:
[<ffffffffc0584d55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc0583db1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc0583e3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc0636a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc0636ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc063817c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0638d95>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc0636360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc061a099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffff828c6f60>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff82e343a6>] sock_sendmsg+0xb6/0xf0
[<ffffffffc04c5928>] ? xfs_file_buffered_aio_write+0x298/0x2c0 [xfs]
[<ffffffff8295985d>] ? call_rcu_sched+0x1d/0x20
[<ffffffff82e34ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffff82a50bf1>] ? __sb_end_write+0x31/0x70
[<ffffffff82a4ddf2>] ? vfs_write+0x182/0x1f0
[<ffffffff82f93ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff82f93ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff82f93ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff82f93ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff82f93ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff82f93ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff82f93ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff82e365ee>] SyS_sendto+0xe/0x10
[<ffffffff82f93f92>] system_call_fastpath+0x25/0x2a
[<ffffffff82f93ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #23934
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff89d6a0f5
Build id: 3.10.0-957.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc036fd15
Build id: 3.10.0-957.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc036ed91
Build id: 3.10.0-957.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc036ee1d
Build id: 3.10.0-957.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0718a75
Build id: 3.10.0-957.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0718ba7
Build id: 3.10.0-957.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc071a1dc
Build id: 3.10.0-957.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc071adf5
Build id: 3.10.0-957.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0718360
Build id: 3.10.0-957.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc06b0fd9
Build id: 3.10.0-957.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffff896c2d00
Build id: 3.10.0-957.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffff89c193a6
Build id: 3.10.0-957.el7.x86_64 		0xb6
13
SYSC_sendto
vmlinux 0xffffffff89c19ad1
Build id: 3.10.0-957.el7.x86_64 		0x121
14
? system_call_after_swapgs
vmlinux 0xffffffff89d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffff89d74d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
16
? system_call_after_swapgs
vmlinux 0xffffffff89d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae
17
? system_call_after_swapgs
vmlinux 0xffffffff89d74d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffff89d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae
19
? system_call_after_swapgs
vmlinux 0xffffffff89d74d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
20
? system_call_after_swapgs
vmlinux 0xffffffff89d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae
21
SyS_sendto
vmlinux 0xffffffff89c1b5ee
Build id: 3.10.0-957.el7.x86_64 		0xe
22
system_call_fastpath
vmlinux 0xffffffff89d74ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
23
? system_call_after_swapgs
vmlinux 0xffffffff89d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [vmtoolsd:6552]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel cryptd vmw_balloon snd_seq_midi snd_seq_midi_event joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg parport_pc parport vmw_vmci
i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx ata_generic pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi libata scsi_transport_spi e1000 nfit mptscsih mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 6552 Comm: vmtoolsd Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8c1df63d5140 ti: ffff8c1dfa02c000 task.ti: ffff8c1dfa02c000
RIP: 0010:[<ffffffff89d6a0f5>]  [<ffffffff89d6a0f5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8c1dfa02faf8  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff8c1df63d57d8 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8c1dfa02faf8 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8c1dfc600000
R13: ffffffff8962a621 R14: ffff8c1dfa02faa8 R15: ffff8c1dfa02fa70
FS:  00007f5c80854880(0000) GS:ffff8c1dfc600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007faee8b4f000 CR3: 0000000038704000 CR4: 00000000000607f0
Call Trace:
[<ffffffffc036fd15>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc036ed91>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc036ee1d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc0718a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc0718ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc071a1dc>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc071adf5>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc0718360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc06b0fd9>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffff896c2d00>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff89c193a6>] sock_sendmsg+0xb6/0xf0
[<ffffffff89c19ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffff89d74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff89d74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff89d74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff89d74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff89d74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff89d74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff89d74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff89c1b5ee>] SyS_sendto+0xe/0x10
[<ffffffff89d74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff89d74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #10423
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffb8983025
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc0761d35
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc0760d91
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc0760e1d
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc08a2a75
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc08a2ba7
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc08a41dc
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc08a4df5
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc08a2360
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc0892fd9
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffffb82c72e0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffffb882f3a6
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xb6
13
? do_last
vmlinux 0xffffffffb845939d
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x66d
14
? ___slab_alloc
vmlinux 0xffffffffb8424a99
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x209
15
SYSC_sendto
vmlinux 0xffffffffb882fad1
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x121
16
? mntput
vmlinux 0xffffffffb846de24
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x24
17
? __fput
vmlinux 0xffffffffb844cbc6
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x186
18
? system_call_after_swapgs
vmlinux 0xffffffffb898de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
19
? system_call_after_swapgs
vmlinux 0xffffffffb898de15
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa2
20
? system_call_after_swapgs
vmlinux 0xffffffffb898de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
21
? system_call_after_swapgs
vmlinux 0xffffffffb898de15
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa2
22
? system_call_after_swapgs
vmlinux 0xffffffffb898de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
23
? system_call_after_swapgs
vmlinux 0xffffffffb898de15
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa2
24
? system_call_after_swapgs
vmlinux 0xffffffffb898de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
25
SyS_sendto
vmlinux 0xffffffffb88315ee
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xe
26
system_call_fastpath
vmlinux 0xffffffffb898dede
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x25
27
? system_call_after_swapgs
vmlinux 0xffffffffb898de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 50s! [vmtoolsd:859]
Modules linked in: binfmt_misc fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables vmw_vsock_vmci_transport vsock iptable_filter ext4 sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel mbcache jbd2 ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix nfit libnvdimm libata crct10dif_pclmul
crct10dif_common mptspi crc32c_intel serio_raw scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 859 Comm: vmtoolsd Kdump: loaded Not tainted 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8b597998d230 ti: ffff8b59799e4000 task.ti: ffff8b59799e4000
RIP: 0010:[<ffffffffb8983025>]  [<ffffffffb8983025>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff8b59799e7af8  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff8b59799e7b10 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff8b59799e7af8 R08: 0000000000000001 R09: 0000000020c0b4ed
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8b59185ff0a0
R13: ffffffffb858c282 R14: ffff8b59799e7a70 R15: 000000d000000002
FS:  00007ffb9e646880(0000) GS:ffff8b59ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffb9e664000 CR3: 00000000b9af8000 CR4: 00000000000407f0
Call Trace:
[<ffffffffc0761d35>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc0760d91>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc0760e1d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc08a2a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc08a2ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc08a41dc>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc08a4df5>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc08a2360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0892fd9>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffffb82c72e0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffb882f3a6>] sock_sendmsg+0xb6/0xf0
[<ffffffffb845939d>] ? do_last+0x66d/0x1290
[<ffffffffb8424a99>] ? ___slab_alloc+0x209/0x4f0
[<ffffffffb882fad1>] SYSC_sendto+0x121/0x1c0
[<ffffffffb846de24>] ? mntput+0x24/0x40
[<ffffffffb844cbc6>] ? __fput+0x186/0x260
[<ffffffffb898de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb898de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffb898de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb898de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffb898de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb898de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffb898de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffb88315ee>] SyS_sendto+0xe/0x10
[<ffffffffb898dede>] system_call_fastpath+0x25/0x2a
[<ffffffffb898de21>] ? system_call_after_swapgs+0xae/0x146
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #10350
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffa41883a5
Build id: 3.10.0-1127.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc0500d55
Build id: 3.10.0-1127.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc04ffdb1
Build id: 3.10.0-1127.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc04ffe3d
Build id: 3.10.0-1127.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0669a75
Build id: 3.10.0-1127.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0669ba7
Build id: 3.10.0-1127.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc066b17c
Build id: 3.10.0-1127.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc066bd95
Build id: 3.10.0-1127.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0669360
Build id: 3.10.0-1127.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc065b099
Build id: 3.10.0-1127.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffffa3ac7780
Build id: 3.10.0-1127.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffffa40333a6
Build id: 3.10.0-1127.el7.x86_64 		0xb6
13
? free_pages_and_swap_cache
vmlinux 0xffffffffa3c0967d
Build id: 3.10.0-1127.el7.x86_64 		0xad
14
SYSC_sendto
vmlinux 0xffffffffa4033ad1
Build id: 3.10.0-1127.el7.x86_64 		0x121
15
? mntput
vmlinux 0xffffffffa3c71054
Build id: 3.10.0-1127.el7.x86_64 		0x24
16
? __fput
vmlinux 0xffffffffa3c4f4e5
Build id: 3.10.0-1127.el7.x86_64 		0x195
17
? system_call_after_swapgs
vmlinux 0xffffffffa4192e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffffa4192e09
Build id: 3.10.0-1127.el7.x86_64 		0x96
19
? system_call_after_swapgs
vmlinux 0xffffffffa4192e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
20
? system_call_after_swapgs
vmlinux 0xffffffffa4192e09
Build id: 3.10.0-1127.el7.x86_64 		0x96
21
? system_call_after_swapgs
vmlinux 0xffffffffa4192e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
22
? system_call_after_swapgs
vmlinux 0xffffffffa4192e09
Build id: 3.10.0-1127.el7.x86_64 		0x96
23
? system_call_after_swapgs
vmlinux 0xffffffffa4192e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2
24
SyS_sendto
vmlinux 0xffffffffa40355ee
Build id: 3.10.0-1127.el7.x86_64 		0xe
25
system_call_fastpath
vmlinux 0xffffffffa4192ed2
Build id: 3.10.0-1127.el7.x86_64 		0x25
26
? system_call_after_swapgs
vmlinux 0xffffffffa4192e15
Build id: 3.10.0-1127.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 25s! [vmtoolsd:803]
Modules linked in: iptable_filter vmw_vsock_vmci_transport vsock iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus iosf_mbi crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel ppdev snd_pcm cryptd vmw_balloon snd_timer pcspkr snd soundcore joydev sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi mptscsih crc32c_intel nfit drm ata_piix e1000 serio_raw libata mptbase libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod
CPU: 1 PID: 803 Comm: vmtoolsd Kdump: loaded Not tainted 3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9ee53422b150 ti: ffff9ee5335d0000 task.ti: ffff9ee5335d0000
RIP: 0010:[<ffffffffa41883a5>]  [<ffffffffa41883a5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff9ee5335d3af8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff9ee5000001c1 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff9ee5335d3af8 R08: 0000000000000001 R09: 00000000d1108bb9
R10: 0000000000000000 R11: 0000000000000000 R12: ffff9ee53ffdb018
R13: ffffffffa3d8fb32 R14: ffff9ee53ffdb000 R15: 0000000000000000
FS:  00007f26fb220880(0000) GS:ffff9ee535640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f26f6aa8268 CR3: 00000000ba5a6000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc0500d55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc04ffdb1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc04ffe3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc0669a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc0669ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc066b17c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc066bd95>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc0669360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc065b099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffffa3ac7780>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffa40333a6>] sock_sendmsg+0xb6/0xf0
[<ffffffffa3c0967d>] ? free_pages_and_swap_cache+0xad/0xd0
[<ffffffffa4033ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffffa3c71054>] ? mntput+0x24/0x40
[<ffffffffa3c4f4e5>] ? __fput+0x195/0x230
[<ffffffffa4192e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa4192e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa4192e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa4192e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa4192e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa4192e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa4192e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa40355ee>] SyS_sendto+0xe/0x10
[<ffffffffa4192ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffa4192e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 0f 1f 40 00 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 c6 07 00 0f 1f 40 00 48 89 f7 57 9d <0f> 1f 44 00 00 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 48
Complete report #9725
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffaad81825
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc063bd35
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc063ad91
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc063ae1d
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc05eda75
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc05edba7
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc05ef1dc
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc05efdf5
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc05ed360
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc05ddfd9
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffffaa6c61c0
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffffaac2d3a6
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xb6
13
? free_pages_and_swap_cache
vmlinux 0xffffffffaa8058ad
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xad
14
SYSC_sendto
vmlinux 0xffffffffaac2dad1
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x121
15
? do_munmap
vmlinux 0xffffffffaa7f7bc7
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x327
16
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce21
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xae
17
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce15
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce21
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xae
19
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce15
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xa2
20
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce21
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xae
21
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce15
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xa2
22
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce21
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xae
23
SyS_sendto
vmlinux 0xffffffffaac2f5ee
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xe
24
system_call_fastpath
vmlinux 0xffffffffaad8cede
Build id: 3.10.0-1062.1.2.el7.x86_64 		0x25
25
? system_call_after_swapgs
vmlinux 0xffffffffaad8ce21
Build id: 3.10.0-1062.1.2.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 39s! [vmtoolsd:1312]
Modules linked in: fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon sb_edac crc32_pclmul ghash_clmulni_intel nfit aesni_intel lrw libnvdimm gf128mul glue_helper ablk_helper cryptd joydev pcspkr pcnet32 sg mii vmw_vmci parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel ahci serio_raw libahci ata_piix vmwgfx libata dm_mirror dm_region_hash dm_log dm_mod vboxvideo(OE) drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm drm drm_panel_orientation_quirks
CPU: 3 PID: 1312 Comm: vmtoolsd Tainted: G           OEL ------------   3.10.0-1062.1.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff99daa828c1c0 ti: ffff99daa963c000 task.ti: ffff99daa963c000
RIP: 0010:[<ffffffffaad81825>]  [<ffffffffaad81825>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff99daa963faf8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff99daa963fa98 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff99daa963faf8 R08: 0000000000000001 R09: 00000000b545e7a4
R10: 0000000000000000 R11: 0000000000000000 R12: ffff99daa828c1c0
R13: 00000000000280da R14: 0000000000000002 R15: 0000000047245a59
FS:  00007f9c66870880(0000) GS:ffff99dabfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9c66883000 CR3: 00000003292fc000 CR4: 00000000000607e0
Call Trace:
[<ffffffffc063bd35>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc063ad91>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc063ae1d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc05eda75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc05edba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc05ef1dc>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc05efdf5>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc05ed360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc05ddfd9>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffffaa6c61c0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffaac2d3a6>] sock_sendmsg+0xb6/0xf0
[<ffffffffaa8058ad>] ? free_pages_and_swap_cache+0xad/0xd0
[<ffffffffaac2dad1>] SYSC_sendto+0x121/0x1c0
[<ffffffffaa7f7bc7>] ? do_munmap+0x327/0x480
[<ffffffffaad8ce21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaad8ce15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaad8ce21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaad8ce15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaad8ce21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaad8ce15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaad8ce21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaac2f5ee>] SyS_sendto+0xe/0x10
[<ffffffffaad8cede>] system_call_fastpath+0x25/0x2a
[<ffffffffaad8ce21>] ? system_call_after_swapgs+0xae/0x146
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #4628
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffff8e583025
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc07e1d35
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc07e0d91
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc07e0e1d
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc07f1a75
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc07f1ba7
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc07f31dc
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc07f3df5
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc07f1360
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc0864fd9
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffff8dec72e0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffff8e42f3a6
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xb6
13
? free_pages_and_swap_cache
vmlinux 0xffffffff8e00776d
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xad
14
SYSC_sendto
vmlinux 0xffffffff8e42fad1
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x121
15
? do_munmap
vmlinux 0xffffffff8dff9a87
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x327
16
? system_call_after_swapgs
vmlinux 0xffffffff8e58de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
17
? system_call_after_swapgs
vmlinux 0xffffffff8e58de15
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffff8e58de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
19
? system_call_after_swapgs
vmlinux 0xffffffff8e58de15
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa2
20
? system_call_after_swapgs
vmlinux 0xffffffff8e58de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
21
? system_call_after_swapgs
vmlinux 0xffffffff8e58de15
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa2
22
? system_call_after_swapgs
vmlinux 0xffffffff8e58de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae
23
SyS_sendto
vmlinux 0xffffffff8e4315ee
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xe
24
system_call_fastpath
vmlinux 0xffffffff8e58dede
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x25
25
? system_call_after_swapgs
vmlinux 0xffffffff8e58de21
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [vmtoolsd:22513]
Modules linked in: 8021q garp mrp stp llc vmw_vsock_vmci_transport vsock sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel joydev lrw gf128mul pcspkr glue_helper ablk_helper cryptd sg vmw_vmci parport_pc parport i2c_piix4 binfmt_misc auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel mptspi serio_raw vmxnet3 scsi_transport_spi ata_piix mptscsih mptbase libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 22513 Comm: vmtoolsd Kdump: loaded Not tainted 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9283b0f6e2a0 ti: ffff92822c8cc000 task.ti: ffff92822c8cc000
RIP: 0010:[<ffffffff8e583025>]  [<ffffffff8e583025>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff92822c8cfaf8  EFLAGS: 00000286
RAX: 0000000000000000 RBX: ffffffff8e116595 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000286 RDI: 0000000000000286
RBP: ffff92822c8cfaf8 R08: 0000000000000001 R09: 000000000f212b7d
R10: 0000000000000000 R11: 0000000000000000 R12: ffff928350961000
R13: ffff92822c8cfb00 R14: 00000000000009de R15: ffff928350961000
FS:  00007fc2e617f880(0000) GS:ffff9283bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc2e619c000 CR3: 000000022c774000 CR4: 00000000000607f0
Call Trace:
[<ffffffffc07e1d35>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc07e0d91>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc07e0e1d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc07f1a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc07f1ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc07f31dc>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc07f3df5>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc07f1360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0864fd9>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffff8dec72e0>] ? wake_up_atomic_t+0x30/0x30
[<ffffffff8e42f3a6>] sock_sendmsg+0xb6/0xf0
[<ffffffff8e00776d>] ? free_pages_and_swap_cache+0xad/0xd0
[<ffffffff8e42fad1>] SYSC_sendto+0x121/0x1c0
[<ffffffff8dff9a87>] ? do_munmap+0x327/0x480
[<ffffffff8e58de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8e58de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8e58de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8e58de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8e58de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8e58de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8e58de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8e4315ee>] SyS_sendto+0xe/0x10
[<ffffffff8e58dede>] system_call_fastpath+0x25/0x2a
[<ffffffff8e58de21>] ? system_call_after_swapgs+0xae/0x146
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48
Complete report #814
Frame # Function Binary Source or offset Line
1
_raw_spin_unlock_irqrestore
vmlinux 0xffffffffac9883c5
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x15
2
vmci_send_datagram
vmw_vmci 0xffffffffc0508d55
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x75
3
vmci_datagram_dispatch
vmw_vmci 0xffffffffc0507db1
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x71
4
vmci_datagram_send
vmw_vmci 0xffffffffc0507e3d
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x1d
5
__vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0581a75
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa5
6
vmci_transport_send_control_pkt
vmw_vsock_vmci_transport 0xffffffffc0581ba7
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa7
7
vmci_transport_send_wrote
vmw_vsock_vmci_transport 0xffffffffc058317c
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x2c
8
vmci_transport_notify_pkt_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0583d95
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x65
9
vmci_transport_notify_send_post_enqueue
vmw_vsock_vmci_transport 0xffffffffc0581360
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x20
10
vsock_stream_sendmsg
vsock 0xffffffffc0538099
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x2e9
11
? wake_up_atomic_t
vmlinux 0xffffffffac2c7780
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x30
12
sock_sendmsg
vmlinux 0xffffffffac8333a6
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xb6
13
? free_pages_and_swap_cache
vmlinux 0xffffffffac4096dd
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xad
14
SYSC_sendto
vmlinux 0xffffffffac833ad1
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x121
15
? __schedule
vmlinux 0xffffffffac985942
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x402
16
? system_call_after_swapgs
vmlinux 0xffffffffac992e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffac992e09
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x96
18
? system_call_after_swapgs
vmlinux 0xffffffffac992e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
19
? system_call_after_swapgs
vmlinux 0xffffffffac992e09
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x96
20
? system_call_after_swapgs
vmlinux 0xffffffffac992e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
21
? system_call_after_swapgs
vmlinux 0xffffffffac992e09
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x96
22
? system_call_after_swapgs
vmlinux 0xffffffffac992e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2
23
SyS_sendto
vmlinux 0xffffffffac8355ee
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xe
24
system_call_fastpath
vmlinux 0xffffffffac992ed2
Build id: 3.10.0-1127.8.2.el7.x86_64 		0x25
25
? system_call_after_swapgs
vmlinux 0xffffffffac992e15
Build id: 3.10.0-1127.8.2.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [vmtoolsd:763]
Modules linked in: fuse binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel joydev lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm sd_mod nfit crc_t10dif crct10dif_generic drm ata_piix libnvdimm libata crct10dif_pclmul crct10dif_common vmxnet3 crc32c_intel serio_raw vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 763 Comm: vmtoolsd Kdump: loaded Tainted: G             L ------------   3.10.0-1127.8.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff923c726641c0 ti: ffff923c76318000 task.ti: ffff923c76318000
RIP: 0010:[<ffffffffac9883c5>]  [<ffffffffac9883c5>] _raw_spin_unlock_irqrestore+0x15/0x20
RSP: 0018:ffff923c7631baf8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff923a47b83150 RCX: 0000000000000000
RDX: 000000000000109c RSI: 0000000000000282 RDI: 0000000000000282
RBP: ffff923c7631baf8 R08: 0000000000000001 R09: 00000000768be6ce
R10: 0000000000000000 R11: 0000000000000000 R12: ffff923a47b83954
R13: 00080000ac22b981 R14: 00000000000000fd R15: 0000000000000096
FS:  00007f01b4b70880(0000) GS:ffff923c7fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f29f461d5b0 CR3: 0000000236b6c000 CR4: 00000000000607f0
Call Trace:
[<ffffffffc0508d55>] vmci_send_datagram+0x75/0x90 [vmw_vmci]
[<ffffffffc0507db1>] vmci_datagram_dispatch+0x71/0xe0 [vmw_vmci]
[<ffffffffc0507e3d>] vmci_datagram_send+0x1d/0x30 [vmw_vmci]
[<ffffffffc0581a75>] __vmci_transport_send_control_pkt+0xa5/0x130 [vmw_vsock_vmci_transport]
[<ffffffffc0581ba7>] vmci_transport_send_control_pkt+0xa7/0xf0 [vmw_vsock_vmci_transport]
[<ffffffffc058317c>] vmci_transport_send_wrote+0x2c/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0583d95>] vmci_transport_notify_pkt_send_post_enqueue+0x65/0xa0 [vmw_vsock_vmci_transport]
[<ffffffffc0581360>] vmci_transport_notify_send_post_enqueue+0x20/0x30 [vmw_vsock_vmci_transport]
[<ffffffffc0538099>] vsock_stream_sendmsg+0x2e9/0x380 [vsock]
[<ffffffffac2c7780>] ? wake_up_atomic_t+0x30/0x30
[<ffffffffac8333a6>] sock_sendmsg+0xb6/0xf0
[<ffffffffac4096dd>] ? free_pages_and_swap_cache+0xad/0xd0
[<ffffffffac833ad1>] SYSC_sendto+0x121/0x1c0
[<ffffffffac985942>] ? __schedule+0x402/0x840
[<ffffffffac992e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffac992e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffac992e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffac992e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffac992e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffac992e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffac992e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffac8355ee>] SyS_sendto+0xe/0x10
[<ffffffffac992ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffac992e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 07 00 66 66 66 90 5d c3 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 c6 07 00 66 66 66 90 48 89 f7 57 9d <66> 66 90 66 90 5d c3 0f 1f 40 00 66 66 66 66 90 55 48 89 e5 48