Complete report #1399761
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
vmstat_start
|
vmlinux |
0xffffffff81bdf1fa
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0xaa
|
|
2
|
seq_read
|
vmlinux |
0xffffffff81c72882
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x1c2
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffff81cc2b80
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x40
|
|
4
|
vfs_read
|
vmlinux |
0xffffffff81c4a6df
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x9f
|
|
5
|
SyS_read
|
vmlinux |
0xffffffff81c4b59f
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x7f
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffff8218dede
Build id: 3.10.0-1062.12.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 51s! [vmtoolsd:738]
Modules linked in: vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr vmw_vmci i2c_piix4 nfit libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix drm crc32c_intel serio_raw libata mptspi e1000 scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 738 Comm: vmtoolsd Tainted: G L ------------ 3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff89aa7a249070 ti: ffff89aa7acbc000 task.ti: ffff89aa7acbc000
RIP: 0010:[<ffffffff81bdf1fa>] [<ffffffff81bdf1fa>] vmstat_start+0xaa/0xc0
RSP: 0018:ffff89aa7acbfe28 EFLAGS: 00010207
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff89aa7acbffd8
RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffffffff82647b68
RBP: ffff89aa7acbfe40 R08: ffffffff8275b2c0 R09: ffff89aa7d801500
R10: ffffffff81bdf186 R11: ffffffff81bddc4d R12: 00000007ff6caabf
R13: 0000000000000000 R14: 0000000000000000 R15: 00000007ff6caabf
FS: 00007ff6caaab880(0000) GS:ffff89aa7dd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff6caabf000 CR3: 000000003a3c0000 CR4: 00000000003407e0
Call Trace:
[<ffffffff81c72882>] seq_read+0x1c2/0x440
[<ffffffff81cc2b80>] proc_reg_read+0x40/0x80
[<ffffffff81c4a6df>] vfs_read+0x9f/0x170
[<ffffffff81c4b59f>] SyS_read+0x7f/0xf0
[<ffffffff8218dede>] system_call_fastpath+0x25/0x2a
Code: ed ad fe ff e8 68 04 ec ff 49 8d bd 40 01 00 00 e8 9c f4 ff ff e8 a7 04 ec ff 49 d1 ad 40 01 00 00 49 d1 ad 48 01 00 00 48 8b 13 <49> 8b 44 24 78 48 8d 04 d0 5b 41 5c 41 5d 5d c3 66 0f 1f 44 00
Complete report #947531
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
sum_vm_events
|
vmlinux |
0xffffffff99beaad0
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0xb0
|
|
2
|
vmstat_start
|
vmlinux |
0xffffffff99beb5b4
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x94
|
|
3
|
seq_read
|
vmlinux |
0xffffffff99c85887
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x1d7
|
|
4
|
proc_reg_read
|
vmlinux |
0xffffffff99cd7800
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x40
|
|
5
|
vfs_read
|
vmlinux |
0xffffffff99c5bb2f
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x9f
|
|
6
|
SyS_read
|
vmlinux |
0xffffffff99c5c9a5
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x55
|
|
7
|
system_call_fastpath
|
vmlinux |
0xffffffff9a1c539a
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 24s! [vmtoolsd:957]
Modules linked in: vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 nfit libnvdimm coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg i2c_piix4 parport_pc vmw_vmci parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi drm libata scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih vmxnet3 crc32c_intel serio_raw mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 14 PID: 957 Comm: vmtoolsd Kdump: loaded Tainted: G L ------------ 3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8f163a5eb180 ti: ffff8f163a12c000 task.ti: ffff8f163a12c000
RIP: 0010:[<ffffffff99beaad0>] [<ffffffff99beaad0>] sum_vm_events+0xb0/0x110
RSP: 0018:ffff8f163a12fe00 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000400 RCX: ffff8f153bc17e60
RDX: 000000000000ffff RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff8f163a12fe18 R08: ffffffff9a75e3a0 R09: ffff8f133b803500
R10: ffffffff99beb556 R11: 0000000000000293 R12: 0000000000000001
R13: 0000000000000001 R14: ffffffff9a1bfb05 R15: ffff8f163a12fd88
FS: 00007f7684d56780(0000) GS:ffff8f163fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7684d6b000 CR3: 000000023a6ea000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff99beb5b4>] vmstat_start+0x94/0xc0
[<ffffffff99c85887>] seq_read+0x1d7/0x460
[<ffffffff99cd7800>] proc_reg_read+0x40/0x80
[<ffffffff99c5bb2f>] vfs_read+0x9f/0x170
[<ffffffff99c5c9a5>] SyS_read+0x55/0xd0
[<ffffffff9a1c539a>] system_call_fastpath+0x25/0x2a
Code: 35 7a 4f b7 00 4c 89 ef 48 63 d2 e8 2b 94 1c 00 3b 05 69 4f b7 00 89 c6 7d 27 48 98 4c 89 e1 48 03 0c c5 00 19 75 9a 31 c0 66 90 <48> 8b 14 01 48 01 14 03 48 83 c0 08 48 3d 80 02 00 00 75 ec eb
Complete report #961001
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
ipv6_route_seq_show
|
vmlinux |
0xffffffff9ff2e301
Build id: 3.10.0-1127.el7.x86_64
|
0x21
|
|
2
|
seq_read
|
vmlinux |
0xffffffff9fa759a0
Build id: 3.10.0-1127.el7.x86_64
|
0x130
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffff9fac62b0
Build id: 3.10.0-1127.el7.x86_64
|
0x40
|
|
4
|
vfs_read
|
vmlinux |
0xffffffff9fa4cfbf
Build id: 3.10.0-1127.el7.x86_64
|
0x9f
|
|
5
|
SyS_read
|
vmlinux |
0xffffffff9fa4de2f
Build id: 3.10.0-1127.el7.x86_64
|
0x7f
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffff9ff92ed2
Build id: 3.10.0-1127.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#17 stuck for 21s! [vmtoolsd:1102]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev pcspkr sg parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci ata_piix nfit crct10dif_pclmul crct10dif_common libata crc32c_intel libnvdimm serio_raw
vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 17 PID: 1102 Comm: vmtoolsd Kdump: loaded Tainted: G L ------------ 3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9ae92ea262a0 ti: ffff9ae932690000 task.ti: ffff9ae932690000
RIP: 0010:[<ffffffff9ff2e301>] [<ffffffff9ff2e301>] ipv6_route_seq_show+0x21/0xe0
RSP: 0018:ffff9ae932693e20 EFLAGS: 00010282
RAX: 0000000000000000 RBX: 000000022498cfb8 RCX: ffff9aec30d5e960
RDX: ffff9aec314042c0 RSI: ffff9aec314041c0 RDI: ffff9aec313889c0
RBP: ffff9ae932693e40 R08: ffffffffa0522180 R09: ffff9ae77fc03300
R10: ffff9ae77fc03300 R11: ffffffff9f9de20d R12: ffff9ae800000000
R13: ffff9ae82498cf30 R14: 0000000090f7eac8 R15: 0000000000000000
FS: 00007fc8533fd880(0000) GS:ffff9aec3f740000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc8533d8000 CR3: 000000062f85c000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffff9fa759a0>] seq_read+0x130/0x440
[<ffffffff9fac62b0>] proc_reg_read+0x40/0x80
[<ffffffff9fa4cfbf>] vfs_read+0x9f/0x170
[<ffffffff9fa4de2f>] SyS_read+0x7f/0xf0
[<ffffffff9ff92ed2>] system_call_fastpath+0x25/0x2a
Code: 5c 41 5d 41 5e 5d c3 0f 1f 00 0f 1f 44 00 00 55 48 8d 96 00 01 00 00 31 c0 48 89 e5 41 55 41 54 49 89 fc 53 48 89 f3 48 83 ec 08 <8b> 8e 10 01 00 00 48 c7 c6 2b a0 30 a0 4c 8b 6f 78 e8 49 6d b4
Complete report #943252
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
if6_seq_next
|
vmlinux |
0xffffffffb0b1d5ba
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xca
|
|
2
|
seq_read
|
vmlinux |
0xffffffffb0675b99
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x279
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffffb06c6330
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x40
|
|
4
|
vfs_read
|
vmlinux |
0xffffffffb064d06f
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x9f
|
|
5
|
SyS_read
|
vmlinux |
0xffffffffb064dedf
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x7f
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffffb0b92ed2
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [vmtoolsd:819]
Modules linked in: rfcomm fuse bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi crc32_pclmul ghash_clmulni_intel ppdev snd_ac97_codec aesni_intel btusb vmw_balloon btrtl btbcm lrw gf128mul glue_helper ac97_bus btintel ablk_helper cryptd snd_seq bluetooth snd_seq_device snd_pcm joydev pcspkr snd_timer rfkill snd sg soundcore vmw_vmci i2c_piix4 parport_pc parport sch_fq_codel binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit libnvdimm crct10dif_pclmul crct10dif_common ata_piix drm crc32c_intel mptspi libata scsi_transport_spi mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash
dm_log dm_mod
CPU: 3 PID: 819 Comm: vmtoolsd Tainted: G L ------------ 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8a34381620e0 ti: ffff8a3437f00000 task.ti: ffff8a3437f00000
RIP: 0010:[<ffffffffb0b1d5ba>] [<ffffffffb0b1d5ba>] if6_seq_next+0xca/0x100
RSP: 0018:ffff8a3437f03e40 EFLAGS: 00000246
RAX: ffff8a34391d6c00 RBX: ffff8a3437f03e20 RCX: 0000000000000002
RDX: ffff8a3437f03e78 RSI: ffff8a343a534000 RDI: ffffffffb1115c80
RBP: ffff8a3437f03e40 R08: ffff8a33e103bbc0 R09: 0000000000000000
R10: 0000000000000000 R11: 000000000000000f R12: ffffffffb06750e1
R13: ffff8a3437f03dc0 R14: ffff8a3412789c80 R15: ffff8a3437f03f18
FS: 00007f6c7e6ef880(0000) GS:ffff8a343b6c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c7e700000 CR3: 0000000077abe000 CR4: 0000000000360fe0
Call Trace:
[<ffffffffb0675b99>] seq_read+0x279/0x440
[<ffffffffb06c6330>] proc_reg_read+0x40/0x80
[<ffffffffb064d06f>] vfs_read+0x9f/0x170
[<ffffffffb064dedf>] SyS_read+0x7f/0xf0
[<ffffffffb0b92ed2>] system_call_fastpath+0x25/0x2a
Code: 00 48 8b 80 d8 00 00 00 48 85 c0 74 c1 48 2d d8 00 00 00 48 85 c0 74 b6 48 8b b0 c8 00 00 00 48 8b 36 48 3b be e8 03 00 00 75 d6 <41> c7 40 0c 01 00 00 00 48 83 02 01 5d c3 0f 1f 84 00 00 00 00
Complete report #940193
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_user_generic_unrolled
|
vmlinux |
0xffffffff81dab901
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x41
|
|
2
|
|
vmlinux |
0xffffffff81c85a8c
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x36c
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffff81cd7950
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x40
|
|
4
|
vfs_read
|
vmlinux |
0xffffffff81c5bb9f
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x9f
|
|
5
|
SyS_read
|
vmlinux |
0xffffffff81c5ca15
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x55
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffff821c539a
Build id: 3.10.0-1160.108.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [nmon:15688]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc overlay(T) vmw_vsock_vmci_transport vsock sunrpc nfit libnvdimm coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg vmw_balloon pcspkr joydev parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops pata_acpi ttm drm crct10dif_pclmul
crct10dif_common mptspi scsi_transport_spi crc32c_intel ata_piix serio_raw mptscsih libata mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 15688 Comm: nmon Kdump: loaded Tainted: G L ------------ T 3.10.0-1160.108.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9da3f4d5d280 ti: ffff9da20314c000 task.ti: ffff9da20314c000
RIP: 0010:[<ffffffff81dab901>] [<ffffffff81dab901>] copy_user_generic_unrolled+0x41/0xc0
RSP: 0018:ffff9da20314fe48 EFLAGS: 00050202
RAX: ffff9da20314c000 RBX: 0000000000000004 RCX: 0000000000000008
RDX: 000000000000001f RSI: ffff9da381d0d000 RDI: 00007f3c13212000
RBP: ffff9da20314feb0 R08: 2020202032202020 R09: 3064662030202020
R10: 2032332030203420 R11: 2030203020323531 R12: 0000000000001000
R13: 0000031a00000305 R14: 0000000000000005 R15: 000003150314fd90
FS: 00007f3c13200740(0000) GS:ffff9da3ffd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f703865b680 CR3: 0000000043620000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff81c85a8c>] ? seq_read+0x36c/0x460
[<ffffffff81cd7950>] proc_reg_read+0x40/0x80
[<ffffffff81c5bb9f>] vfs_read+0x9f/0x170
[<ffffffff81c5ca15>] SyS_read+0x55/0xd0
[<ffffffff821c539a>] system_call_fastpath+0x25/0x2a
Code: f7 d9 29 ca 8a 06 88 07 48 ff c6 48 ff c7 ff c9 75 f2 89 d1 83 e2 3f c1 e9 06 74 4a 4c 8b 06 4c 8b 4e 08 4c 8b 56 10 4c 8b 5e 18 <4c> 89 07 4c 89 4f 08 4c 89 57 10 4c 89 5f 18 4c 8b 46 20 4c 8b
Complete report #570003
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
mutex_lock
|
vmlinux |
0xffffffff8837d2c8
Build id: 3.10.0-1062.el7.x86_64
|
0x18
|
|
2
|
seq_read
|
vmlinux |
0xffffffff87e7050a
Build id: 3.10.0-1062.el7.x86_64
|
0x4a
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffff87ec09b0
Build id: 3.10.0-1062.el7.x86_64
|
0x40
|
|
4
|
vfs_read
|
vmlinux |
0xffffffff87e484ff
Build id: 3.10.0-1062.el7.x86_64
|
0x9f
|
|
5
|
SyS_read
|
vmlinux |
0xffffffff87e493bf
Build id: 3.10.0-1062.el7.x86_64
|
0x7f
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffff8838bede
Build id: 3.10.0-1062.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 36s! [vmtoolsd:1463]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper pcspkr vmw_balloon sg cryptd joydev vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod vmwgfx cdrom sd_mod ata_generic crc_t10dif crct10dif_generic pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit libnvdimm ata_piix mptspi libata scsi_transport_spi mptscsih e1000 crct10dif_pclmul crct10dif_common
mptbase crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 8 PID: 1463 Comm: vmtoolsd Kdump: loaded Tainted: G L ------------ 3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff9d20d5c0d230 ti: ffff9d20d77d0000 task.ti: ffff9d20d77d0000
RIP: 0010:[<ffffffff8837d2c8>] [<ffffffff8837d2c8>] mutex_lock+0x18/0x2f
RSP: 0018:ffff9d20d77d3e38 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9d20bb7fb600 RCX: ffff9d20d77d3fd8
RDX: 0000000000000000 RSI: 00007f6f00972000 RDI: ffff9d1be9aed8f8
RBP: ffff9d20d77d3e40 R08: 0000000000000000 R09: ffff9d20d9aec4e4
R10: 0000000000000001 R11: 0000000000000293 R12: ffff9d20d77d3f18
R13: ffff9d1be9aed8c0 R14: ffffffff88382768 R15: ffff9d20d77d3eb0
FS: 00007f6f00955880(0000) GS:ffff9d20dd800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6f00972000 CR3: 000000081767c000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff87e7050a>] seq_read+0x4a/0x440
[<ffffffff87ec09b0>] proc_reg_read+0x40/0x80
[<ffffffff87e484ff>] vfs_read+0x9f/0x170
[<ffffffff87e493bf>] SyS_read+0x7f/0xf0
[<ffffffff8838bede>] system_call_fastpath+0x25/0x2a
Code: 05 e8 fd 0a 00 00 5d c3 0f 1f 00 83 ea 01 89 50 10 eb d9 0f 1f 44 00 00 55 48 89 e5 53 48 89 fb e8 6e 21 00 00 48 89 df f0 ff 0f <79> 05 e8 61 0b 00 00 65 48 8b 04 25 80 0e 01 00 48 89 43 18 5b
Complete report #356162
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
swap_stop
|
vmlinux |
0xffffffff9160b130
Build id: 3.10.0-1160.53.1.el7.x86_64
|
-
|
|
2
|
|
vmlinux |
0xffffffff91676f03
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x323
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffff916c7770
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x40
|
|
4
|
vfs_read
|
vmlinux |
0xffffffff9164e3ff
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x9f
|
|
5
|
SyS_read
|
vmlinux |
0xffffffff9164f27f
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x7f
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffff91b95f92
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [udisksd:588]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev edac_mce_amd crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi aesni_intel snd_ac97_codec ac97_bus lrw gf128mul snd_seq glue_helper ablk_helper cryptd vmw_balloon snd_seq_device
snd_pcm snd_timer snd pcspkr joydev soundcore sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic crct10dif_pclmul crct10dif_common crc32c_intel pata_acpi serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm mptspi nfit libata scsi_transport_spi e1000 libnvdimm mptscsih mptbase drm_panel_orientation_quirks fuse
CPU: 1 PID: 588 Comm: udisksd Kdump: loaded Tainted: G L ------------ 3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff98e377364200 ti: ffff98e376530000 task.ti: ffff98e376530000
RIP: 0010:[<ffffffff9160b130>] [<ffffffff9160b130>] swap_stop+0x0/0x20
RSP: 0018:ffff98e376533e48 EFLAGS: 00000246
RAX: ffffffff9160b130 RBX: ffff98e376533e28 RCX: 0000000000000001
RDX: ffff98e376533e78 RSI: 0000000000000000 RDI: ffff98e375120c00
RBP: ffff98e376533eb0 R08: 000000000000000a R09: 0000000000000067
R10: 0000000000000000 R11: ffff98e376533cae R12: ffffffff91676423
R13: ffff98e376533e08 R14: ffff98e378b90000 R15: ffffffff916763a1
FS: 00007ff865c418c0(0000) GS:ffff98e37b640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4f26340f08 CR3: 000000007652e000 CR4: 00000000001407e0
Call Trace:
[<ffffffff91676f03>] ? seq_read+0x323/0x450
[<ffffffff916c7770>] proc_reg_read+0x40/0x80
[<ffffffff9164e3ff>] vfs_read+0x9f/0x170
[<ffffffff9164f27f>] SyS_read+0x7f/0xf0
[<ffffffff91b95f92>] system_call_fastpath+0x25/0x2a
Code: 48 83 78 10 00 74 06 48 83 eb 01 74 0f 8b 05 64 bf fb 00 83 c2 01 39 c2 72 d1 31 c0 5b 5d c3 66 90 66 2e 0f 1f 84 00 00 00 00 00 <66> 66 66 66 90 55 48 c7 c7 e0 e4 09 92 48 89 e5 e8 3b c4 57 00
Complete report #240115
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
frag_start
|
vmlinux |
0xffffffff811a1315
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x5
|
|
2
|
|
vmlinux |
0xffffffff812225c7
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x177
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffff8126b75d
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x3d
|
|
4
|
vfs_read
|
vmlinux |
0xffffffff811fe0fe
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x9e
|
|
5
|
SyS_read
|
vmlinux |
0xffffffff811feccf
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x7f
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffff816967c9
Build id: 3.10.0-514.2.2.el7.x86_64
|
0x16
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 48s! [vmtoolsd:675]
Modules linked in: vmw_vsock_vmci_transport vsock intel_powerclamp coretemp crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon sg pcspkr parport_pc parport shpchp vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul crct10dif_common crc32c_intel sysimgblt mptspi fb_sys_fops ttm serio_raw drm scsi_transport_spi ata_piix mptscsih libata e1000 i2c_core mptbase floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 675 Comm: vmtoolsd Tainted: G L ------------ 3.10.0-514.2.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/07/2011
task: ffff88007b818000 ti: ffff88007ba54000 task.ti: ffff88007ba54000
RIP: 0010:[<ffffffff811a1315>] [<ffffffff811a1315>] frag_start+0x5/0x40
RSP: 0018:ffff88007ba57e70 EFLAGS: 00000246
RAX: ffffffff816cace0 RBX: 0000000000000000 RCX: 00000000002d8b22
RDX: fffffffffffffff4 RSI: ffff88007ba57e98 RDI: ffff88007831fe80
RBP: ffff88007ba57ed8 R08: 0000000000019b80 R09: ffffffff81222057
R10: ffff88007f801300 R11: 0000000000000246 R12: 00000007f57c4b2a
R13: 0000000000000000 R14: ffffffff811b70aa R15: ffff88007ba57e50
FS: 00007f57c4b18880(0000) GS:ffff88007fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f57c4b2a000 CR3: 0000000079de6000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff812225c7 ffff880079c00900 00007f57c4b2a000 ffff88007831feb8
0000000000000400 0000000000000000 ffff88007ba57ed0 0000000092f1c37a
ffff88007c218cc0 00007f57c4b2a000 ffff88007ba57f48 0000000000000400
Call Trace:
[<ffffffff812225c7>] ? seq_read+0x177/0x3a0
[<ffffffff8126b75d>] proc_reg_read+0x3d/0x80
[<ffffffff811fe0fe>] vfs_read+0x9e/0x170
[<ffffffff811feccf>] SyS_read+0x7f/0xe0
[<ffffffff816967c9>] system_call_fastpath+0x16/0x1b
Code: 84 00 00 00 00 00 66 66 66 66 90 48 83 02 01 55 48 89 f7 48 89 e5 e8 2b f1 ff ff 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 <55> 48 89 e5 53 48 8b 1e e8 ce f0 ff ff 48 85 c0 75 1a eb 1d 0f
Complete report #210527
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
init_irq_proc
|
vmlinux |
0xffffffff8113a660
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x90
|
|
2
|
|
vmlinux |
0xffffffff8122b30c
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x26c
|
|
3
|
proc_reg_read
|
vmlinux |
0xffffffff812756b0
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x40
|
|
4
|
vfs_read
|
vmlinux |
0xffffffff812054ef
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x9f
|
|
5
|
SyS_read
|
vmlinux |
0xffffffff812063bf
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x7f
|
|
6
|
system_call_fastpath
|
vmlinux |
0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x1c
|
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 36s! [microsoft-depen:12106]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 36s! [irqbalance:658]
Modules linked in: CyProtectDrv(OE) microsoft_dependency_agent(POE) bluechannel(POE) sctp_diag sctp dccp_diag dccp unix_diag af_packet_diag netlink_diag rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache udp_diag tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg nfit libnvdimm parport_pc vmw_vmci parport shpchp i2c_piix4 nfsd auth_rpcgss
nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crct10dif_pclmul ttm crct10dif_common crc32c_intel serio_raw drm ata_piix libata mptspi scsi_transport_spi mptscsih vmxnet3 mptbase i2c_core floppy [last unloaded: CyProtectDrv]
CPU: 1 PID: 658 Comm: irqbalance Tainted: P OEL ------------ 3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
task: ffff880034cedee0 ti: ffff880425ac4000 task.ti: ffff880425ac4000
RIP: 0010:[<ffffffff8113a660>] [<ffffffff8113a660>] init_irq_proc+0x90/0x90
RSP: 0018:ffff880425ac7e48 EFLAGS: 00000287
RAX: ffffffff8113a660 RBX: ffff880425ac7dd0 RCX: 00000000000001c5
RDX: ffff880425ac7e78 RSI: ffff880425ac7e78 RDI: ffff88036de6fa80
RBP: ffff880425ac7eb0 R08: 0000000000000002 R09: 00000000000002f1
R10: 0000000000000000 R11: ffff880425ac7c8e R12: 000000002876eb81
R13: ffff880425ac7db8 R14: ffff880425ac7df8 R15: ffff880300000018
FS: 00007f643151b780(0000) GS:ffff88043fc40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6431536000 CR3: 0000000425f92000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8122b30c>] ? seq_read+0x26c/0x3e0
[<ffffffff812756b0>] proc_reg_read+0x40/0x80
[<ffffffff812054ef>] vfs_read+0x9f/0x170
[<ffffffff812063bf>] SyS_read+0x7f/0xe0
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: ff ff 83 c3 01 89 df e8 ff 97 ff ff 39 1d 41 6e 91 00 77 df 5b 5d c3 0f 1f 40 00 0f 1f 44 00 00 55 31 c0 48 89 e5 5d c3 0f 1f 00 <0f> 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 83 ec