Complete report #734794
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffffb7805512
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x12
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffffb7dbf6fd
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x13d
|
|
3
|
|
vmlinux |
0xffffffffb76e1170
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x20
|
|
4
|
do_page_fault
|
vmlinux |
0xffffffffb7dbfb05
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x35
|
|
5
|
page_fault
|
vmlinux |
0xffffffffb7dbb7b8
Build id: 3.10.0-1160.88.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [node:70308]
Modules linked in: binfmt_misc vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel ac97_bus snd_seq lrw gf128mul vmw_balloon glue_helper ablk_helper snd_seq_device cryptd snd_pcm joydev pcspkr snd_timer sg snd soundcore parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom vmwgfx ata_generic pata_acpi drm_kms_helper sd_mod syscopyarea crc_t10dif sysfillrect crct10dif_generic sysimgblt fb_sys_fops ttm ahci crct10dif_pclmul crct10dif_common drm crc32c_intel libahci ata_piix libata serio_raw e1000 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 70308 Comm: node Kdump: loaded Tainted: G L ------------ 3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff907038122100 ti: ffff9071625b4000 task.ti: ffff9071625b4000
RIP: 0010:[<ffffffffb7805512>] [<ffffffffb7805512>] find_vma+0x12/0x70
RSP: 0000:ffff9071625b7ea8 EFLAGS: 00010286
RAX: ffff907260c8f7a0 RBX: 00000000ec1be19a RCX: ffff9071625b7fd8
RDX: 0000000000000000 RSI: 0000155d499d5088 RDI: ffff907277980c80
RBP: ffff9071625b7ea8 R08: 000000000000000f R09: 00003933820c1309
R10: 00007fffd12349b8 R11: 000000000000009f R12: 0000000000726576
R13: 7265730065646f6e R14: ffffffffb7db7006 R15: 00000000000112a4
FS: 00007f000d42e7c0(0000) GS:ffff9072ad600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000155d499d5088 CR3: 0000000364388000 CR4: 00000000001607f0
Call Trace:
[<ffffffffb7dbf6fd>] __do_page_fault+0x13d/0x510
[<ffffffffb76e1170>] ? wake_up_state+0x20/0x20
[<ffffffffb7dbfb05>] do_page_fault+0x35/0x90
[<ffffffffb7dbb7b8>] page_fault+0x28/0x30
Code: 77 48 73 04 48 89 77 48 5d c3 cc cc cc cc 90 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 8b 47 10 48 89 e5 48 85 c0 74 06 <48> 39 70 08 77 48 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f
[drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:33:plane-0] flip_done timed out
Complete report #627530
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffff86405208
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x28
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffff869bf6fd
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x13d
|
|
3
|
do_page_fault
|
vmlinux |
0xffffffff869bfb05
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x35
|
|
4
|
page_fault
|
vmlinux |
0xffffffff869bb7b8
Build id: 3.10.0-1160.81.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 259s! [Compositor:3409]
<EOI>
Modules linked in:
tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vboxsf(OE) sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device nfit libnvdimm snd_pcm ppdev joydev snd_timer sg pcspkr iosf_mbi snd parport_pc parport i2c_piix4 vboxguest(OE) video soundcore ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi ahci libahci ata_piix vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crc32c_intel serio_raw e1000 libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log
dm_mod fuse
CPU: 1 PID: 3409 Comm: Compositor Kdump: loaded Tainted: G W OE ------------ 3.10.0-1160.81.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff91ac9823d280 ti: ffff91a990470000 task.ti: ffff91a990470000
RIP: 0010:[<ffffffff86405208>] [<ffffffff86405208>] find_vma+0x28/0x70
RSP: 0000:ffff91a990473ea8 EFLAGS: 00010283
RAX: ffff91ac583b6000 RBX: ffff91a990473e58 RCX: ffff91a990473fd8
RDX: ffff91ac3e2e77b8 RSI: 00007f0fe3e53010 RDI: ffff91ac9824e400
RBP: ffff91a990473ea8 R08: 00007f0fe3e53000 R09: 0000000000000000
R10: 0000000000000000 R11: 00007f1037314810 R12: ffff91a990473e58
R13: ffffffff862e5bb8 R14: ffff91a990473e10 R15: ffff91ac9823d2e8
FS: 00007f10119fb700(0000) GS:ffff91acdfd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0fe3e53010 CR3: 00000003d4522000 CR4: 00000000000206e0
Call Trace:
[<ffffffff869bf6fd>] __do_page_fault+0x13d/0x510
[<ffffffff869bfb05>] do_page_fault+0x35/0x90
[<ffffffff869bb7b8>] page_fault+0x28/0x30
Code: 00 00 00 0f 1f 44 00 00 55 48 8b 47 10 48 89 e5 48 85 c0 74 06 48 39 70 08 77 48 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f 00 <48> 3b 72 e0 48 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f 48 3b
Complete report #592116
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffffb9e05221
Build id: 3.10.0-1160.80.1.el7.x86_64
|
0x41
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffffba3bf6fd
Build id: 3.10.0-1160.80.1.el7.x86_64
|
0x13d
|
|
3
|
do_page_fault
|
vmlinux |
0xffffffffba3bfb05
Build id: 3.10.0-1160.80.1.el7.x86_64
|
0x35
|
|
4
|
page_fault
|
vmlinux |
0xffffffffba3bb7b8
Build id: 3.10.0-1160.80.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 4129s! [cpptools:17932]
Modules linked in: tcp_lp vboxsf(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc intel_powerclamp crc32_pclmul ghash_clmulni_intel snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device joydev snd_pcm ppdev aesni_intel snd_timer lrw gf128mul glue_helper ablk_helper cryptd snd pcspkr sg vboxguest(OE) i2c_piix4 soundcore
parport_pc video parport ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix drm libahci libata e1000 crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 17932 Comm: cpptools Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.80.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff920b45ac8000 ti: ffff920af2edc000 task.ti: ffff920af2edc000
RIP: 0010:[<ffffffffb9e05221>] [<ffffffffb9e05221>] find_vma+0x41/0x70
RSP: 0000:ffff920af2edfea8 EFLAGS: 00000202
RAX: ffff920b0ab05d80 RBX: 00000000000000a9 RCX: ffff920af2edffd8
RDX: ffff920b1878abf0 RSI: 00000000020e0ff8 RDI: ffff920b55579900
RBP: ffff920af2edfea8 R08: 00007efe87dd89a0 R09: 00007efe84055010
R10: 00007ffc3cfb6020 R11: 0000000000000246 R12: 0000000300000000
R13: 0000000000000001 R14: 0000000100000001 R15: ffff920af2edc000
FS: 00007efea6d50780(0000) GS:ffff920b5fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000020e0ff8 CR3: 00000000d207e000 CR4: 00000000000606f0
Call Trace:
[<ffffffffba3bf6fd>] __do_page_fault+0x13d/0x510
[<ffffffffba3bfb05>] do_page_fault+0x35/0x90
[<ffffffffba3bb7b8>] page_fault+0x28/0x30
Code: 77 48 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f 00 48 3b 72 e0 48 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f 48 3b 72 e8 72 e7 <48> 8b 52 08 48 85 d2 75 f1 48 85 c0 74 04 48 89 47 10 5d c3 cc
Complete report #522647
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffffbcdf8e01
Build id: 3.10.0-1160.el7.x86_64
|
0x41
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffffbd38e57d
Build id: 3.10.0-1160.el7.x86_64
|
0x13d
|
|
3
|
|
vmlinux |
0xffffffffbce5398e
Build id: 3.10.0-1160.el7.x86_64
|
0x3e
|
|
4
|
do_page_fault
|
vmlinux |
0xffffffffbd38e975
Build id: 3.10.0-1160.el7.x86_64
|
0x35
|
|
5
|
|
vmlinux |
0xffffffffbd38aac9
Build id: 3.10.0-1160.el7.x86_64
|
0xaa
|
|
6
|
page_fault
|
vmlinux |
0xffffffffbd38a778
Build id: 3.10.0-1160.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 6413s! [abrt-hook-ccpp:20558]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device aesni_intel joydev snd_pcm lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg parport_pc parport snd_timer video snd i2c_piix4 soundcore
ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm e1000 ahci crct10dif_pclmul crct10dif_common libahci ata_piix crc32c_intel serio_raw libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 20558 Comm: abrt-hook-ccpp Kdump: loaded Not tainted 3.10.0-1160.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff967fd711e300 ti: ffff967f9ea14000 task.ti: ffff967f9ea14000
RIP: 0010:[<ffffffffbcdf8e01>] [<ffffffffbcdf8e01>] find_vma+0x41/0x60
RSP: 0000:ffff967f9ea17ea8 EFLAGS: 00000202
RAX: ffff967fbdb00e58 RBX: 0000000000000000 RCX: ffff967f9ea17fd8
RDX: ffff967fd71b7cc8 RSI: 00007fee051d3468 RDI: ffff967ff5ef6400
RBP: ffff967f9ea17ea8 R08: 000055ca0308ebf0 R09: 000055ca03083fc0
R10: 00007fff83a88060 R11: 0000000000000246 R12: 0000000000000001
R13: 00000000020a7bd2 R14: 000000000000fd00 R15: ffffffffbce53915
FS: 00007fee090b9900(0000) GS:ffff967fffc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fee051d3468 CR3: 000000001e92c000 CR4: 00000000000606f0
Call Trace:
[<ffffffffbd38e57d>] __do_page_fault+0x13d/0x500
[<ffffffffbce5398e>] ? SYSC_newstat+0x3e/0x60
[<ffffffffbd38e975>] do_page_fault+0x35/0x90
[<ffffffffbd38aac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffbd38a778>] page_fault+0x28/0x30
Code: 77 40 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f 00 48 3b 72 e0 48 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f 48 3b 72 e8 72 e7 <48> 8b 52 08 48
Complete report #371057
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffff915f8f2b
Build id: 3.10.0-1127.10.1.el7.x86_64
|
0x3b
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffff91b8d57d
Build id: 3.10.0-1127.10.1.el7.x86_64
|
0x13d
|
|
3
|
do_page_fault
|
vmlinux |
0xffffffff91b8d975
Build id: 3.10.0-1127.10.1.el7.x86_64
|
0x35
|
|
4
|
|
vmlinux |
0xffffffff91b89ac9
Build id: 3.10.0-1127.10.1.el7.x86_64
|
0xaa
|
|
5
|
page_fault
|
vmlinux |
0xffffffff91b89778
Build id: 3.10.0-1127.10.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [httpd:7690]
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt sd_mod fb_sys_fops
Modules linked in: talpa_vfshook(OE) talpa_pedconnector(OE) talpa_vcdevice(OE) talpa_core(OE) talpa_linux(OE) talpa_syscallhook(OE) tcp_diag inet_diag binfmt_misc ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 ip6table_filter nf_log_ipv4 nf_log_common nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_conntrack nf_conntrack_ftp xt_LOG xt_limit talpa_pedevice(OE) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle xt_multiport ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 iptable_filter sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc vmw_vmci parport i2c_piix4 tcp_htcp
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt sd_mod fb_sys_fops crc_t10dif ttm ata_piix crct10dif_generic crct10dif_pclmul drm crct10dif_common libata serio_raw e1000 crc32c_intel drm_panel_orientation_quirks vmw_pvscsi floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: talpa_syscallhook]
CPU: 6 PID: 7690 Comm: httpd Kdump: loaded Tainted: G W OEL ------------ 3.10.0-1127.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff9f1aae7d1070 ti: ffff9f16b94e0000 task.ti: ffff9f16b94e0000
RIP: 0010:[<ffffffff915f8f2b>] [<ffffffff915f8f2b>] find_vma+0x3b/0x60
RSP: 0000:ffff9f16b94e3ea8 EFLAGS: 00010286
RAX: ffff9f1aacfd4ca8 RBX: 00000000000000a8 RCX: ffff9f16b94e3fd8
RDX: ffff9f1829e43460 RSI: 00007f785818b2c0 RDI: ffff9f1a34b9a580
RBP: ffff9f16b94e3ea8 R08: 00005620dbea8350 R09: 00005620dbea8360
R10: 000000000000001e R11: 00007f7859e12ed0 R12: 0000000000001ea3
R13: 00005620dbeaa1f5 R14: ffffffff9164c5e3 R15: ffff9f16b94e3ed0
FS: 00007f785a6c7880(0000) GS:ffff9f1abfd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f785818b2c0 CR3: 000000005a922000 CR4: 00000000000407e0
Call Trace:
[<ffffffff91b8d57d>] __do_page_fault+0x13d/0x500
[<ffffffff91b8d975>] do_page_fault+0x35/0x90
[<ffffffff91b89ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffff91b89778>] page_fault+0x28/0x30
Code: 74 06 48 39 70 08 77 40 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f 00 48 3b 72 e0 48 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f <48> 3b 72 e8 72 e7 48 8b 52 08 48 85 d2 75 f1 48 85 c0 74 04 48
Complete report #363037
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffff8eff9748
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x18
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffff8f59057d
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x13d
|
|
3
|
do_page_fault
|
vmlinux |
0xffffffff8f590975
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x35
|
|
4
|
|
vmlinux |
0xffffffff8f58cac9
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0xaa
|
|
5
|
page_fault
|
vmlinux |
0xffffffff8f58c778
Build id: 3.10.0-1160.53.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [oracle_2958_ddm:2958]
Modules linked in: vboxsf(OE) sunrpc ppdev vboxguest(OE) joydev iosf_mbi pcspkr sg parport_pc video i2c_piix4 parport sch_fq_codel binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt ata_piix fb_sys_fops ttm libata drm serio_raw crc32c_intel e1000 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 2958 Comm: oracle_2958_ddm Tainted: G OEL ------------ 3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9d9fe506d280 ti: ffff9d9ee195c000 task.ti: ffff9d9ee195c000
RIP: 0010:[<ffffffff8eff9748>] [<ffffffff8eff9748>] find_vma+0x18/0x60
RSP: 0000:ffff9d9ee195fea8 EFLAGS: 00010287
RAX: ffff9d9ee685fca8 RBX: 00000000000000a9 RCX: ffff9d9ee195ffd8
RDX: 0000000000000000 RSI: 00000000f51bba60 RDI: ffff9d9fe7d044c0
RBP: ffff9d9ee195fea8 R08: 0000000000000000 R09: 0000000000000050
R10: 00007f1fc99cd280 R11: 00007fff3ce86b90 R12: 0000000000000001
R13: ffff9d9ee195fe90 R14: ffffffff8ee97b34 R15: ffff9d9ee195fe18
FS: 00007f1fc99cd280(0000) GS:ffff9da0dfd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000f51bba60 CR3: 000000001e9d4000 CR4: 00000000000206e0
Call Trace:
[<ffffffff8f59057d>] __do_page_fault+0x13d/0x500
[<ffffffff8f590975>] do_page_fault+0x35/0x90
[<ffffffff8f58cac9>] ? error_swapgs+0xaa/0xc0
[<ffffffff8f58c778>] page_fault+0x28/0x30
Code: 77 48 5d c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 8b 47 10 48 89 e5 48 85 c0 74 06 48 39 70 08 77 40 <48> 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f 00 48 3b 72 e0 48
Complete report #94172
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffff8e5ef1bb
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x3b
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffff8eb7050f
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x12f
|
|
3
|
|
vmlinux |
0xffffffff8e6475fe
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x3e
|
|
4
|
do_page_fault
|
vmlinux |
0xffffffff8eb70915
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x35
|
|
5
|
|
vmlinux |
0xffffffff8eb6ca96
Build id: 3.10.0-957.10.1.el7.x86_64
|
0xa7
|
|
6
|
page_fault
|
vmlinux |
0xffffffff8eb6c758
Build id: 3.10.0-957.10.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ada_daemon:5818]
Modules linked in: osst st ch iscsit(OE) vtldev(OE) vtlcore(OE) qla2xxx(OE) nvme_fc nvme_fabrics nvme_core scsi_transport_fc scsi_tgt xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel ext4 aesni_intel lrw gf128mul glue_helper ablk_helper cryptd mbcache jbd2 vmw_balloon pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw e1000 vmw_pvscsi vmwgfx drm_kms_helper ata_generic pata_acpi floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit ata_piix libnvdimm libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 5818 Comm: ada_daemon Kdump: loaded Tainted: G OEL ------------ 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff8fb8784a1040 ti: ffff8fb879380000 task.ti: ffff8fb879380000
RIP: 0010:[<ffffffff8e5ef1bb>] [<ffffffff8e5ef1bb>] find_vma+0x3b/0x60
RSP: 0000:ffff8fb879383ea8 EFLAGS: 00010286
RAX: ffff8fb87a004288 RBX: 0000000000000000 RCX: ffff8fb879383fd8
RDX: ffff8fb8791957b8 RSI: 0000000003ce19e8 RDI: ffff8fb8786ccc80
RBP: ffff8fb879383ea8 R08: 0000000000000602 R09: 0000000000000000
R10: 0000000000000000 R11: 00000000010263d0 R12: 0000000000000001
R13: 0000000000000511 R14: 000000000000fd01 R15: ffffffff8e647585
FS: 00007f334eec8740(0000) GS:ffff8fb87fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000003ce19e8 CR3: 000000007a61c000 CR4: 00000000000607f0
Call Trace:
[<ffffffff8eb7050f>] __do_page_fault+0x12f/0x500
[<ffffffff8e6475fe>] ? SYSC_newstat+0x3e/0x60
[<ffffffff8eb70915>] do_page_fault+0x35/0x90
[<ffffffff8eb6ca96>] ? error_swapgs+0xa7/0xbd
[<ffffffff8eb6c758>] page_fault+0x28/0x30
Code: 74 06 48 39 70 08 77 40 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f 00 48 3b 72 e0 48 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f <48> 3b 72 e8 72 e7 48 8b 52 08 48 85 d2 75 f1 48 85 c0 74 04 48
Complete report #53565
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffffb37f8ff8
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x58
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffffb3d8e57d
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x13d
|
|
3
|
do_page_fault
|
vmlinux |
0xffffffffb3d8e975
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x35
|
|
4
|
|
vmlinux |
0xffffffffb3d8aac9
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0xaa
|
|
5
|
page_fault
|
vmlinux |
0xffffffffb3d8a778
Build id: 3.10.0-1160.11.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 40s! [mcpcom:23817]
Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd grace fscache xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nls_utf8 cifs ccm dns_resolver nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper
ablk_helper cryptd pcspkr joydev sg vmw_vmci parport_pc parport i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi serio_raw scsi_transport_spi mptscsih e1000 libata mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 23817 Comm: mcpcom Tainted: G L ------------ 3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff905e0cdac200 ti: ffff905ec0080000 task.ti: ffff905ec0080000
RIP: 0010:[<ffffffffb37f8ff8>] [<ffffffffb37f8ff8>] find_vma+0x58/0x60
RSP: 0000:ffff905ec0083ea8 EFLAGS: 00000212
RAX: ffff905f3bbf6360 RBX: 00000000000000a9 RCX: ffff905ec0083fd8
RDX: 0000000000000000 RSI: 00000000062d7008 RDI: ffff905ef99ee400
RBP: ffff905ec0083ea8 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000004304c90 R11: 00000000000000c8 R12: ffff905f3bbf6360
R13: 8000000106d2d867 R14: ffff905f3bbf6360 R15: ffff905ec0083e20
FS: 00007ffb587f9780(0000) GS:ffff905fffc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000062d7008 CR3: 000000018f4b6000 CR4: 00000000000407e0
Call Trace:
[<ffffffffb3d8e57d>] __do_page_fault+0x13d/0x500
[<ffffffffb3d8e975>] do_page_fault+0x35/0x90
[<ffffffffb3d8aac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffb3d8a778>] page_fault+0x28/0x30
Code: 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f 48 3b 72 e8 72 e7 48 8b 52 08 48 85 d2 75 f1 48 85 c0 74 04 48 89 47 10 5d c3 0f 1f 00 <48> 3b 30 72 bb 5d c3 90 0f 1f 44 00 00 55 48 89 e5 5d c3 0f 1f
Complete report #14489
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffff91df6d72
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x12
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffff9238857d
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x13d
|
|
3
|
do_page_fault
|
vmlinux |
0xffffffff92388975
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x35
|
|
4
|
page_fault
|
vmlinux |
0xffffffff92384778
Build id: 3.10.0-1062.18.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [pool:24448]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod ppdev vmw_balloon sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr
sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci ata_piix libahci drm libata nfit libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks
CPU: 4 PID: 24448 Comm: pool Kdump: loaded Tainted: G ------------ T 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9b809e169070 ti: ffff9b7e301f4000 task.ti: ffff9b7e301f4000
RIP: 0010:[<ffffffff91df6d72>] [<ffffffff91df6d72>] find_vma+0x12/0x60
RSP: 0000:ffff9b7e301f7ea8 EFLAGS: 00010282
RAX: ffff9b80ec338e58 RBX: ffff9b80ffd1ac80 RCX: ffff9b7e301f7fd8
RDX: 0000000000000000 RSI: 00007f377000f028 RDI: ffff9b80eb995780
RBP: ffff9b7e301f7ea8 R08: 00007f377000ebd0 R09: 0000000000000420
R10: 0000000000000040 R11: 0000000000000002 R12: ffff9b7e301f7e38
R13: ffffffff91ce28c9 R14: ffff9b7e301f7e38 R15: ffffffff91ce665b
FS: 00007f377e1bf700(0000) GS:ffff9b80ffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f377000f028 CR3: 0000000428f9a000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9238857d>] __do_page_fault+0x13d/0x500
[<ffffffff92388975>] do_page_fault+0x35/0x90
[<ffffffff92384778>] page_fault+0x28/0x30
Code: 77 48 73 04 48 89 77 48 5d c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 8b 47 10 48 89 e5 48 85 c0 74 06 <48> 39 70 08 77 40 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f
Complete report #12998
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
find_vma
|
vmlinux |
0xffffffff811b99cb
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x3b
|
|
2
|
__do_page_fault
|
vmlinux |
0xffffffff816bb4c8
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x118
|
|
3
|
do_page_fault
|
vmlinux |
0xffffffff816bb835
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x35
|
|
4
|
page_fault
|
vmlinux |
0xffffffff816b7768
Build id: 3.10.0-693.21.1.el7.x86_64
|
0x28
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [systemd-journal:597]
Modules linked in: iptable_filter fuse binfmt_misc vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev ppdev vmw_balloon sg parport_pc vmw_vmci parport shpchp i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul libata mptspi crct10dif_common crc32c_intel scsi_transport_spi serio_raw mptscsih e1000 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 597 Comm: systemd-journal Tainted: G W L ------------ 3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff88071e4f8fd0 ti: ffff880719e7c000 task.ti: ffff880719e7c000
RIP: 0010:[<ffffffff811b99cb>] [<ffffffff811b99cb>] find_vma+0x3b/0x60
RSP: 0000:ffff880719e7feb8 EFLAGS: 00010282
RAX: 0000000000000000 RBX: 000009c126b44c61 RCX: ffff880719e7ffd8
RDX: ffff88071e468608 RSI: 00007f0674e0d578 RDI: ffff88071e427080
RBP: ffff880719e7feb8 R08: 0000000000000010 R09: 0000558ca88fd670
R10: 0000558ca88f93f0 R11: 0000000000000002 R12: 0000000000000004
R13: ffff88073fd65840 R14: 0000000000000001 R15: 00000000b1ed365d
FS: 00007f067a5ac880(0000) GS:ffff88073fc40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0674e0d578 CR3: 000000071a380000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff816bb4c8>] __do_page_fault+0x118/0x450
[<ffffffff816bb835>] do_page_fault+0x35/0x90
[<ffffffff816b7768>] page_fault+0x28/0x30
Code: 74 06 48 39 70 08 77 40 48 8b 57 08 31 c0 48 85 d2 75 18 eb 2e 0f 1f 00 48 3b 72 e0 48 8d 42 e0 73 1d 48 8b 52 10 48 85 d2 74 0f <48> 3b 72 e8 72 e7 48 8b 52 08 48 85 d2 75 f1 48 85 c0 74 04 48