Info

Permalink
Function
copy_page_rep
First occurrence
2020-09-22
Last occurrence
2020-09-22
State
NEW
Components
kernel
Quality
-15

Statistics

Operating system Count
CentOS 7 91
Architecture Count
x86_64 91
Related packages Count
kernel 91
 0:3.10.0-1062.18.1.el7 40
 0:3.10.0-1160.el7 12
 0:3.10.0-1127.13.1.el7 7
 0:3.10.0-1160.15.2.el7 7
 0:3.10.0-1062.9.1.el7 5
 0:3.10.0-1160.102.1.el7 4
 0:3.10.0-1160.119.1.el7 4
 0:3.10.0-1127.19.1.el7 2
 0:3.10.0-693.11.1.el7 2
 0:3.10.0-1160.95.1.el7 2
 0:3.10.0-1062.4.1.el7 2
 0:3.10.0-514.el7 1
 0:3.10.0-327.el7 1
 0:3.10.0-514.26.2.el7 1
 0:3.10.0-1160.11.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #219412
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff9b393b75
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff9b1ef1b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff9b1f17eb
Build id: 3.10.0-1127.19.1.el7.x86_64
0xfb
4
? do_swap_page
vmlinux 0xffffffff9b1f22a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x498
5
? unlock_page
vmlinux 0xffffffff9b1be7ab
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2b
6
handle_mm_fault
vmlinux 0xffffffff9b1f5e82
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb22
7
__do_page_fault
vmlinux 0xffffffff9b78d653
Build id: 3.10.0-1127.19.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff9b78da26
Build id: 3.10.0-1127.19.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff9b78cfa2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff9b7897a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [sshd:11529]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [sshd:11529]
joydev snd_pcm snd_timer parport_pc snd sg soundcore parport virtio_balloon xfs libcrc32c sr_mod cdrom ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm virtio_blk virtio_console ata_piix virtio_net net_failover failover libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw floppy virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 11529 Comm: sshd Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9248987ad230 ti: ffff92485e6f0000 task.ti: ffff92485e6f0000
RIP: 0010:[<ffffffff9b393b75>]  [<ffffffff9b393b75>] copy_page_rep+0x5/0x10
RSP: 0000:ffff92485e6f3ca0  EFLAGS: 00010286
RAX: fffff91800761980 RBX: ffff9248548bc5e8 RCX: 0000000000000200
RDX: ffffffff9ba8d5a0 RSI: ffff924859eaa000 RDI: ffff92485d866000
RBP: ffff92485e6f3d08 R08: ffffffff9ba8d58f R09: fffff918007619c0
R10: 000000000000366c R11: ffff9248548bc5e8 R12: 0000000000000000
R13: 00000000000200da R14: 000000002ddbc152 R15: ffff92485e6f3c50
FS:  00007fbf7d51f8c0(0000) GS:ffff92489fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055f6f748d7b0 CR3: 0000000010026000 CR4: 00000000000406f0
Call Trace:
[<ffffffff9b1ef1b0>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff9b1f17eb>] do_wp_page+0xfb/0x720
[<ffffffff9b1f22a8>] ? do_swap_page+0x498/0x7c0
[<ffffffff9b1be7ab>] ? unlock_page+0x2b/0x30
[<ffffffff9b1f5e82>] handle_mm_fault+0xb22/0xfb0
[<ffffffff9b78d653>] __do_page_fault+0x213/0x500
[<ffffffff9b78da26>] trace_do_page_fault+0x56/0x150
[<ffffffff9b78cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9b7897a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 66 66 66 90 66 66 66 90 
Complete report #1078274
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffffaffab7b5
Build id: 3.10.0-1160.119.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffffafdfb499
Build id: 3.10.0-1160.119.1.el7.x86_64
0x119
3
do_wp_page
vmlinux 0xffffffffafdfdbbb
Build id: 3.10.0-1160.119.1.el7.x86_64
0xfb
4
handle_mm_fault
vmlinux 0xffffffffafe022da
Build id: 3.10.0-1160.119.1.el7.x86_64
0xb3a
5
? destroy_inode
vmlinux 0xffffffffafe7ab0b
Build id: 3.10.0-1160.119.1.el7.x86_64
0x3b
6
__do_page_fault
vmlinux 0xffffffffb03bf7d3
Build id: 3.10.0-1160.119.1.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffffb03bfbb6
Build id: 3.10.0-1160.119.1.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffffb03bf112
Build id: 3.10.0-1160.119.1.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffffb03bb7e8
Build id: 3.10.0-1160.119.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 72s! [httpd:30646]
Modules linked in: binfmt_misc xt_multiport xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper sg cryptd joydev pcspkr virtio_balloon i2c_piix4 ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ata_generic
pata_acpi virtio_net virtio_scsi net_failover failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci floppy virtio_ring virtio drm_panel_orientation_quirks fuse
CPU: 2 PID: 30646 Comm: httpd Kdump: loaded Tainted: G             L ------------   3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
task: ffff9b2564a09080 ti: ffff9b2131930000 task.ti: ffff9b2131930000
RIP: 0010:[<ffffffffaffab7b5>]  [<ffffffffaffab7b5>] copy_page_rep+0x5/0x10
RSP: 0000:ffff9b2131933ca0  EFLAGS: 00010286
RAX: ffffeee04ef08540 RBX: ffff9b201d57b870 RCX: 0000000000000200
RDX: ffffffffb06902a8 RSI: ffff9b225bba3000 RDI: ffff9b23bc215000
RBP: ffff9b2131933d08 R08: ffffffffb0690297 R09: ffffeee04ef08580
R10: 0000000000004aac R11: ffff9b201d57b870 R12: 0000000000000000
R13: 00000000000200da R14: 000000005c3fc7d9 R15: ffff9b2131933c20
FS:  00007fc077051880(0000) GS:ffff9b27bfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000562314ad4398 CR3: 000000010a764000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffafdfb499>] ? wp_page_copy.isra.73+0x119/0x5d0
[<ffffffffafdfdbbb>] do_wp_page+0xfb/0x730
[<ffffffffafe022da>] handle_mm_fault+0xb3a/0x1190
[<ffffffffafe7ab0b>] ? destroy_inode+0x3b/0x70
[<ffffffffb03bf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb03bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffb03bf112>] do_async_page_fault+0x22/0x100
[<ffffffffb03bb7e8>] async_page_fault+0x28/0x30
Code: 3b 56 08 75 11 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 cc cc cc cc 9d 30 c0 c3 cc cc cc cc 90 90 90 90 90 90 90 90 b9 00 02 00 00 <f3> 48 a5 e9 43 2d 00 00 0f 1f 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #1055180
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff86d94955
Build id: 3.10.0-1160.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff86bef080
Build id: 3.10.0-1160.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff86bf16bb
Build id: 3.10.0-1160.el7.x86_64
0xfb
4
? try_to_wake_up
vmlinux 0xffffffff86ada830
Build id: 3.10.0-1160.el7.x86_64
0x190
5
handle_mm_fault
vmlinux 0xffffffff86bf5d32
Build id: 3.10.0-1160.el7.x86_64
0xb22
6
? futex_wake
vmlinux 0xffffffff86b12040
Build id: 3.10.0-1160.el7.x86_64
0x90
7
__do_page_fault
vmlinux 0xffffffff8718e653
Build id: 3.10.0-1160.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff8718ea26
Build id: 3.10.0-1160.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff8718dfa2
Build id: 3.10.0-1160.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff8718a7a8
Build id: 3.10.0-1160.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [pritunl:28738]
Modules linked in: xt_comment udp_diag xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) tun tcp_diag inet_diag snd_hda_codec_generic nfit snd_hda_intel libnvdimm snd_hda_codec snd_hda_core iosf_mbi snd_hwdep crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel ppdev snd_pcm parport_pc aesni_intel parport lrw snd_timer gf128mul glue_helper ablk_helper cryptd snd pcspkr joydev virtio_balloon soundcore i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_blk drm ata_piix 8139too libata crct10dif_pclmul crct10dif_common
8139cp serio_raw virtio_pci crc32c_intel virtio_ring mii virtio floppy drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 28738 Comm: pritunl Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9751034ae300 ti: ffff97522e650000 task.ti: ffff97522e650000
RIP: 0010:[<ffffffff86d94955>]  [<ffffffff86d94955>] copy_page_rep+0x5/0x10
RSP: 0000:ffff97522e653ca0  EFLAGS: 00010286
RAX: fffff9b486eed540 RBX: fffff9b486eed540 RCX: 0000000000000200
RDX: ffffffff8748e488 RSI: ffff9750251bc000 RDI: ffff9751bbb55000
RBP: ffff97522e653d08 R08: ffffffff8748e477 R09: fffff9b486eed580
R10: 0000000000002f04 R11: ffffffffffffffff R12: fffff9b480946f00
R13: 00007fcfe80033a0 R14: ffff9750239537a0 R15: ffff9751034ae300
FS:  00007fd01a7fc700(0000) GS:ffff9752bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000cb000000 CR3: 00000001753f2000 CR4: 00000000003606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff86bef080>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff86bf16bb>] do_wp_page+0xfb/0x720
[<ffffffff86ada830>] ? try_to_wake_up+0x190/0x390
[<ffffffff86bf5d32>] handle_mm_fault+0xb22/0xfb0
[<ffffffff86b12040>] ? futex_wake+0x90/0x180
[<ffffffff8718e653>] __do_page_fault+0x213/0x500
[<ffffffff8718ea26>] trace_do_page_fault+0x56/0x150
[<ffffffff8718dfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8718a7a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #1049736
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff86d94955
Build id: 3.10.0-1160.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff86bef080
Build id: 3.10.0-1160.el7.x86_64
0x120
3
? schedule
vmlinux 0xffffffff87187169
Build id: 3.10.0-1160.el7.x86_64
0x29
4
do_wp_page
vmlinux 0xffffffff86bf16bb
Build id: 3.10.0-1160.el7.x86_64
0xfb
5
handle_mm_fault
vmlinux 0xffffffff86bf5d32
Build id: 3.10.0-1160.el7.x86_64
0xb22
6
__do_page_fault
vmlinux 0xffffffff8718e653
Build id: 3.10.0-1160.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffff8718ea26
Build id: 3.10.0-1160.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffff8718dfa2
Build id: 3.10.0-1160.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffff8718a7a8
Build id: 3.10.0-1160.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 21s! [pritunl:15550]
Modules linked in: xt_comment udp_diag xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) tun tcp_diag inet_diag snd_hda_codec_generic nfit snd_hda_intel libnvdimm snd_hda_codec snd_hda_core iosf_mbi snd_hwdep crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel ppdev snd_pcm parport_pc aesni_intel parport lrw snd_timer gf128mul glue_helper ablk_helper cryptd snd pcspkr joydev virtio_balloon soundcore i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_blk drm ata_piix 8139too libata crct10dif_pclmul crct10dif_common
8139cp serio_raw virtio_pci crc32c_intel virtio_ring mii virtio floppy drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 15550 Comm: pritunl Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9751034fa100 ti: ffff97529c1c0000 task.ti: ffff97529c1c0000
RIP: 0010:[<ffffffff86d94955>]  [<ffffffff86d94955>] copy_page_rep+0x5/0x10
RSP: 0000:ffff97529c1c3ca0  EFLAGS: 00010286
RAX: fffff9b482298e40 RBX: fffff9b482298e40 RCX: 0000000000000200
RDX: ffffffff8748e488 RSI: ffff9750926d5000 RDI: ffff97508a639000
RBP: ffff97529c1c3d08 R08: ffffffff8748e477 R09: fffff9b482298e80
R10: 0000000000002e3d R11: ffffffffffffffff R12: fffff9b48249b540
R13: 00007f583cff7e38 R14: ffff975105f651b8 R15: ffff9751034fa100
FS:  00007f583cff9700(0000) GS:ffff9752bfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000f1b65006 CR3: 0000000100486000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff86bef080>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff87187169>] ? schedule+0x29/0x70
[<ffffffff86bf16bb>] do_wp_page+0xfb/0x720
[<ffffffff86bf5d32>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8718e653>] __do_page_fault+0x213/0x500
[<ffffffff8718ea26>] trace_do_page_fault+0x56/0x150
[<ffffffff8718dfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8718a7a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #954046
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffffb69ab775
Build id: 3.10.0-1160.102.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffffb67fb449
Build id: 3.10.0-1160.102.1.el7.x86_64
0x119
3
do_wp_page
vmlinux 0xffffffffb67fdb6b
Build id: 3.10.0-1160.102.1.el7.x86_64
0xfb
4
? ___sys_recvmsg
vmlinux 0xffffffffb6c5f803
Build id: 3.10.0-1160.102.1.el7.x86_64
0x133
5
handle_mm_fault
vmlinux 0xffffffffb680228a
Build id: 3.10.0-1160.102.1.el7.x86_64
0xb3a
6
? wake_up_state
vmlinux 0xffffffffb66e1170
Build id: 3.10.0-1160.102.1.el7.x86_64
0x20
7
__do_page_fault
vmlinux 0xffffffffb6dbf7d3
Build id: 3.10.0-1160.102.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffffb6dbfbb6
Build id: 3.10.0-1160.102.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffffb6dbf112
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffffb6dbb7e8
Build id: 3.10.0-1160.102.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [QXcbEventQueue:5032]
Modules linked in: tcp_diag inet_diag tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr virtio_balloon i2c_piix4 auth_rpcgss sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_scsi virtio_net net_failover failover ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul libata
virtio_pci crct10dif_common crc32c_intel virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 5032 Comm: QXcbEventQueue Kdump: loaded Tainted: G             L ------------   3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014
task: ffff9c3666b1a100 ti: ffff9b035bf4c000 task.ti: ffff9b035bf4c000
RIP: 0010:[<ffffffffb69ab775>]  [<ffffffffb69ab775>] copy_page_rep+0x5/0x10
RSP: 0000:ffff9b035bf4fca0  EFLAGS: 00010286
RAX: fffff234bb4b5040 RBX: ffff9c1d68afeaf8 RCX: 0000000000000200
RDX: ffffffffb708ffa8 RSI: ffff9ada92ffd000 RDI: ffff9ad852d41000
RBP: ffff9b035bf4fd08 R08: ffffffffb708ff97 R09: fffff234bb4b5080
R10: 000000000000dae8 R11: ffff9c1d68afeaf8 R12: 0000000000000000
R13: 00000000000200da R14: 00000000551d6e80 R15: ffff9b035bf4fc50
FS:  00007ffb54b3c700(0000) GS:ffff9c3a64b40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffb5ba3daf0 CR3: 000001c16f5ea000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb67fb449>] ? wp_page_copy.isra.73+0x119/0x5d0
[<ffffffffb67fdb6b>] do_wp_page+0xfb/0x730
[<ffffffffb6c5f803>] ? ___sys_recvmsg+0x133/0x2d0
[<ffffffffb680228a>] handle_mm_fault+0xb3a/0x1190
[<ffffffffb66e1170>] ? wake_up_state+0x20/0x20
[<ffffffffb6dbf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb6dbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffb6dbf112>] do_async_page_fault+0x22/0x100
[<ffffffffb6dbb7e8>] async_page_fault+0x28/0x30
Code: 3b 56 08 75 11 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 cc cc cc cc 9d 30 c0 c3 cc cc cc cc 90 90 90 90 90 90 90 90 b9 00 02 00 00 <f3> 48 a5 e9 43 2d 00 00 0f 1f 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #924750
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffffb69ab775
Build id: 3.10.0-1160.102.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffffb67fb449
Build id: 3.10.0-1160.102.1.el7.x86_64
0x119
3
do_wp_page
vmlinux 0xffffffffb67fdb6b
Build id: 3.10.0-1160.102.1.el7.x86_64
0xfb
4
handle_mm_fault
vmlinux 0xffffffffb680228a
Build id: 3.10.0-1160.102.1.el7.x86_64
0xb3a
5
? cpumask_next_and
vmlinux 0xffffffffb699c5d5
Build id: 3.10.0-1160.102.1.el7.x86_64
0x35
6
__do_page_fault
vmlinux 0xffffffffb6dbf7d3
Build id: 3.10.0-1160.102.1.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffffb6dbfbb6
Build id: 3.10.0-1160.102.1.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffffb6dbf112
Build id: 3.10.0-1160.102.1.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffffb6dbb7e8
Build id: 3.10.0-1160.102.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [sh:1227]
Modules linked in: tcp_diag inet_diag tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter nfsv3 nfs_acl nfs lockd grace fscache kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev sg pcspkr virtio_balloon i2c_piix4 auth_rpcgss sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_scsi virtio_net net_failover failover ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul libata
virtio_pci crct10dif_common crc32c_intel virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 8 PID: 1227 Comm: sh Kdump: loaded Tainted: G             L ------------   3.10.0-1160.102.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014
task: ffff9b5a0c110000 ti: ffff9c1646fe0000 task.ti: ffff9c1646fe0000
RIP: 0010:[<ffffffffb69ab775>]  [<ffffffffb69ab775>] copy_page_rep+0x5/0x10
RSP: 0000:ffff9c1646fe3ca0  EFLAGS: 00010286
RAX: fffff232e188d480 RBX: ffff9bf38a6add88 RCX: 0000000000000200
RDX: ffffffffb708ffa8 RSI: ffff9bb7be94c000 RDI: ffff9a61e2352000
RBP: ffff9c1646fe3d08 R08: ffffffffb708ff97 R09: fffff232e188d4c0
R10: 000000000000dae8 R11: ffff9bf38a6add88 R12: 0000000000000000
R13: 00000000000200da R14: 00000000051a52cd R15: ffff9c1646fe3c50
FS:  00007fde0ef27740(0000) GS:ffff9c3a64c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fde0e8d8aa0 CR3: 0000018b94f3e000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb67fb449>] ? wp_page_copy.isra.73+0x119/0x5d0
[<ffffffffb67fdb6b>] do_wp_page+0xfb/0x730
[<ffffffffb680228a>] handle_mm_fault+0xb3a/0x1190
[<ffffffffb699c5d5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb6dbf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb6dbfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffb6dbf112>] do_async_page_fault+0x22/0x100
[<ffffffffb6dbb7e8>] async_page_fault+0x28/0x30
Code: 3b 56 08 75 11 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 cc cc cc cc 9d 30 c0 c3 cc cc cc cc 90 90 90 90 90 90 90 90 b9 00 02 00 00 <f3> 48 a5 e9 43 2d 00 00 0f 1f 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #590694
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff81d903d5
Build id: 3.10.0-1062.18.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff81bed000
Build id: 3.10.0-1062.18.1.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff81bef63b
Build id: 3.10.0-1062.18.1.el7.x86_64
0xfb
4
? sock_aio_read.part.9
vmlinux 0xffffffff8202e6ec
Build id: 3.10.0-1062.18.1.el7.x86_64
0x14c
5
handle_mm_fault
vmlinux 0xffffffff81bf3cd2
Build id: 3.10.0-1062.18.1.el7.x86_64
0xb22
6
__do_page_fault
vmlinux 0xffffffff82188653
Build id: 3.10.0-1062.18.1.el7.x86_64
0x213
7
trace_do_page_fault
vmlinux 0xffffffff82188a26
Build id: 3.10.0-1062.18.1.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffff82187fa2
Build id: 3.10.0-1062.18.1.el7.x86_64
0x22
9
async_page_fault
vmlinux 0xffffffff821847a8
Build id: 3.10.0-1062.18.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 25s! [ofork.Daemon.pl:20613]
Modules linked in: nfsv3 nfs_acl nfs lockd grace fscache sunrpc ext4 dm_mirror dm_region_hash dm_log dm_mod mbcache jbd2 snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device iosf_mbi ppdev snd_pcm crc32_pclmul ghash_clmulni_intel aesni_intel snd_timer lrw gf128mul snd glue_helper ablk_helper cryptd sg virtio_rng soundcore pcspkr i2c_piix4 joydev virtio_balloon parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console qxl virtio_net virtio_blk drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix serio_raw libata virtio_pci virtio_ring virtio floppy drm_panel_orientation_quirks
CPU: 0 PID: 20613 Comm: ofork.Daemon.pl Kdump: loaded Not tainted 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff9d8c7bbea0e0 ti: ffff9d8d5270c000 task.ti: ffff9d8d5270c000
RIP: 0010:[<ffffffff81d903d5>]  [<ffffffff81d903d5>] copy_page_rep+0x5/0x10
RSP: 0000:ffff9d8d5270fca0  EFLAGS: 00010286
RAX: ffffdd0c87e30f80 RBX: ffffdd0c87e30f80 RCX: 0000000000000200
RDX: ffffffff8248b210 RSI: ffff9d8d59a66000 RDI: ffff9d8db8c3e000
RBP: ffff9d8d5270fd08 R08: ffffffff8248b1ff R09: ffffdd0c87e30fc0
R10: 0000000000003476 R11: ffff9d8bdecbc5e8 R12: ffffdd0c86669980
R13: 000000000307fd98 R14: ffff9d8bdecbc5e8 R15: ffff9d8c7bbea0e0
FS:  00007f7de23f0740(0000) GS:ffff9d8dffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000d3000000 CR3: 000000012b0aa000 CR4: 00000000003606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff81bed000>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff81bef63b>] do_wp_page+0xfb/0x720
[<ffffffff8202e6ec>] ? sock_aio_read.part.9+0x14c/0x170
[<ffffffff81bf3cd2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff82188653>] __do_page_fault+0x213/0x500
[<ffffffff82188a26>] trace_do_page_fault+0x56/0x150
[<ffffffff82187fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff821847a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #566286
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff95b94e35
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff959ef200
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff959f183b
Build id: 3.10.0-1160.15.2.el7.x86_64
0xfb
4
handle_mm_fault
vmlinux 0xffffffff959f5eb2
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb22
5
__do_page_fault
vmlinux 0xffffffff95f8f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
6
trace_do_page_fault
vmlinux 0xffffffff95f8fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
7
do_async_page_fault
vmlinux 0xffffffff95f8efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
8
async_page_fault
vmlinux 0xffffffff95f8b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#133 stuck for 26s! [python:136110]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14306(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core qxl snd_hwdep ttm drm_kms_helper snd_seq syscopyarea snd_seq_device sysfillrect snd_pcm sysimgblt fb_sys_fops snd_timer
ppdev iTCO_wdt iTCO_vendor_support crc32_pclmul ghash_clmulni_intel snd drm aesni_intel parport_pc lrw gf128mul glue_helper ablk_helper joydev pcspkr nfsd lpc_ich virtio_balloon soundcore drm_panel_orientation_quirks i2c_i801 virtio_rng cryptd parport nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net net_failover virtio_console virtio_blk failover ahci libahci crct10dif_pclmul virtio_pci crct10dif_common virtio_ring crc32c_intel libata serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 133 PID: 136110 Comm: python Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff922f877a0000 ti: ffff922bcd39c000 task.ti: ffff922bcd39c000
RIP: 0010:[<ffffffff95b94e35>]  [<ffffffff95b94e35>] copy_page_rep+0x5/0x10
RSP: 0000:ffff922bcd39fca0  EFLAGS: 00000286
RAX: fffff90658eff040 RBX: ffff922c41f9fb00 RCX: 0000000000000200
RDX: ffffffff9628e578 RSI: ffff92c371167000 RDI: ffff92b93bfc1000
RBP: ffff922bcd39fd08 R08: ffffffff9628e567 R09: fffff90658eff080
R10: ffff94177faaf6e0 R11: ffff922c41f9fb00 R12: 0000000000000000
R13: 00000000000200da R14: 00000000faf9ff03 R15: ffff922bcd39fc50
FS:  00007f4cf964d740(0000) GS:ffff940fad740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4904ea0078 CR3: 00000008cd39a000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffff959ef200>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff959f183b>] do_wp_page+0xfb/0x720
[<ffffffff959f5eb2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff95f8f653>] __do_page_fault+0x213/0x500
[<ffffffff95f8fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff95f8efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff95f8b7a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 66 66 66 90 66 66 66 90 
Complete report #299001
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff96194e35
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff95fef200
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff95ff183b
Build id: 3.10.0-1160.15.2.el7.x86_64
0xfb
4
? unlock_page
vmlinux 0xffffffff95fbe15b
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2b
5
? do_read_fault.isra.63
vmlinux 0xffffffff95fee659
Build id: 3.10.0-1160.15.2.el7.x86_64
0x139
6
handle_mm_fault
vmlinux 0xffffffff95ff5eb2
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb22
7
? xfs_iunlock
xfs 0xffffffffc0317389
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa9
8
__do_page_fault
vmlinux 0xffffffff9658f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffff9658fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff9658efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffff9658b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 135s! [lfd - (child) c:28320]
Modules linked in: ip_set nfnetlink nf_log_ipv6 nf_log_ipv4 nf_log_common nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_LOG xt_limit iptable_nat nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_nat_ipv6 nf_nat iptable_mangle iptable_raw ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables nf_conntrack_ftp ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_multiport xt_conntrack nf_conntrack iptable_filter ext4 mbcache jbd2 sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper sg cryptd joydev pcspkr virtio_balloon parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc fuse ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover
virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata virtio_pci crct10dif_pclmul crct10dif_common virtio_ring crc32c_intel drm_panel_orientation_quirks virtio serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 28320 Comm: lfd - (child) c Kdump: loaded Tainted: G             L ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
task: ffff8cd2afff3180 ti: ffff8cdad60d8000 task.ti: ffff8cdad60d8000
RIP: 0010:[<ffffffff96194e35>]  [<ffffffff96194e35>] copy_page_rep+0x5/0x10
RSP: 0000:ffff8cdad60dbca0  EFLAGS: 00010286
RAX: ffffe6da6e3df5c0 RBX: ffff8cdabda14d80 RCX: 0000000000000200
RDX: ffffffff9688e578 RSI: ffff8cdc85fb8000 RDI: ffff8cdd0f7d7000
RBP: ffff8cdad60dbd08 R08: ffffffff9688e567 R09: ffffe6da6e3df600
R10: 0000000000004dd2 R11: ffff8cdabda14d80 R12: 0000000000000000
R13: 00000000000200da R14: 000000000dddb5d5 R15: ffffffff95fcd00c
FS:  00007fc902321740(0000) GS:ffff8cde0d100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001e9e4b0 CR3: 000000092a33a000 CR4: 0000000000360fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff95fef200>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff95ff183b>] do_wp_page+0xfb/0x720
[<ffffffff95fbe15b>] ? unlock_page+0x2b/0x30
[<ffffffff95fee659>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff95ff5eb2>] handle_mm_fault+0xb22/0xfb0
[<ffffffffc0317389>] ? xfs_iunlock+0xa9/0x120 [xfs]
[<ffffffff9658f653>] __do_page_fault+0x213/0x500
[<ffffffff9658fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff9658efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9658b7a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #242730
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff82b94b55
Build id: 3.10.0-1160.11.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff829ef260
Build id: 3.10.0-1160.11.1.el7.x86_64
0x120
3
? path_lookupat
vmlinux 0xffffffff82a5cb2a
Build id: 3.10.0-1160.11.1.el7.x86_64
0x1ea
4
do_wp_page
vmlinux 0xffffffff829f189b
Build id: 3.10.0-1160.11.1.el7.x86_64
0xfb
5
? putname
vmlinux 0xffffffff82a5fd3d
Build id: 3.10.0-1160.11.1.el7.x86_64
0x3d
6
handle_mm_fault
vmlinux 0xffffffff829f5f12
Build id: 3.10.0-1160.11.1.el7.x86_64
0xb22
7
__do_page_fault
vmlinux 0xffffffff82f8e653
Build id: 3.10.0-1160.11.1.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff82f8ea26
Build id: 3.10.0-1160.11.1.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff82f8dfa2
Build id: 3.10.0-1160.11.1.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff82f8a7a8
Build id: 3.10.0-1160.11.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 71s! [miniserv.pl:17153]
Modules linked in: xt_multiport xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables tcp_lp tun devlink bridge stp llc libcrc32c ip_set nfnetlink sunrpc dm_mirror dm_region_hash dm_log dm_mod squashfs loop sb_edac iosf_mbi crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg virtio_balloon pcspkr joydev parport_pc
parport i2c_piix4 ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover virtio_scsi failover vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata crc32c_intel virtio_pci serio_raw virtio_ring drm_panel_orientation_quirks virtio floppy fuse [last unloaded: ip_tables]
CPU: 2 PID: 17153 Comm: miniserv.pl Kdump: loaded Tainted: G             L ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
task: ffff931a3518c200 ti: ffff931c3552c000 task.ti: ffff931c3552c000
RIP: 0010:[<ffffffff82b94b55>]  [<ffffffff82b94b55>] copy_page_rep+0x5/0x10
RSP: 0000:ffff931c3552fca0  EFLAGS: 00010286
RAX: ffffd4f888828f00 RBX: ffff931b70bfd438 RCX: 0000000000000200
RDX: ffffffff8328e588 RSI: ffff931aade7a000 RDI: ffff931c20a3c000
RBP: ffff931c3552fd08 R08: ffffffff8328e577 R09: ffffd4f888828f40
R10: 00000000000034a8 R11: ffff931b70bfd438 R12: 0000000000000000
R13: 00000000000200da R14: 0000000047ca5080 R15: ffffffff82a59b38
FS:  00007f5a89c3f740(0000) GS:ffff931c3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000100be60 CR3: 0000000236e3a000 CR4: 0000000000360fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff829ef260>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff82a5cb2a>] ? path_lookupat+0x1ea/0x8d0
[<ffffffff829f189b>] do_wp_page+0xfb/0x720
[<ffffffff82a5fd3d>] ? putname+0x3d/0x60
[<ffffffff829f5f12>] handle_mm_fault+0xb22/0xfb0
[<ffffffff82f8e653>] __do_page_fault+0x213/0x500
[<ffffffff82f8ea26>] trace_do_page_fault+0x56/0x150
[<ffffffff82f8dfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff82f8a7a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #233849
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff8b98e875
Build id: 3.10.0-1062.4.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff8b7ebad0
Build id: 3.10.0-1062.4.1.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff8b7ee10b
Build id: 3.10.0-1062.4.1.el7.x86_64
0xfb
4
? unlock_page
vmlinux 0xffffffff8b7bc0cb
Build id: 3.10.0-1062.4.1.el7.x86_64
0x2b
5
? do_read_fault.isra.63
vmlinux 0xffffffff8b7eaf29
Build id: 3.10.0-1062.4.1.el7.x86_64
0x139
6
handle_pte_fault
vmlinux 0xffffffff8b7efa95
Build id: 3.10.0-1062.4.1.el7.x86_64
0x3d5
7
handle_mm_fault
vmlinux 0xffffffff8b7f25fd
Build id: 3.10.0-1062.4.1.el7.x86_64
0x39d
8
__do_page_fault
vmlinux 0xffffffff8bd86653
Build id: 3.10.0-1062.4.1.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffff8bd86a26
Build id: 3.10.0-1062.4.1.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff8bd85fa2
Build id: 3.10.0-1062.4.1.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffff8bd827a8
Build id: 3.10.0-1062.4.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 70s! [zabbix_agentd:28096]
Modules linked in: ip_set_hash_ip nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ppdev iosf_mbi snd_hda_codec_generic crc32_pclmul ghash_clmulni_intel snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep aesni_intel snd_seq lrw gf128mul glue_helper ablk_helper snd_seq_device cryptd pcspkr snd_pcm joydev sg snd_timer snd virtio_balloon soundcore i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk qxl virtio_console virtio_net drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash
dm_log dm_mod
CPU: 1 PID: 28096 Comm: zabbix_agentd Not tainted 3.10.0-1062.4.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff98cf363b8000 ti: ffff98ce11e40000 task.ti: ffff98ce11e40000
RIP: 0010:[<ffffffff8b98e875>]  [<ffffffff8b98e875>] copy_page_rep+0x5/0x10
RSP: 0000:ffff98ce11e43c18  EFLAGS: 00010286
RAX: fffffe8180966200 RBX: fffffe8180966200 RCX: 0000000000000200
RDX: ffffffff8c08a9f8 RSI: ffff98cf12206000 RDI: ffff98ce25988000
RBP: ffff98ce11e43c80 R08: ffffffff8c08a9e7 R09: fffffe8180966240
R10: 00000000000041e1 R11: 00003ffffffff000 R12: fffffe8184488180
R13: 00000000006440d8 R14: ffff98ce0fd1d6c0 R15: ffff98cf363b8000
FS:  00007fa6909a4840(0000) GS:ffff98cf3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000ddc90001 CR3: 000000000fc26000 CR4: 00000000000406e0
Call Trace:
[<ffffffff8b7ebad0>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff8b7ee10b>] do_wp_page+0xfb/0x720
[<ffffffff8b7bc0cb>] ? unlock_page+0x2b/0x30
[<ffffffff8b7eaf29>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff8b7efa95>] handle_pte_fault+0x3d5/0xe20
[<ffffffff8b7f25fd>] handle_mm_fault+0x39d/0x9b0
[<ffffffff8bd86653>] __do_page_fault+0x213/0x500
[<ffffffff8bd86a26>] trace_do_page_fault+0x56/0x150
[<ffffffff8bd85fa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8bd827a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 66 66 66 90 66 66 66 90 
Complete report #220359
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff81325ca5
Build id: 3.10.0-514.26.2.el7.x86_64
0x5
2
? wp_page_copy.isra.56
vmlinux 0xffffffff811ad53b
Build id: 3.10.0-514.26.2.el7.x86_64
0xeb
3
do_wp_page
vmlinux 0xffffffff811af245
Build id: 3.10.0-514.26.2.el7.x86_64
0xe5
4
? __enqueue_entity
vmlinux 0xffffffff810c9bf8
Build id: 3.10.0-514.26.2.el7.x86_64
0x78
5
? enqueue_entity
vmlinux 0xffffffff810d065c
Build id: 3.10.0-514.26.2.el7.x86_64
0x26c
6
handle_mm_fault
vmlinux 0xffffffff811b14dc
Build id: 3.10.0-514.26.2.el7.x86_64
0x72c
7
__do_page_fault
vmlinux 0xffffffff81692cc4
Build id: 3.10.0-514.26.2.el7.x86_64
0x154
8
? __switch_to
vmlinux 0xffffffff810295ec
Build id: 3.10.0-514.26.2.el7.x86_64
0x15c
9
trace_do_page_fault
vmlinux 0xffffffff816930a6
Build id: 3.10.0-514.26.2.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff8169274b
Build id: 3.10.0-514.26.2.el7.x86_64
0x1b
11
async_page_fault
vmlinux 0xffffffff8168f238
Build id: 3.10.0-514.26.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [crond:684]
Modules linked in: arc4 md4 nls_utf8 cifs dns_resolver snd_hda_codec_generic crc32_pclmul ghash_clmulni_intel snd_hda_intel snd_hda_codec aesni_intel lrw gf128mul snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm glue_helper virtio_balloon pcspkr snd_timer ppdev ablk_helper snd cryptd soundcore i2c_piix4 parport_pc parport ip_tables xfs libcrc32c ata_generic pata_acpi qxl drm_kms_helper virtio_blk virtio_net virtio_console syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_pci drm ata_piix crct10dif_pclmul crct10dif_common virtio_ring crc32c_intel floppy libata i2c_core serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 684 Comm: crond Tainted: G             L ------------   3.10.0-514.26.2.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff880036a69f60 ti: ffff88005b1f4000 task.ti: ffff88005b1f4000
RIP: 0010:[<ffffffff81325ca5>]  [<ffffffff81325ca5>] copy_page_rep+0x5/0x10
RSP: 0000:ffff88005b1f7d18  EFLAGS: 00010286
RAX: 000000002d995000 RBX: ffffea0000b66540 RCX: 0000000000000200
RDX: 0000000000000000 RSI: ffff88003f0c1000 RDI: ffff88002d995000
RBP: ffff88005b1f7d80 R08: ffffffff818e0a6f R09: ffffea0000b66580
R10: 000000000000366c R11: 800000003f0c1865 R12: ffffea0000fc3040
R13: ffff88005b1f4000 R14: ffff88005b7cc5e8 R15: ffff88005b7e1e08
FS:  00007f5801163840(0000) GS:ffff88005fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000027c8f001 CR3: 000000005b01c000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811ad53b 00007f58006ec000 ffff880036840640 ffffea00016dfd70
800000003f0c1865 00007ffe382af000 00007ffe382af028 ffff880036840640
ffff88005b7cc5e8 00007ffe382af028 ffffea00016c03b0 ffffea0000fc3040
Call Trace:
[<ffffffff811ad53b>] ? wp_page_copy.isra.56+0xeb/0x470
[<ffffffff811af245>] do_wp_page+0xe5/0x530
[<ffffffff810c9bf8>] ? __enqueue_entity+0x78/0x80
[<ffffffff810d065c>] ? enqueue_entity+0x26c/0xb60
[<ffffffff811b14dc>] handle_mm_fault+0x72c/0x1000
[<ffffffff81692cc4>] __do_page_fault+0x154/0x450
[<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[<ffffffff816930a6>] trace_do_page_fault+0x56/0x150
[<ffffffff8169274b>] do_async_page_fault+0x1b/0xd0
[<ffffffff8168f238>] async_page_fault+0x28/0x30
Code: b7 90 90 90 90 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 66 66 66 90 66 66 66 90 
Complete report #219043
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff86f94e35
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff86def200
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff86df183b
Build id: 3.10.0-1160.15.2.el7.x86_64
0xfb
4
? unlock_page
vmlinux 0xffffffff86dbe15b
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2b
5
? do_read_fault.isra.63
vmlinux 0xffffffff86dee659
Build id: 3.10.0-1160.15.2.el7.x86_64
0x139
6
handle_mm_fault
vmlinux 0xffffffff86df5eb2
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb22
7
__do_page_fault
vmlinux 0xffffffff8738f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff8738fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff8738efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff8738b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#30 stuck for 23s! [bash:22932]
Modules linked in: fidelis(OE) veth tcp_diag inet_diag nfnetlink_queue nfnetlink_log vxlan ip6_udp_tunnel udp_tunnel xt_nat xt_statistic ipt_REJECT nf_reject_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) sunrpc ppdev sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel parport_pc joydev aesni_intel virtio_balloon parport i2c_piix4 lrw gf128mul glue_helper ablk_helper pcspkr cryptd ip_tables xfs libcrc32c ata_generic pata_acpi virtio_net net_failover virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul virtio_pci crct10dif_common crc32c_intel virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: fidelis]
CPU: 30 PID: 22932 Comm: bash Kdump: loaded Tainted: G           OEL ------------ T 3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: OpenStack Foundation OpenStack Nova, BIOS 1.10.2-1.1~u16.04+mcp2 04/01/2014
task: ffffa0d029835280 ti: ffffa0d5031ac000 task.ti: ffffa0d5031ac000
RIP: 0010:[<ffffffff86f94e35>]  [<ffffffff86f94e35>] copy_page_rep+0x5/0x10
RSP: 0000:ffffa0d5031afca0  EFLAGS: 00010286
RAX: fffff00b8c4ba1c0 RBX: ffffa0b9a7392870 RCX: 0000000000000200
RDX: ffffffff8768e578 RSI: ffffa0d6ced4e000 RDI: ffffa0cad2e87000
RBP: ffffa0d5031afd08 R08: ffffffff8768e567 R09: fffff00b8c4ba200
R10: ffffa0d7ffb7be20 R11: ffffa0b9a7392870 R12: 0000000000000000
R13: 00000000000200da R14: 000000008349abdf R15: ffffffff86dc0bdd
FS:  0000000000000000(0000) GS:ffffa0d77f580000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8b4c0c7bc0 CR3: 0000000147bbc000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff86def200>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff86df183b>] do_wp_page+0xfb/0x720
[<ffffffff86dbe15b>] ? unlock_page+0x2b/0x30
[<ffffffff86dee659>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff86df5eb2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8738f653>] __do_page_fault+0x213/0x500
[<ffffffff8738fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff8738efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8738b7a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #164343
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff98994e35
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff987ef200
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff987f183b
Build id: 3.10.0-1160.15.2.el7.x86_64
0xfb
4
? unlock_page
vmlinux 0xffffffff987be15b
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2b
5
? do_read_fault.isra.63
vmlinux 0xffffffff987ee659
Build id: 3.10.0-1160.15.2.el7.x86_64
0x139
6
handle_mm_fault
vmlinux 0xffffffff987f5eb2
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb22
7
__do_page_fault
vmlinux 0xffffffff98d8f653
Build id: 3.10.0-1160.15.2.el7.x86_64
0x213
8
trace_do_page_fault
vmlinux 0xffffffff98d8fa26
Build id: 3.10.0-1160.15.2.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff98d8efa2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
10
async_page_fault
vmlinux 0xffffffff98d8b7a8
Build id: 3.10.0-1160.15.2.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [rpm:14282]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [abrt-action-gen:14418]
Modules linked in: binfmt_misc ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables nf_conntrack_ftp ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c iptable_filter dm_mirror dm_region_hash dm_log dm_mod ppdev parport_pc parport joydev pcspkr virtio_balloon i2c_piix4 ip_tables ext4 mbcache jbd2 ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_net net_failover virtio_blk failover ata_piix drm libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy
CPU: 1 PID: 14418 Comm: abrt-action-gen Tainted: G             L ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffffa142569e4200 ti: ffffa14246c40000 task.ti: ffffa14246c40000
RIP: 0010:[<ffffffff98994e35>]  [<ffffffff98994e35>] copy_page_rep+0x5/0x10
RSP: 0000:ffffa14246c43ca0  EFLAGS: 00010286
RAX: ffffe00c00f25180 RBX: ffffa1424f5c35e8 RCX: 0000000000000200
RDX: ffffffff9908e578 RSI: ffffa142ad6d7000 RDI: ffffa1427c946000
RBP: ffffa14246c43d08 R08: ffffffff9908e567 R09: ffffe00c00f251c0
R10: 0000000000003688 R11: ffffa1424f5c35e8 R12: 0000000000000000
R13: 00000000000200da R14: 00000000a1d8e31d R15: ffffa14246c43c50
FS:  00007f6796e87740(0000) GS:ffffa142bfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6796c69050 CR3: 00000000062f6000 CR4: 00000000000006e0
Call Trace:
[<ffffffff987ef200>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff987f183b>] do_wp_page+0xfb/0x720
[<ffffffff987be15b>] ? unlock_page+0x2b/0x30
[<ffffffff987ee659>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff987f5eb2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff98d8f653>] __do_page_fault+0x213/0x500
[<ffffffff98d8fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff98d8efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff98d8b7a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #162989
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff812ffe75
Build id: 3.10.0-327.el7.x86_64
0x5
2
? do_wp_page
vmlinux 0xffffffff8119490a
Build id: 3.10.0-327.el7.x86_64
0x13a
3
handle_mm_fault
vmlinux 0xffffffff8119712c
Build id: 3.10.0-327.el7.x86_64
0x65c
4
__do_page_fault
vmlinux 0xffffffff81640e22
Build id: 3.10.0-327.el7.x86_64
0x152
5
trace_do_page_fault
vmlinux 0xffffffff816411b3
Build id: 3.10.0-327.el7.x86_64
0x43
6
do_async_page_fault
vmlinux 0xffffffff816408d9
Build id: 3.10.0-327.el7.x86_64
0x29
7
async_page_fault
vmlinux 0xffffffff8163d438
Build id: 3.10.0-327.el7.x86_64
0x28
BUG: soft lockup - CPU#2 stuck for 23s! [zabbix_agentd:5731]
Modules linked in: fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd soundcore i2c_piix4 sg virtio_balloon ppdev pcspkr parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper ttm virtio_net virtio_blk drm floppy ata_piix serio_raw virtio_pci libata virtio_ring virtio i2c_core
CPU: 2 PID: 5731 Comm: zabbix_agentd Tainted: G             L ------------   3.10.0-327.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8800b2e5a280 ti: ffff880111558000 task.ti: ffff880111558000
RIP: 0010:[<ffffffff812ffe75>]  [<ffffffff812ffe75>] copy_page_rep+0x5/0x10
RSP: 0000:ffff88011155bd88  EFLAGS: 00010286
RAX: 00000001b6d42000 RBX: ffffffff81e99780 RCX: 0000000000000200
RDX: ffff8800b2e5a280 RSI: ffff8801ff08d000 RDI: ffff8801b6d42000
RBP: ffff88011155be10 R08: ffffffff8187797f R09: ffffea0006db50c0
R10: 0000000000002f3b R11: 80000001ff08d045 R12: ffff8800b2e5a280
R13: 0000000027cc798f R14: ffff880212070900 R15: 0000000000000014
FS:  00007fbcfc442840(0000) GS:ffff88021fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007ffe021fa878 CR3: 00000001073db000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff8119490a ffff880107202540 ffffea00041c80b0 0000000000000000
0000000000000200 ffff8801015b8080 ffff8800ce4b0000 ffff8800000000a8
0000000000000014 00007fbcfaea8000 ffffea00036fb480 0000000027cc798f
Call Trace:
[<ffffffff8119490a>] ? do_wp_page+0x13a/0x800
[<ffffffff8119712c>] handle_mm_fault+0x65c/0xf50
[<ffffffff81640e22>] __do_page_fault+0x152/0x420
[<ffffffff816411b3>] trace_do_page_fault+0x43/0x110
[<ffffffff816408d9>] do_async_page_fault+0x29/0xe0
[<ffffffff8163d438>] async_page_fault+0x28/0x30
Code: b7 90 90 90 90 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #130305
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff8132ffa5
Build id: 3.10.0-693.11.1.el7.x86_64
0x5
2
? wp_page_copy.isra.58
vmlinux 0xffffffff811addfe
Build id: 3.10.0-693.11.1.el7.x86_64
0xee
3
do_wp_page
vmlinux 0xffffffff811afb8b
Build id: 3.10.0-693.11.1.el7.x86_64
0xeb
4
? do_read_fault.isra.44
vmlinux 0xffffffff811ad616
Build id: 3.10.0-693.11.1.el7.x86_64
0xe6
5
handle_mm_fault
vmlinux 0xffffffff811b1efc
Build id: 3.10.0-693.11.1.el7.x86_64
0x70c
6
__do_page_fault
vmlinux 0xffffffff816b0074
Build id: 3.10.0-693.11.1.el7.x86_64
0x154
7
trace_do_page_fault
vmlinux 0xffffffff816b0456
Build id: 3.10.0-693.11.1.el7.x86_64
0x56
8
do_async_page_fault
vmlinux 0xffffffff816afaea
Build id: 3.10.0-693.11.1.el7.x86_64
0x1a
9
async_page_fault
vmlinux 0xffffffff816ac5f8
Build id: 3.10.0-693.11.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 25s! [sshd:14765]
Modules linked in: binfmt_misc tcp_lp fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun bridge ebtable_filter ebtables ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ppdev sb_edac edac_core iosf_mbi pcspkr virtio_balloon parport_pc joydev parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace ip_tables xfs libcrc32c ata_generic pata_acpi 8021q garp stp llc mrp virtio_blk virtio_net virtio_console qxl drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect sysimgblt fb_sys_fops crc32_pclmul ttm crc32c_intel drm ghash_clmulni_intel scsi_transport_iscsi ata_piix libata aesni_intel lrw gf128mul glue_helper ablk_helper cryptd virtio_pci
virtio_ring serio_raw floppy i2c_core virtio sunrpc dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 14765 Comm: sshd Tainted: G             L ------------   3.10.0-693.11.1.el7.x86_64 #1
Hardware name: OpenStack Foundation OpenStack Nova, BIOS 1.10.2-1ubuntu1 04/01/2014
task: ffff8800b1d3bf40 ti: ffff88041a208000 task.ti: ffff88041a208000
RIP: 0010:[<ffffffff8132ffa5>]  [<ffffffff8132ffa5>] copy_page_rep+0x5/0x10
RSP: 0000:ffff88041a20bd10  EFLAGS: 00010286
RAX: ffffea0010260c00 RBX: ffffea0010260c00 RCX: 0000000000000200
RDX: 0000000000000000 RSI: ffff8804189ce000 RDI: ffff880409830000
RBP: ffff88041a20bd78 R08: ffffffff8191118f R09: ffffea0010260c40
R10: ffffea0010260c40 R11: 80000004189ce025 R12: ffffea0010627380
R13: ffff88041a208000 R14: ffff8803e46ea948 R15: ffff880409232ee8
FS:  00007f1e42e738c0(0000) GS:ffff88043fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000181a1003 CR3: 0000000424f46000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff811addfe 0000000000000000 ffff88041a20bdc0 ffff8803e46ea948
80000004189ce025 00007f1e3ba6a000 00007f1e3ba6aee0 ffff88041c2ff080
ffff8803e46ea948 00007f1e3ba6aee0 ffffea000f66fdf0 ffff880409232ee8
Call Trace:
[<ffffffff811addfe>] ? wp_page_copy.isra.58+0xee/0x470
[<ffffffff811afb8b>] do_wp_page+0xeb/0x5c0
[<ffffffff811ad616>] ? do_read_fault.isra.44+0xe6/0x130
[<ffffffff811b1efc>] handle_mm_fault+0x70c/0xfa0
[<ffffffff816b0074>] __do_page_fault+0x154/0x450
[<ffffffff816b0456>] trace_do_page_fault+0x56/0x150
[<ffffffff816afaea>] do_async_page_fault+0x1a/0xd0
[<ffffffff816ac5f8>] async_page_fault+0x28/0x30
Code: b7 90 90 90 90 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #53383
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff86593b75
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff863ef1b0
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff863f17eb
Build id: 3.10.0-1127.19.1.el7.x86_64
0xfb
4
handle_mm_fault
vmlinux 0xffffffff863f5e82
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb22
5
__do_page_fault
vmlinux 0xffffffff8698d653
Build id: 3.10.0-1127.19.1.el7.x86_64
0x213
6
trace_do_page_fault
vmlinux 0xffffffff8698da26
Build id: 3.10.0-1127.19.1.el7.x86_64
0x56
7
do_async_page_fault
vmlinux 0xffffffff8698cfa2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22
8
async_page_fault
vmlinux 0xffffffff869897a8
Build id: 3.10.0-1127.19.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [gunicorn: worke:3692]
Modules linked in: nf_conntrack_netlink xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter xt_set ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ip_set_hash_net ip_set_hash_ip ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel snd_hda_codec_generic aesni_intel snd_hda_intel snd_hda_codec lrw snd_hda_core snd_hwdep snd_seq gf128mul glue_helper ablk_helper
cryptd snd_seq_device snd_pcm ppdev snd_timer snd soundcore pcspkr i2c_piix4 joydev virtio_rng parport_pc virtio_balloon parport ip_tables xfs libcrc32c qxl ata_generic pata_acpi virtio_blk drm_kms_helper virtio_net net_failover syscopyarea virtio_console failover sysfillrect ata_piix sysimgblt fb_sys_fops ttm libata floppy crct10dif_pclmul drm crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks
CPU: 1 PID: 3692 Comm: gunicorn: worke Kdump: loaded Tainted: G             L ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff923e67e35230 ti: ffff923bbf3a4000 task.ti: ffff923bbf3a4000
RIP: 0010:[<ffffffff86593b75>]  [<ffffffff86593b75>] copy_page_rep+0x5/0x10
RSP: 0000:ffff923bbf3a7ca0  EFLAGS: 00010286
RAX: fffff6d18617f9c0 RBX: fffff6d18617f9c0 RCX: 0000000000000200
RDX: ffffffff86c8d5a0 RSI: ffff923bf6039000 RDI: ffff923cc5fe7000
RBP: ffff923bbf3a7d08 R08: ffffffff86c8d58f R09: fffff6d18617fa00
R10: 0000000000003eae R11: ffff923d8bfc5bd0 R12: fffff6d182d80e40
R13: 00007f8f7e179c58 R14: ffff923d8bfc5bd0 R15: ffff923e67e35230
FS:  00007f8f8b42b740(0000) GS:ffff923e7fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000060c90001 CR3: 00000000a5c6c000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff863ef1b0>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff863f17eb>] do_wp_page+0xfb/0x720
[<ffffffff863f5e82>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8698d653>] __do_page_fault+0x213/0x500
[<ffffffff8698da26>] trace_do_page_fault+0x56/0x150
[<ffffffff8698cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff869897a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #24730
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff9b993b35
Build id: 3.10.0-1127.13.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffff9b7ef190
Build id: 3.10.0-1127.13.1.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffff9b7f17cb
Build id: 3.10.0-1127.13.1.el7.x86_64
0xfb
4
? unlock_page
vmlinux 0xffffffff9b7be75b
Build id: 3.10.0-1127.13.1.el7.x86_64
0x2b
5
? do_read_fault.isra.63
vmlinux 0xffffffff9b7ee5e9
Build id: 3.10.0-1127.13.1.el7.x86_64
0x139
6
handle_mm_fault
vmlinux 0xffffffff9b7f5e62
Build id: 3.10.0-1127.13.1.el7.x86_64
0xb22
7
? futex_wake
vmlinux 0xffffffff9b712a20
Build id: 3.10.0-1127.13.1.el7.x86_64
0x90
8
__do_page_fault
vmlinux 0xffffffff9bd8d653
Build id: 3.10.0-1127.13.1.el7.x86_64
0x213
9
trace_do_page_fault
vmlinux 0xffffffff9bd8da26
Build id: 3.10.0-1127.13.1.el7.x86_64
0x56
10
do_async_page_fault
vmlinux 0xffffffff9bd8cfa2
Build id: 3.10.0-1127.13.1.el7.x86_64
0x22
11
async_page_fault
vmlinux 0xffffffff9bd897a8
Build id: 3.10.0-1127.13.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [lpqd:9087]
Modules linked in: tcp_lp uinput fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink arc4 md4 nls_utf8 cifs ccm dns_resolver sunrpc ext4 mbcache jbd2 snd_hda_codec_generic ppdev snd_hda_intel snd_hda_codec iosf_mbi snd_hda_core crc32_pclmul snd_hwdep snd_seq snd_seq_device ghash_clmulni_intel snd_pcm aesni_intel lrw snd_timer gf128mul snd joydev glue_helper ablk_helper soundcore cryptd i2c_piix4 pcspkr parport_pc parport virtio_balloon ip_tables xfs libcrc32c qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_generic pata_acpi virtio_blk ttm virtio_console
virtio_net net_failover failover drm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 9087 Comm: lpqd Kdump: loaded Not tainted 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8a1a4ebfe2a0 ti: ffff8a18f78fc000 task.ti: ffff8a18f78fc000
RIP: 0010:[<ffffffff9b993b35>]  [<ffffffff9b993b35>] copy_page_rep+0x5/0x10
RSP: 0000:ffff8a18f78ffca0  EFLAGS: 00010286
RAX: ffffed82c73e47c0 RBX: ffffed82c73e47c0 RCX: 0000000000000200
RDX: ffffffff9c08d4f0 RSI: ffff8a1a6905a000 RDI: ffff8a1a0f91f000
RBP: ffff8a18f78ffd08 R08: ffffffff9c08d4df R09: ffffed82c73e4800
R10: 0000000000003476 R11: ffff8a1991857bd0 R12: ffffed82c8a41680
R13: 000056189e10d560 R14: ffff8a1991857bd0 R15: ffff8a1a4ebfe2a0
FS:  00007f290d54c900(0000) GS:ffff8a1a7fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000099c90001 CR3: 00000000babc4000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9b7ef190>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffff9b7f17cb>] do_wp_page+0xfb/0x720
[<ffffffff9b7be75b>] ? unlock_page+0x2b/0x30
[<ffffffff9b7ee5e9>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffff9b7f5e62>] handle_mm_fault+0xb22/0xfb0
[<ffffffff9b712a20>] ? futex_wake+0x90/0x180
[<ffffffff9bd8d653>] __do_page_fault+0x213/0x500
[<ffffffff9bd8da26>] trace_do_page_fault+0x56/0x150
[<ffffffff9bd8cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff9bd897a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #13446
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffffb29901c5
Build id: 3.10.0-1062.9.1.el7.x86_64
0x5
2
? wp_page_copy.isra.73
vmlinux 0xffffffffb27ed3a0
Build id: 3.10.0-1062.9.1.el7.x86_64
0x120
3
do_wp_page
vmlinux 0xffffffffb27ef9db
Build id: 3.10.0-1062.9.1.el7.x86_64
0xfb
4
? unlock_page
vmlinux 0xffffffffb27bd99b
Build id: 3.10.0-1062.9.1.el7.x86_64
0x2b
5
? do_read_fault.isra.63
vmlinux 0xffffffffb27ec7f9
Build id: 3.10.0-1062.9.1.el7.x86_64
0x139
6
handle_pte_fault
vmlinux 0xffffffffb27f1365
Build id: 3.10.0-1062.9.1.el7.x86_64
0x3d5
7
? alloc_pages_current
vmlinux 0xffffffffb2816b28
Build id: 3.10.0-1062.9.1.el7.x86_64
0x98
8
handle_mm_fault
vmlinux 0xffffffffb27f3ecd
Build id: 3.10.0-1062.9.1.el7.x86_64
0x39d
9
__do_page_fault
vmlinux 0xffffffffb2d88653
Build id: 3.10.0-1062.9.1.el7.x86_64
0x213
10
trace_do_page_fault
vmlinux 0xffffffffb2d88a26
Build id: 3.10.0-1062.9.1.el7.x86_64
0x56
11
do_async_page_fault
vmlinux 0xffffffffb2d87fa2
Build id: 3.10.0-1062.9.1.el7.x86_64
0x22
12
async_page_fault
vmlinux 0xffffffffb2d847a8
Build id: 3.10.0-1062.9.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 31s! [ksmtuned:394]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter binfmt_misc sunrpc nfit libnvdimm ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper sg ablk_helper cryptd joydev pcspkr virtio_balloon parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom cirrus ata_generic drm_kms_helper pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_console virtio_blk ata_piix crct10dif_pclmul drm 8139too libata crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw 8139cp floppy drm_panel_orientation_quirks
mii virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 394 Comm: ksmtuned Kdump: loaded Tainted: G             L ------------   3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff8b3235a28000 ti: ffff8b312e608000 task.ti: ffff8b312e608000
RIP: 0010:[<ffffffffb29901c5>]  [<ffffffffb29901c5>] copy_page_rep+0x5/0x10
RSP: 0000:ffff8b312e60bc18  EFLAGS: 00010286
RAX: ffffd6dcc1bab700 RBX: ffffd6dcc1bab700 RCX: 0000000000000200
RDX: ffffffffb308afc0 RSI: ffff8b3131135000 RDI: ffff8b312eadc000
RBP: ffff8b312e60bc80 R08: ffffffffb308afaf R09: ffffd6dcc1bab740
R10: 0000000000003852 R11: 00003ffffffff000 R12: ffffd6dcc1c44d40
R13: 00000000022c1cd4 R14: ffff8b315f57e000 R15: ffff8b3235a28000
FS:  00007f8895f1f740(0000) GS:ffff8b323a080000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000bdc90001 CR3: 000000006f418000 CR4: 00000000007606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffb27ed3a0>] ? wp_page_copy.isra.73+0x120/0x5b0
[<ffffffffb27ef9db>] do_wp_page+0xfb/0x720
[<ffffffffb27bd99b>] ? unlock_page+0x2b/0x30
[<ffffffffb27ec7f9>] ? do_read_fault.isra.63+0x139/0x1b0
[<ffffffffb27f1365>] handle_pte_fault+0x3d5/0xe20
[<ffffffffb2816b28>] ? alloc_pages_current+0x98/0x110
[<ffffffffb27f3ecd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb2d88653>] __do_page_fault+0x213/0x500
[<ffffffffb2d88a26>] trace_do_page_fault+0x56/0x150
[<ffffffffb2d87fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffb2d847a8>] async_page_fault+0x28/0x30
Code: ff ff ff ff c3 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 0f 1f 84 00 00 00 00 00 
Complete report #363
Frame # Function Binary Source or offset Line
1
copy_page_rep
vmlinux 0xffffffff81325265
Build id: 3.10.0-514.el7.x86_64
0x5
2
? wp_page_copy.isra.56
vmlinux 0xffffffff811ace9b
Build id: 3.10.0-514.el7.x86_64
0xeb
3
do_wp_page
vmlinux 0xffffffff811aeba5
Build id: 3.10.0-514.el7.x86_64
0xe5
4
? __enqueue_entity
vmlinux 0xffffffff810c95a8
Build id: 3.10.0-514.el7.x86_64
0x78
5
? enqueue_entity
vmlinux 0xffffffff810d000c
Build id: 3.10.0-514.el7.x86_64
0x26c
6
handle_mm_fault
vmlinux 0xffffffff811b0e3c
Build id: 3.10.0-514.el7.x86_64
0x72c
7
__do_page_fault
vmlinux 0xffffffff81691a94
Build id: 3.10.0-514.el7.x86_64
0x154
8
trace_do_page_fault
vmlinux 0xffffffff81691e76
Build id: 3.10.0-514.el7.x86_64
0x56
9
do_async_page_fault
vmlinux 0xffffffff8169151b
Build id: 3.10.0-514.el7.x86_64
0x1b
10
async_page_fault
vmlinux 0xffffffff8168e0b8
Build id: 3.10.0-514.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [bash:27237]
Modules linked in: fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic iosf_mbi crc32_pclmul snd_hda_intel ghash_clmulni_intel aesni_intel snd_hda_codec lrw gf128mul glue_helper ablk_helper snd_hda_core snd_hwdep snd_seq ppdev snd_seq_device cryptd snd_pcm snd_timer snd pcspkr parport_pc virtio_balloon parport sg i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect
crc32c_intel sysimgblt ata_piix fb_sys_fops serio_raw ttm drm virtio_pci virtio_ring virtio libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 27237 Comm: bash Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff880131ff0fb0 ti: ffff880121164000 task.ti: ffff880121164000
RIP: 0010:[<ffffffff81325265>]  [<ffffffff81325265>] copy_page_rep+0x5/0x10
RSP: 0000:ffff880121167d18  EFLAGS: 00010286
RAX: 000000006b9d6000 RBX: ffffea0001ae7580 RCX: 0000000000000200
RDX: 0000000000000000 RSI: ffff88000be10000 RDI: ffff88006b9d6000
RBP: ffff880121167d80 R08: ffffffff818df727 R09: ffffea0001ae75c0
R10: 000000000000312e R11: 800000000be10865 R12: ffffea00002f8400
R13: ffff880121164000 R14: ffff88022fa8da20 R15: ffff880131fee088
FS:  00007fa810624740(0000) GS:ffff880239d80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000e15b6003 CR3: 000000022df95000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811ace9b 00000000022e2000 ffff88008ac0d140 ffffea0004c7fbf0
800000000be10865 00000000022e0000 00000000022e0fe8 ffff88008ac0d140
ffff88022fa8da20 00000000022e0fe8 ffffea0004c7fbf0 ffffea00002f8400
Call Trace:
[<ffffffff811ace9b>] ? wp_page_copy.isra.56+0xeb/0x470
[<ffffffff811aeba5>] do_wp_page+0xe5/0x530
[<ffffffff810c95a8>] ? __enqueue_entity+0x78/0x80
[<ffffffff810d000c>] ? enqueue_entity+0x26c/0xb60
[<ffffffff811b0e3c>] handle_mm_fault+0x72c/0xfe0
[<ffffffff81691a94>] __do_page_fault+0x154/0x450
[<ffffffff81691e76>] trace_do_page_fault+0x56/0x150
[<ffffffff8169151b>] do_async_page_fault+0x1b/0xd0
[<ffffffff8168e0b8>] async_page_fault+0x28/0x30
Code: b7 90 90 90 90 90 9c fa 65 48 3b 06 75 14 65 48 3b 56 08 75 0d 65 48 89 1e 65 48 89 4e 08 9d b0 01 c3 9d 30 c0 c3 b9 00 02 00 00 <f3> 48 a5 c3 0f 1f 80 00 00 00 00 eb ee 66 66 66 90 66 66 66 90