Info

Permalink
Function
generic_exec_single
First occurrence
2020-10-15
Last occurrence
2025-02-06
State
NEW
Components
kernel
Quality
-34

Statistics

Operating system Count
CentOS 7 128
Architecture Count
x86_64 128
Related packages Count
kernel 128
 0:3.10.0-957.el7 25
 0:3.10.0-1160.71.1.el7 17
 0:3.10.0-1160.el7 13
 0:3.10.0-1160.62.1.el7 12
 0:3.10.0-862.el7 10
 0:3.10.0-1160.119.1.el7 7
 0:3.10.0-1160.45.1.el7 7
 0:3.10.0-1160.49.1.el7 4
 0:3.10.0-1062.el7 4
 0:3.10.0-1160.88.1.el7 3
 0:3.10.0-1127.18.2.el7 3
 0:3.10.0-1127.13.1.el7 2
 0:3.10.0-1160.108.1.el7 2
 0:3.10.0-1160.59.1.el7 2
 0:3.10.0-1127.el7 2
 0:3.10.0-1160.42.2.el7 2
 0:3.10.0-1160.90.1.el7 2
 0:3.10.0-862.14.4.el7 2
 0:3.10.0-1160.105.1.el7 2
 0:3.10.0-1160.95.1.el7 1
 0:3.10.0-1160.21.1.el7 1
 0:3.10.0-1160.66.1.el7 1
 0:3.10.0-1160.25.1.el7 1
 0:3.10.0-1160.76.1.el7 1
 0:3.10.0-957.5.1.el7 1
 0:3.10.0-693.11.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1418884
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f933e
Build id: 3.10.0-693.11.1.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.11.1.el7.x86_64 		0x70
5
smp_call_function_single
vmlinux 0xffffffff810f943f
Build id: 3.10.0-693.11.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff81322645
Build id: 3.10.0-693.11.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99d3
Build id: 3.10.0-693.11.1.el7.x86_64 		0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.11.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.11.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff811c2494
Build id: 3.10.0-693.11.1.el7.x86_64 		0x54
11
wp_page_copy.isra.58
vmlinux 0xffffffff811adf50
Build id: 3.10.0-693.11.1.el7.x86_64 		0x240
12
do_wp_page
vmlinux 0xffffffff811afb8b
Build id: 3.10.0-693.11.1.el7.x86_64 		0xeb
13
? do_read_fault.isra.44
vmlinux 0xffffffff811ad616
Build id: 3.10.0-693.11.1.el7.x86_64 		0xe6
14
handle_mm_fault
vmlinux 0xffffffff811b1efc
Build id: 3.10.0-693.11.1.el7.x86_64 		0x70c
15
__do_page_fault
vmlinux 0xffffffff816b0074
Build id: 3.10.0-693.11.1.el7.x86_64 		0x154
16
? mprotect_fixup
vmlinux 0xffffffff811ba02a
Build id: 3.10.0-693.11.1.el7.x86_64 		0x14a
17
trace_do_page_fault
vmlinux 0xffffffff816b0456
Build id: 3.10.0-693.11.1.el7.x86_64 		0x56
18
do_async_page_fault
vmlinux 0xffffffff816afaea
Build id: 3.10.0-693.11.1.el7.x86_64 		0x1a
19
async_page_fault
vmlinux 0xffffffff816ac5f8
Build id: 3.10.0-693.11.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [java:13627]
Modules linked in: binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bochs_drm ttm drm_kms_helper syscopyarea sysfillrect ppdev sysimgblt fb_sys_fops pcspkr drm sg joydev shpchp parport_pc i2c_piix4 virtio_balloon parport i2c_core nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk ata_piix libata serio_raw virtio_pci virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 13627 Comm: java Tainted: G             L ------------   3.10.0-693.11.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff8800a4d65ee0 ti: ffff8801a9f18000 task.ti: ffff8801a9f18000
RIP: 0010:[<ffffffff810f933e>]  [<ffffffff810f933e>] generic_exec_single+0xfe/0x1a0
RSP: 0000:ffff8801a9f1bba0  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff8801a9f1bb70 RCX: 0000000000000830
RDX: 0000000000000002 RSI: 0000000000000002 RDI: 0000000000000286
RBP: ffff8801a9f1bbe8 R08: ffffffff816ce6e0 R09: ffffea00084b2f00
R10: ffffea00084b2f00 R11: 8000000001cec225 R12: ffff8801a9f1bba8
R13: ffff8801a9f1bc40 R14: 0000000000000002 R15: 00000000cbf63620
FS:  00007f98bb7f7700(0000) GS:ffff88027fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f98d73039c8 CR3: 00000002652a6000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff8801a9f1bc70
0000000000000003 00000000063477db 0000000000000001 ffffffff8106e4a0
ffffffff8106e4a0 ffff8801a9f1bc18 ffffffff810f943f ffff8801a9f1bc18
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f943f>] smp_call_function_single+0x5f/0xa0
[<ffffffff81322645>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99d3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2494>] ptep_clear_flush+0x54/0x80
[<ffffffff811adf50>] wp_page_copy.isra.58+0x240/0x470
[<ffffffff811afb8b>] do_wp_page+0xeb/0x5c0
[<ffffffff811ad616>] ? do_read_fault.isra.44+0xe6/0x130
[<ffffffff811b1efc>] handle_mm_fault+0x70c/0xfa0
[<ffffffff816b0074>] __do_page_fault+0x154/0x450
[<ffffffff811ba02a>] ? mprotect_fixup+0x14a/0x240
[<ffffffff816b0456>] trace_do_page_fault+0x56/0x150
[<ffffffff816afaea>] do_async_page_fault+0x1a/0xd0
[<ffffffff816ac5f8>] async_page_fault+0x28/0x30
Code: 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 ba d9 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 7f
Complete report #1404428
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb6d1109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffb6c7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffb6c7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffb6c7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffb6d111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb6f75465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb6d1175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb6c7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffb6c7a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffb6dfd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffffb6de55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffffb6de782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? check_preempt_curr
vmlinux 0xffffffffb6cd2eb2
Build id: 3.10.0-957.el7.x86_64 		0x92
14
handle_pte_fault
vmlinux 0xffffffffb6de91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
15
? try_to_wake_up
vmlinux 0xffffffffb6cd64f0
Build id: 3.10.0-957.el7.x86_64 		0x190
16
handle_mm_fault
vmlinux 0xffffffffb6debc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffffb736f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
18
trace_do_page_fault
vmlinux 0xffffffffb736f9c6
Build id: 3.10.0-957.el7.x86_64 		0x56
19
do_async_page_fault
vmlinux 0xffffffffb736ef42
Build id: 3.10.0-957.el7.x86_64 		0x22
20
async_page_fault
vmlinux 0xffffffffb736b788
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [kube-controller:34644]
Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel xt_addrtype xt_set veth ip_set_hash_ipportnet ip_set_hash_ipport ip_set_bitmap_port ip_set_hash_ipportip ip_set dummy iptable_raw xt_CT ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc ppdev sg joydev virtio_balloon parport_pc parport i2c_piix4 pcspkr br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_scsi virtio_console virtio_net cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata serio_raw drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 34644 Comm: kube-controller Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff8cfe55f72080 ti: ffff8cff5d67c000 task.ti: ffff8cff5d67c000
RIP: 0010:[<ffffffffb6d1109a>]  [<ffffffffb6d1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff8cff5d67fab0  EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff8cff5d67fa80 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000282
RBP: ffff8cff5d67fb00 R08: ffffffffb7412d80 R09: ffffcfe307630840
R10: 0000000000004ed9 R11: 00003ffffffff000 R12: 0000000000000040
R13: 0000000000000002 R14: ffff8cffbff9f008 R15: 0000000000000000
FS:  000000c00219cb10(0000) GS:ffff8cff7eb40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb45f817a48 CR3: 0000000f1eff4000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb6c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb6c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb6c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb6d111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb6f75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffb6d1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffb6c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb6c7a734>] flush_tlb_page+0x54/0xa0
[<ffffffffb6dfd587>] ptep_clear_flush+0x67/0xa0
[<ffffffffb6de55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffffb6de782b>] do_wp_page+0xfb/0x720
[<ffffffffb6cd2eb2>] ? check_preempt_curr+0x92/0xa0
[<ffffffffb6de91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffffb6cd64f0>] ? try_to_wake_up+0x190/0x390
[<ffffffffb6debc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb736f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb736f9c6>] trace_do_page_fault+0x56/0x150
[<ffffffffb736ef42>] do_async_page_fault+0x22/0xf0
[<ffffffffb736b788>] async_page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 94 b7 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #21640
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffac316f72
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffac27e730
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffac27e730
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffac27e730
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffac31708f
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffac584fc5
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffac31763b
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffac27e8f8
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffac27ea94
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffac407888
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffac3ef3c5
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffac3f17eb
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xfb
13
? xfs_iunlock
xfs 0xffffffffc051335c
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xfc
14
? xfs_file_buffered_aio_read
xfs 0xffffffffc0504fd5
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xa5
15
handle_mm_fault
vmlinux 0xffffffffac3f5e82
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xb22
16
__do_page_fault
vmlinux 0xffffffffac98d653
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x213
17
do_page_fault
vmlinux 0xffffffffac98d975
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x35
18
? error_swapgs
vmlinux 0xffffffffac989ac9
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xaa
19
page_fault
vmlinux 0xffffffffac989778
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [httpd:23745]
Modules linked in: nfsv3 nfs_acl nf_conntrack_netlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm ppdev snd_timer snd soundcore sg parport_pc pcspkr joydev parport virtio_balloon i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom
ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_blk drm ata_piix libata serio_raw 8139too virtio_pci virtio_ring virtio 8139cp mii drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 23745 Comm: httpd Kdump: loaded Tainted: G             L ------------   3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
task: ffffa00342e4a0e0 ti: ffffa00355dd0000 task.ti: ffffa00355dd0000
RIP: 0010:[<ffffffffac316f72>]  [<ffffffffac316f72>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffffa00355dd3b60  EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffffa003059ab968 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffffa00355dd3ba8 R08: 0000000000000001 R09: ffffea64011e0cc0
R10: 0000000000004a2d R11: ffffa00284b035e8 R12: 0000000000000002
R13: ffffa0039ffd7008 R14: 0000000000000000 R15: 0000000300000000
FS:  00007f4b76e4d880(0000) GS:ffffa0039fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4b52f07000 CR3: 000000009a9a2000 CR4: 00000000000006e0
Call Trace:
[<ffffffffac27e730>] ? leave_mm+0x120/0x120
[<ffffffffac27e730>] ? leave_mm+0x120/0x120
[<ffffffffac27e730>] ? leave_mm+0x120/0x120
[<ffffffffac31708f>] smp_call_function_single+0x5f/0xa0
[<ffffffffac584fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffffac31763b>] smp_call_function_many+0x22b/0x270
[<ffffffffac27e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffac27ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffac407888>] ptep_clear_flush+0x68/0xa0
[<ffffffffac3ef3c5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffac3f17eb>] do_wp_page+0xfb/0x720
[<ffffffffc051335c>] ? xfs_iunlock+0xfc/0x120 [xfs]
[<ffffffffc0504fd5>] ? xfs_file_buffered_aio_read+0xa5/0x110 [xfs]
[<ffffffffac3f5e82>] handle_mm_fault+0xb22/0xfb0
[<ffffffffac98d653>] __do_page_fault+0x213/0x500
[<ffffffffac98d975>] do_page_fault+0x35/0x90
[<ffffffffac989ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffac989778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 a0 10 f5 ac 48 89 df e8 c7 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1085669
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb7b111ba
Build id: 3.10.0-957.5.1.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffb7a7a400
Build id: 3.10.0-957.5.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffb7a7a400
Build id: 3.10.0-957.5.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffb7a7a400
Build id: 3.10.0-957.5.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffb7b112cf
Build id: 3.10.0-957.5.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb7d75a45
Build id: 3.10.0-957.5.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb7b1187b
Build id: 3.10.0-957.5.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb7a7a5c8
Build id: 3.10.0-957.5.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffb7a7a764
Build id: 3.10.0-957.5.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffb7bfd937
Build id: 3.10.0-957.5.1.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffffb7be596b
Build id: 3.10.0-957.5.1.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffffb7be7bdb
Build id: 3.10.0-957.5.1.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffffb7c32de2
Build id: 3.10.0-957.5.1.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffffb7be956d
Build id: 3.10.0-957.5.1.el7.x86_64 		0x37d
15
? cpu_timer_sample_group
vmlinux 0xffffffffb7ac3b70
Build id: 3.10.0-957.5.1.el7.x86_64 		0x30
16
handle_mm_fault
vmlinux 0xffffffffb7bec01d
Build id: 3.10.0-957.5.1.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffffb816f5e3
Build id: 3.10.0-957.5.1.el7.x86_64 		0x203
18
do_page_fault
vmlinux 0xffffffffb816f915
Build id: 3.10.0-957.5.1.el7.x86_64 		0x35
19
? error_swapgs
vmlinux 0xffffffffb816ba96
Build id: 3.10.0-957.5.1.el7.x86_64 		0xa7
20
page_fault
vmlinux 0xffffffffb816b758
Build id: 3.10.0-957.5.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [httpd:4359]
Modules linked in: binfmt_misc nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev i2c_piix4 parport_pc parport vmw_vmci sg ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmwgfx floppy ata_generic pata_acpi drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi vmxnet3 mptscsih ata_piix mptbase libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 4359 Comm: httpd Kdump: loaded Tainted: G             L ------------   3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9c9d75c39040 ti: ffff9c9db7cdc000 task.ti: ffff9c9db7cdc000
RIP: 0010:[<ffffffffb7b111ba>]  [<ffffffffb7b111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff9c9db7cdfae0  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff9c9db7cdfab0 RCX: 000000000000003e
RDX: 0000000000000003 RSI: 0000000000000002 RDI: 0000000000000282
RBP: ffff9c9db7cdfb30 R08: ffffffffb8212dc0 R09: ffffe0b301806f40
R10: 0000000000003688 R11: 00003ffffffff000 R12: 0000000000000286
R13: 0000000000000001 R14: ffff9c9dbffdb008 R15: 0000000000000000
FS:  00007fb77f7f6700(0000) GS:ffff9c9dbfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb73766a080 CR3: 0000000077832000 CR4: 00000000000607e0
Call Trace:
[<ffffffffb7a7a400>] ? leave_mm+0x110/0x110
[<ffffffffb7a7a400>] ? leave_mm+0x110/0x110
[<ffffffffb7a7a400>] ? leave_mm+0x110/0x110
[<ffffffffb7b112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffffb7d75a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffb7b1187b>] smp_call_function_many+0x22b/0x270
[<ffffffffb7a7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb7a7a764>] flush_tlb_page+0x54/0xa0
[<ffffffffb7bfd937>] ptep_clear_flush+0x67/0xa0
[<ffffffffb7be596b>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffffb7be7bdb>] do_wp_page+0xfb/0x720
[<ffffffffb7c32de2>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffffb7be956d>] handle_pte_fault+0x37d/0xd10
[<ffffffffb7ac3b70>] ? cpu_timer_sample_group+0x30/0xa0
[<ffffffffb7bec01d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb816f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb816f915>] do_page_fault+0x35/0x90
[<ffffffffb816ba96>] ? error_swapgs+0xa7/0xbd
[<ffffffffb816b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 74 b8 48 89 df e8 da b6 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #518581
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff88514a22
Build id: 3.10.0-1062.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8847d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff8847d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff8847d1d0
Build id: 3.10.0-1062.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff88514b3f
Build id: 3.10.0-1062.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8877f475
Build id: 3.10.0-1062.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff885150eb
Build id: 3.10.0-1062.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8847d398
Build id: 3.10.0-1062.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8847d534
Build id: 3.10.0-1062.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff88603b18
Build id: 3.10.0-1062.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff885eb6f5
Build id: 3.10.0-1062.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff885edb1b
Build id: 3.10.0-1062.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff88639cc2
Build id: 3.10.0-1062.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffff885ef4a5
Build id: 3.10.0-1062.el7.x86_64 		0x3d5
15
handle_mm_fault
vmlinux 0xffffffff885f200d
Build id: 3.10.0-1062.el7.x86_64 		0x39d
16
__do_page_fault
vmlinux 0xffffffff88b86633
Build id: 3.10.0-1062.el7.x86_64 		0x213
17
do_page_fault
vmlinux 0xffffffff88b86955
Build id: 3.10.0-1062.el7.x86_64 		0x35
18
page_fault
vmlinux 0xffffffff88b82768
Build id: 3.10.0-1062.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [runc:7999]
Modules linked in: xt_recent veth nf_tables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache xt_CHECKSUM tun ebtable_filter ebtables ip6table_filter devlink ip6_tables nf_conntrack_netlink xt_statistic xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 xt_mark xt_addrtype xt_comment xt_conntrack iptable_filter iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ip_set nfnetlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio sb_edac ppdev iosf_mbi vmw_balloon crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper parport_pc pcspkr joydev sg i2c_piix4 cryptd parport
vmw_vmci br_netfilter auth_rpcgss bridge stp sunrpc llc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata mptspi crct10dif_pclmul crct10dif_common scsi_transport_spi drm serio_raw crc32c_intel mptscsih vmxnet3 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 7999 Comm: runc Kdump: loaded Tainted: G             L ------------   3.10.0-1062.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff955d0bc45230 ti: ffff955f8026c000 task.ti: ffff955f8026c000
RIP: 0010:[<ffffffff88514a22>]  [<ffffffff88514a22>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff955f8026fad0  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffff955f8026faa0 RCX: 0000000000000030
RDX: 000000000000ffff RSI: 0000000000000010 RDI: 0000000000000286
RBP: ffff955f8026fb20 R08: ffffffff88c13500 R09: ffffc6ed0c557ec0
R10: 0000000000002ccc R11: 00003ffffffff000 R12: ffff955d0bc45230
R13: 0000000000000002 R14: ffff955e7ffdb008 R15: 0000000000000000
FS:  00007f9d38476740(0000) GS:ffff955e6fd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c000140000 CR3: 000000013c87c000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8847d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8847d1d0>] ? leave_mm+0x110/0x110
[<ffffffff8847d1d0>] ? leave_mm+0x110/0x110
[<ffffffff88514b3f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8877f475>] ? cpumask_next_and+0x35/0x50
[<ffffffff885150eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8847d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8847d534>] flush_tlb_page+0x54/0xa0
[<ffffffff88603b18>] ptep_clear_flush+0x68/0xa0
[<ffffffff885eb6f5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff885edb1b>] do_wp_page+0xfb/0x720
[<ffffffff88639cc2>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff885ef4a5>] handle_pte_fault+0x3d5/0xe20
[<ffffffff885f200d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff88b86633>] __do_page_fault+0x213/0x500
[<ffffffff88b86955>] do_page_fault+0x35/0x90
[<ffffffff88b82768>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 e0 d7 14 89 48 89 df e8 d7 1a 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #443266
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9d91109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff9d87a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9d87a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9d87a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff9d9111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9db75465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9d91175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9d87a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9d87a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff9d9fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffff9d9e55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffff9d9e782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff9da329b2
Build id: 3.10.0-957.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffff9d9e91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
15
handle_mm_fault
vmlinux 0xffffffff9d9ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
16
__do_page_fault
vmlinux 0xffffffff9df6f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
17
do_page_fault
vmlinux 0xffffffff9df6f915
Build id: 3.10.0-957.el7.x86_64 		0x35
18
page_fault
vmlinux 0xffffffff9df6b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [gnome-shell:20188]
Modules linked in: fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm sg snd_timer snd soundcore vmw_vmci i2c_piix4 parport_pc parport sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm e1000 libata drm_panel_orientation_quirks mptspi scsi_transport_spi mptscsih mptbase nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 20188 Comm: gnome-shell Kdump: loaded Not tainted 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff944e35075140 ti: ffff944ea9000000 task.ti: ffff944ea9000000
RIP: 0010:[<ffffffff9d91109a>]  [<ffffffff9d91109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff944ea9003ae0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff944ea9003ab0 RCX: 0000000000000001
RDX: ffffffff9e012d90 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff944ea9003b30 R08: ffffffff9e012d88 R09: fffffbeb049eb300
R10: 0000000000001494 R11: 00003ffffffff000 R12: 0000000039655900
R13: 0000000000000002 R14: ffff944f3ffd9008 R15: 0000000000000000
FS:  00007f19f909ea80(0000) GS:ffff944f39640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000338e000 CR3: 00000000ad7b6000 CR4: 0000000000740fe0
PKRU: 55555554
Call Trace:
[<ffffffff9d87a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9d87a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9d87a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9d9111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9db75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9d91175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9d87a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9d87a734>] flush_tlb_page+0x54/0xa0
[<ffffffff9d9fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff9d9e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffff9d9e782b>] do_wp_page+0xfb/0x720
[<ffffffff9da329b2>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff9d9e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffff9d9ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9df6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff9df6f915>] do_page_fault+0x35/0x90
[<ffffffff9df6b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 54 9e 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #438208
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9930779a
Build id: 3.10.0-862.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff99276e50
Build id: 3.10.0-862.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff99276e50
Build id: 3.10.0-862.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff99276e50
Build id: 3.10.0-862.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff993078af
Build id: 3.10.0-862.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9954a275
Build id: 3.10.0-862.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff99307e5b
Build id: 3.10.0-862.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff99277018
Build id: 3.10.0-862.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff992771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff993d8837
Build id: 3.10.0-862.el7.x86_64 		0x67
11
wp_page_copy.isra.70
vmlinux 0xffffffff993c18be
Build id: 3.10.0-862.el7.x86_64 		0x29e
12
do_wp_page
vmlinux 0xffffffff993c3abb
Build id: 3.10.0-862.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff9940ca92
Build id: 3.10.0-862.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffff993c536d
Build id: 3.10.0-862.el7.x86_64 		0x36d
15
? __switch_to
vmlinux 0xffffffff99229621
Build id: 3.10.0-862.el7.x86_64 		0x151
16
handle_mm_fault
vmlinux 0xffffffff993c747d
Build id: 3.10.0-862.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffff9991a597
Build id: 3.10.0-862.el7.x86_64 		0x197
18
do_page_fault
vmlinux 0xffffffff9991a925
Build id: 3.10.0-862.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffff99916768
Build id: 3.10.0-862.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [containerd:118592]
Modules linked in: tcp_lp nls_utf8 isofs fuse veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel lrw ppdev ac97_bus gf128mul glue_helper snd_seq ablk_helper cryptd snd_seq_device vmw_balloon snd_pcm snd_timer snd pcspkr joydev nfit soundcore sg libnvdimm parport_pc vmw_vmci parport i2c_piix4 shpchp ip_tables xfs libcrc32c sr_mod cdrom ata_generic
pata_acpi sd_mod crc_t10dif crct10dif_generic ahci crct10dif_pclmul libahci crct10dif_common ata_piix vmwgfx drm_kms_helper mptspi scsi_transport_spi crc32c_intel syscopyarea mptscsih sysfillrect sysimgblt fb_sys_fops ttm drm e1000 libata serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 118592 Comm: containerd Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8dcb8d83eeb0 ti: ffff8dcbb8d48000 task.ti: ffff8dcbb8d48000
RIP: 0010:[<ffffffff9930779a>]  [<ffffffff9930779a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff8dcbb8d4baf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8dcbb8d4bac0 RCX: 0000000000000001
RDX: ffffffff99a13170 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8dcbb8d4bb38 R08: ffffffff99a13168 R09: ffffc2d6c956f4c0
R10: ffffc2d6c956f4c0 R11: 00003ffffffff000 R12: 0000000100000000
R13: 00000000fffda000 R14: ffffffff9939b54a R15: 000280da00000000
FS:  00007f6187a48740(0000) GS:ffff8dcdf3640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00015c4a0 CR3: 000000007d066000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff99276e50>] ? leave_mm+0x110/0x110
[<ffffffff99276e50>] ? leave_mm+0x110/0x110
[<ffffffff99276e50>] ? leave_mm+0x110/0x110
[<ffffffff993078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9954a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff99307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff99277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff992771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff993d8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff993c18be>] wp_page_copy.isra.70+0x29e/0x4f0
[<ffffffff993c3abb>] do_wp_page+0xfb/0x710
[<ffffffff9940ca92>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff993c536d>] handle_pte_fault+0x36d/0xc30
[<ffffffff99229621>] ? __switch_to+0x151/0x580
[<ffffffff993c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff9991a597>] __do_page_fault+0x197/0x4f0
[<ffffffff9991a925>] do_page_fault+0x35/0x90
[<ffffffff99916768>] page_fault+0x28/0x30
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 f3 99 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #403723
Frame # Function Binary Source or offset Line
1
? leave_mm
vmlinux 0xffffffffa307e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
2
? leave_mm
vmlinux 0xffffffffa307e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
3
smp_call_function_single
vmlinux 0xffffffffa311667f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
4
? cpumask_next_and
vmlinux 0xffffffffa3385de5
Build id: 3.10.0-1160.el7.x86_64 		0x35
5
smp_call_function_many
vmlinux 0xffffffffa3116c2b
Build id: 3.10.0-1160.el7.x86_64 		0x22b
6
native_flush_tlb_others
vmlinux 0xffffffffa307e868
Build id: 3.10.0-1160.el7.x86_64 		0xb8
7
flush_tlb_page
vmlinux 0xffffffffa307ea04
Build id: 3.10.0-1160.el7.x86_64 		0x54
8
ptep_clear_flush
vmlinux 0xffffffffa3207728
Build id: 3.10.0-1160.el7.x86_64 		0x68
9
wp_page_copy.isra.73
vmlinux 0xffffffffa31ef295
Build id: 3.10.0-1160.el7.x86_64 		0x335
10
do_wp_page
vmlinux 0xffffffffa31f16bb
Build id: 3.10.0-1160.el7.x86_64 		0xfb
11
? __mem_cgroup_commit_charge
vmlinux 0xffffffffa323e322
Build id: 3.10.0-1160.el7.x86_64 		0xe2
12
handle_mm_fault
vmlinux 0xffffffffa31f5d32
Build id: 3.10.0-1160.el7.x86_64 		0xb22
13
__do_page_fault
vmlinux 0xffffffffa378e653
Build id: 3.10.0-1160.el7.x86_64 		0x213
14
do_page_fault
vmlinux 0xffffffffa378e975
Build id: 3.10.0-1160.el7.x86_64 		0x35
15
page_fault
vmlinux 0xffffffffa378a778
Build id: 3.10.0-1160.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 24s! [kworker/u256:2:3066]
[<ffffffffa307e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa307e6a0>] ? leave_mm+0x120/0x120
[<ffffffffa311667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa3385de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa3116c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffa307e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa307ea04>] flush_tlb_page+0x54/0xa0
[<ffffffffa3207728>] ptep_clear_flush+0x68/0xa0
[<ffffffffa31ef295>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffa31f16bb>] do_wp_page+0xfb/0x720
[<ffffffffa323e322>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffffa31f5d32>] handle_mm_fault+0xb22/0xfb0
[<ffffffffa378e653>] __do_page_fault+0x213/0x500
[<ffffffffa378e975>] do_page_fault+0x35/0x90
[<ffffffffa378a778>] page_fault+0x28/0x30
Complete report #393908
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff81b16a32
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff81a7ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff81a7ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff81a7ddd0
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff81b16b4f
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff81d86a45
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff81b170fb
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff81a7e258
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff81a7e3f4
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff81c08108
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff81befc05
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff81bf202b
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff81c3ed32
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xe2
14
handle_mm_fault
vmlinux 0xffffffff81bf66a2
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xb22
15
__do_page_fault
vmlinux 0xffffffff82190653
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x213
16
do_page_fault
vmlinux 0xffffffff82190975
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x35
17
? error_swapgs
vmlinux 0xffffffff8218cac9
Build id: 3.10.0-1160.49.1.el7.x86_64 		0xaa
18
page_fault
vmlinux 0xffffffff8218c778
Build id: 3.10.0-1160.49.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kubelet:5353]
Modules linked in: tcp_lp nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) sunrpc snd_intel8x0 joydev vmwgfx snd_ac97_codec ac97_bus ttm ppdev drm_kms_helper snd_seq snd_seq_device pcspkr snd_pcm vboxguest(OE) syscopyarea sysfillrect sysimgblt fb_sys_fops sg drm snd_timer snd parport_pc drm_panel_orientation_quirks video parport soundcore i2c_piix4 ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi ahci crc32c_intel ata_piix serio_raw
libahci libata e1000 dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 5353 Comm: kubelet Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-1160.49.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8cf0a343b180 ti: ffff8cf113ad0000 task.ti: ffff8cf113ad0000
RIP: 0010:[<ffffffff81b16a32>]  [<ffffffff81b16a32>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff8cf113ad3b60  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 00000001000280da RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff8cf113ad3ba8 R08: 0000000000000001 R09: ffffd20f81a88b80
R10: 00000000000034b1 R11: ffff8cf116ffb1b0 R12: 0000000000000000
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fe7cce8a740(0000) GS:ffff8cf120100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe7a50a8f88 CR3: 00000001a3774000 CR4: 00000000000006e0
Call Trace:
[<ffffffff81a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff81a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff81a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff81b16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff81d86a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff81b170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff81a7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81a7e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffff81c08108>] ptep_clear_flush+0x68/0xa0
[<ffffffff81befc05>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff81bf202b>] do_wp_page+0xfb/0x720
[<ffffffff81c3ed32>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff81bf66a2>] handle_mm_fault+0xb22/0xfb0
[<ffffffff82190653>] __do_page_fault+0x213/0x500
[<ffffffff82190975>] do_page_fault+0x35/0x90
[<ffffffff8218cac9>] ? error_swapgs+0xaa/0xc0
[<ffffffff8218c778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 60 15 75 82 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #266493
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff83516842
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8347e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8347e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8347e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8351695f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff83786395
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff83516f0b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8347e868
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8347ea04
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff836079d8
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff835ef4e5
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff835f190b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff8363e5d2
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xe2
14
handle_mm_fault
vmlinux 0xffffffff835f5f82
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xb22
15
__do_page_fault
vmlinux 0xffffffff83b90653
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x213
16
do_page_fault
vmlinux 0xffffffff83b90975
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x35
17
? error_swapgs
vmlinux 0xffffffff83b8cac9
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xaa
18
page_fault
vmlinux 0xffffffff83b8c778
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [service-catalog:57572]
Modules linked in: xt_statistic xt_recent vport_vxlan xt_REDIRECT nf_nat_redirect ip_vs_rr xt_ipvs ip_vs vxlan ip6_udp_tunnel udp_tunnel veth nf_conntrack_netlink nfnetlink openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_defrag_ipv6 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache nfsd auth_rpcgss nfs_acl lockd grace iptable_mangle br_netfilter bridge stp llc scsi_transport_iscsi ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_nat iptable_nat nf_nat_ipv4 nf_nat ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_comment xt_mark xt_addrtype xt_conntrack overlay(T) nf_conntrack iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon i2c_piix4 joydev pcspkr sg vmw_vmci
parport_pc parport ip_tables xfs sr_mod cdrom ata_generic pata_acpi sd_mod vmwgfx drm_kms_helper crc32c_intel e1000e syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw drm ata_piix ptp mptsas libata pps_core scsi_transport_sas mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log fuse target_core_user uio target_core_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common dm_multipath dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio dm_mod libcrc32c
CPU: 1 PID: 57572 Comm: service-catalog Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff93e4da7de300 ti: ffff93e755220000 task.ti: ffff93e755220000
RIP: 0010:[<ffffffff83516842>]  [<ffffffff83516842>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff93e755223b60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff93e755223b30 RCX: 0000000000000004
RDX: ffffffff83c13f50 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff93e755223ba8 R08: ffffffff83c13f48 R09: ffffdbdd44f3c700
R10: 00000000000043b2 R11: ffff93e4dc01a000 R12: 0000000000000002
R13: ffff93e8bffcf008 R14: 0000000000000000 R15: 00000003e9257700
FS:  00007f46d3d17700(0000) GS:ffff93e8ad640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c422405000 CR3: 000000034cbaa000 CR4: 00000000000607e0
Call Trace:
[<ffffffff8347e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8347e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8347e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8351695f>] smp_call_function_single+0x5f/0xa0
[<ffffffff83786395>] ? cpumask_next_and+0x35/0x50
[<ffffffff83516f0b>] smp_call_function_many+0x22b/0x270
[<ffffffff8347e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8347ea04>] flush_tlb_page+0x54/0xa0
[<ffffffff836079d8>] ptep_clear_flush+0x68/0xa0
[<ffffffff835ef4e5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff835f190b>] do_wp_page+0xfb/0x720
[<ffffffff8363e5d2>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff835f5f82>] handle_mm_fault+0xb22/0xfb0
[<ffffffff83b90653>] __do_page_fault+0x213/0x500
[<ffffffff83b90975>] do_page_fault+0x35/0x90
[<ffffffff83b8cac9>] ? error_swapgs+0xaa/0xc0
[<ffffffff83b8c778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 a0 15 15 84 48 89 df e8 c7 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #263429
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9151109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff9147a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9147a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9147a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff915111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff91775465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9151175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9147a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9147a734
Build id: 3.10.0-957.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff915fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffff915e55bb
Build id: 3.10.0-957.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffff915e782b
Build id: 3.10.0-957.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff916329b2
Build id: 3.10.0-957.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffff915e91bd
Build id: 3.10.0-957.el7.x86_64 		0x37d
15
? set_huge_zero_page.isra.42.part.43
vmlinux 0xffffffff91628531
Build id: 3.10.0-957.el7.x86_64 		0xb1
16
handle_mm_fault
vmlinux 0xffffffff915ebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
17
__do_page_fault
vmlinux 0xffffffff91b6f5e3
Build id: 3.10.0-957.el7.x86_64 		0x203
18
do_page_fault
vmlinux 0xffffffff91b6f915
Build id: 3.10.0-957.el7.x86_64 		0x35
19
page_fault
vmlinux 0xffffffff91b6b758
Build id: 3.10.0-957.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [containerd:3008]
Modules linked in: xt_set ip_set_hash_net vxlan ip6_udp_tunnel udp_tunnel xt_nat xt_mark xfrm6_mode_tunnel xfrm4_mode_tunnel veth esp4 drbg ansi_cprng nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event
snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi ac97_bus snd_seq snd_seq_device crc32_pclmul snd_pcm ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper snd_timer cryptd snd pcspkr nfit ppdev vmw_balloon soundcore joydev sg libnvdimm parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops mptspi ttm scsi_transport_spi ata_piix serio_raw mptscsih drm mptbase libata e1000 drm_panel_orientation_quirks
CPU: 1 PID: 3008 Comm: containerd Kdump: loaded Tainted: G               ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff98c16172a080 ti: ffff98c35f6c4000 task.ti: ffff98c35f6c4000
RIP: 0010:[<ffffffff9151109a>]  [<ffffffff9151109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff98c35f6c7ae0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff98c35f6c7ab0 RCX: 0000000000000001
RDX: ffffffff91c12d90 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff98c35f6c7b30 R08: ffffffff91c12d88 R09: ffffe34c475f6d40
R10: 00000000000043b2 R11: 00003ffffffff000 R12: ffff98c4b5f687d0
R13: 0000000000000002 R14: ffff98c4fffdb008 R15: 0000000000000000
FS:  00007fbe8968b740(0000) GS:ffff98c4ed640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbe84714bb8 CR3: 00000002dbfb2000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9147a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9147a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9147a3d0>] ? leave_mm+0x110/0x110
[<ffffffff915111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff91775465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9151175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9147a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9147a734>] flush_tlb_page+0x54/0xa0
[<ffffffff915fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff915e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffff915e782b>] do_wp_page+0xfb/0x720
[<ffffffff916329b2>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff915e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffff91628531>] ? set_huge_zero_page.isra.42.part.43+0xb1/0xd0
[<ffffffff915ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff91b6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff91b6f915>] do_page_fault+0x35/0x90
[<ffffffff91b6b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 92 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #24195
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9390779e
Build id: 3.10.0-862.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffff93876e50
Build id: 3.10.0-862.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff93876e50
Build id: 3.10.0-862.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff93876e50
Build id: 3.10.0-862.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff939078af
Build id: 3.10.0-862.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff93b4a275
Build id: 3.10.0-862.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff93907e5b
Build id: 3.10.0-862.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff93877018
Build id: 3.10.0-862.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff938771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff939d8837
Build id: 3.10.0-862.el7.x86_64 		0x67
11
wp_page_copy.isra.70
vmlinux 0xffffffff939c18be
Build id: 3.10.0-862.el7.x86_64 		0x29e
12
do_wp_page
vmlinux 0xffffffff939c3abb
Build id: 3.10.0-862.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff93a0ca92
Build id: 3.10.0-862.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffff939c536d
Build id: 3.10.0-862.el7.x86_64 		0x36d
15
handle_mm_fault
vmlinux 0xffffffff939c747d
Build id: 3.10.0-862.el7.x86_64 		0x39d
16
__do_page_fault
vmlinux 0xffffffff93f1a597
Build id: 3.10.0-862.el7.x86_64 		0x197
17
do_page_fault
vmlinux 0xffffffff93f1a925
Build id: 3.10.0-862.el7.x86_64 		0x35
18
page_fault
vmlinux 0xffffffff93f16768
Build id: 3.10.0-862.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [docker-containe:28781]
Modules linked in: xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_vs nf_conntrack devlink overlay(T) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq coretemp snd_seq_device iosf_mbi crc32_pclmul snd_pcm ghash_clmulni_intel snd_timer ppdev aesni_intel snd lrw gf128mul glue_helper soundcore vmw_vmci ablk_helper sg vmw_balloon joydev cryptd pcspkr i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ttm scsi_transport_spi ata_piix mptscsih crct10dif_pclmul drm crct10dif_common serio_raw e1000 mptbase libata crc32c_intel i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 28781 Comm: docker-containe Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
task: ffff8f1ea6281fa0 ti: ffff8f1e7bcf8000 task.ti: ffff8f1e7bcf8000
RIP: 0010:[<ffffffff9390779e>]  [<ffffffff9390779e>] generic_exec_single+0xfe/0x1b0
RSP: 0000:ffff8f1e7bcfbaf0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8f1e7bcfbac0 RCX: 0000000000000006
RDX: ffffffff94013df0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8f1e7bcfbb38 R08: ffffffff94013de8 R09: ffffe6f5048e9540
R10: ffffe6f5048e9540 R11: 00003ffffffff000 R12: 0000000100000000
R13: ffffffff938c7a6f R14: ffff8f1e7bcfba88 R15: 000280da00000000
FS:  00007f3da2ff9700(0000) GS:ffff8f1f39780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3da16a2038 CR3: 000000007bd74000 CR4: 00000000003607e0
Call Trace:
[<ffffffff93876e50>] ? leave_mm+0x110/0x110
[<ffffffff93876e50>] ? leave_mm+0x110/0x110
[<ffffffff93876e50>] ? leave_mm+0x110/0x110
[<ffffffff939078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff93b4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff93907e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff93877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff938771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff939d8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff939c18be>] wp_page_copy.isra.70+0x29e/0x4f0
[<ffffffff939c3abb>] do_wp_page+0xfb/0x710
[<ffffffff93a0ca92>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff939c536d>] handle_pte_fault+0x36d/0xc30
[<ffffffff939c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff93f1a597>] __do_page_fault+0x197/0x4f0
[<ffffffff93f1a925>] do_page_fault+0x35/0x90
[<ffffffff93f16768>] page_fault+0x28/0x30
Code: 48 89 de 48 03 14 c5 a0 53 53 94 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87
Complete report #21679
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa810aa2a
Build id: 3.10.0-862.14.4.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffa8079c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa8079c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa8079c90
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa810ab3f
Build id: 3.10.0-862.14.4.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa834edb5
Build id: 3.10.0-862.14.4.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa810b0eb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa8079e58
Build id: 3.10.0-862.14.4.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffa8079ff4
Build id: 3.10.0-862.14.4.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffa81dc0f7
Build id: 3.10.0-862.14.4.el7.x86_64 		0x67
11
wp_page_copy.isra.70
vmlinux 0xffffffffa81c49cb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x30b
12
do_wp_page
vmlinux 0xffffffffa81c6c3b
Build id: 3.10.0-862.14.4.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffffa8210f12
Build id: 3.10.0-862.14.4.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffffa81c85cd
Build id: 3.10.0-862.14.4.el7.x86_64 		0x37d
15
handle_mm_fault
vmlinux 0xffffffffa81caefd
Build id: 3.10.0-862.14.4.el7.x86_64 		0x39d
16
__do_page_fault
vmlinux 0xffffffffa8720547
Build id: 3.10.0-862.14.4.el7.x86_64 		0x197
17
do_page_fault
vmlinux 0xffffffffa87208d5
Build id: 3.10.0-862.14.4.el7.x86_64 		0x35
18
page_fault
vmlinux 0xffffffffa871c758
Build id: 3.10.0-862.14.4.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 29s! [nginx:1584]
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [thread_pool.rb*:30625]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_lsm_pinned_10703(E) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock falcon_lsm_pinned_10404(E) sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon nfit pcspkr joydev sg libnvdimm vmw_vmci shpchp i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel mptspi serio_raw libata scsi_transport_spi mptscsih mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_lsm_serviceable]
CPU: 3 PID: 30625 Comm: thread_pool.rb* Kdump: loaded Tainted: P            E  ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9b78c5b9cf10 ti: ffff9b7955f6c000 task.ti: ffff9b7955f6c000
RIP: 0010:[<ffffffffa810aa2a>]  [<ffffffffa810aa2a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff9b7955f6faf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9b7955f6fac0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff9b7955f6fb38 R08: ffffffffa88132a0 R09: fffff71888965840
R10: 00000000000043b5 R11: 00003ffffffff000 R12: 0000000000000001
R13: ffffffffa80beb31 R14: 0000000000000002 R15: ffff9b7cfffdb008
FS:  00007fac7c7ff700(0000) GS:ffff9b7cffcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fac6e53e3f0 CR3: 0000000007b14000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa8079c90>] ? leave_mm+0x110/0x110
[<ffffffffa8079c90>] ? leave_mm+0x110/0x110
[<ffffffffa8079c90>] ? leave_mm+0x110/0x110
[<ffffffffa810ab3f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa834edb5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa810b0eb>] smp_call_function_many+0x22b/0x270
[<ffffffffa8079e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa8079ff4>] flush_tlb_page+0x54/0xa0
[<ffffffffa81dc0f7>] ptep_clear_flush+0x67/0xa0
[<ffffffffa81c49cb>] wp_page_copy.isra.70+0x30b/0x560
[<ffffffffa81c6c3b>] do_wp_page+0xfb/0x720
[<ffffffffa8210f12>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffffa81c85cd>] handle_pte_fault+0x37d/0xd10
[<ffffffffa81caefd>] handle_mm_fault+0x39d/0x9b0
[<ffffffffa8720547>] __do_page_fault+0x197/0x4f0
[<ffffffffa87208d5>] do_page_fault+0x35/0x90
[<ffffffffa871c758>] page_fault+0x28/0x30
Code: c0 96 01 00 48 89 de 48 03 14 c5 a0 54 d3 a8 48 89 df e8 6a a1 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #12994
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa0f16f26
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffa0e7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa0e7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa0e7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa0f1703f
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa1184fc5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa0f175eb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa0e7e8f8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffa0e7ea94
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffa1007858
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffa0fef3a5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffffa0ff17cb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffffa103e262
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xe2
14
handle_mm_fault
vmlinux 0xffffffffa0ff5e62
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb22
15
__do_page_fault
vmlinux 0xffffffffa158d653
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x213
16
do_page_fault
vmlinux 0xffffffffa158d975
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35
17
page_fault
vmlinux 0xffffffffa1589778
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 28s! [docker-runc:81310]
Modules linked in: binfmt_misc veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat ip6table_filter ip6_tables xt_conntrack nf_conntrack iptable_filter sunrpc snd_seq_midi snd_seq_midi_event sb_edac snd_ens1371 snd_rawmidi snd_ac97_codec coretemp ac97_bus iosf_mbi snd_seq crc32_pclmul ghash_clmulni_intel aesni_intel snd_seq_device snd_pcm lrw gf128mul snd_timer glue_helper snd ablk_helper cryptd soundcore ppdev sg pcspkr joydev vmw_balloon parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c
sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix ttm libata mptspi scsi_transport_spi drm crct10dif_pclmul crct10dif_common mptscsih crc32c_intel e1000 mptbase serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod vfio_pci irqbypass vfio_iommu_type1 vfio
CPU: 6 PID: 81310 Comm: docker-runc Kdump: loaded Tainted: G             L ------------ T 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff91f4bedf8000 ti: ffff91f417c30000 task.ti: ffff91f417c30000
RIP: 0010:[<ffffffffa0f16f26>]  [<ffffffffa0f16f26>] generic_exec_single+0x106/0x1c0
RSP: 0000:ffff91f417c33b60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff91f417c33b30 RCX: 0000000000000008
RDX: ffffffffa1614850 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff91f417c33ba8 R08: ffffffffa1614848 R09: ffffea1b4af07800
R10: 00000000000043b2 R11: ffff91f5e9b6b1b8 R12: 0000000000000002
R13: ffff91f5fffdb008 R14: 0000000000000000 R15: 0000000300000010
FS:  00007f082b02b740(0000) GS:ffff91f5ed780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c420113000 CR3: 00000003004c6000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa0e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa0e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa0e7e730>] ? leave_mm+0x120/0x120
[<ffffffffa0f1703f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa1184fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa0f175eb>] smp_call_function_many+0x22b/0x270
[<ffffffffa0e7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa0e7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffa1007858>] ptep_clear_flush+0x68/0xa0
[<ffffffffa0fef3a5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffa0ff17cb>] do_wp_page+0xfb/0x720
[<ffffffffa103e262>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffffa0ff5e62>] handle_mm_fault+0xb22/0xfb0
[<ffffffffa158d653>] __do_page_fault+0x213/0x500
[<ffffffffa158d975>] do_page_fault+0x35/0x90
[<ffffffffa1589778>] page_fault+0x28/0x30
Code: 14 c5 60 10 b5 a1 48 89 df e8 17 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #8655
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8ed16f26
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffff8ec7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8ec7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8ec7e730
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8ed1703f
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8ef84fc5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8ed175eb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8ec7e8f8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff8ec7ea94
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff8ee07858
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffff8edef3a5
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x335
12
do_wp_page
vmlinux 0xffffffff8edf17cb
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffff8ee3e262
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xe2
14
handle_mm_fault
vmlinux 0xffffffff8edf5e62
Build id: 3.10.0-1127.13.1.el7.x86_64 		0xb22
15
__do_page_fault
vmlinux 0xffffffff8f38d653
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x213
16
trace_do_page_fault
vmlinux 0xffffffff8f38da26
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x56
17
do_async_page_fault
vmlinux 0xffffffff8f38cfa2
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x22
18
async_page_fault
vmlinux 0xffffffff8f3897a8
Build id: 3.10.0-1127.13.1.el7.x86_64 		0x28 
NMI watchdog: BUG: soft lockup - CPU#9 stuck for 23s! [java:14001]
Modules linked in: ip6_tables veth ipt_REJECT nf_reject_ipv4 vxlan ip6_udp_tunnel udp_tunnel xt_statistic xt_nat ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) sunrpc sb_edac iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper parport_pc cryptd virtio_balloon joydev parport i2c_piix4 pcspkr ip_tables xfs libcrc32c ata_generic pata_acpi virtio_net net_failover virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix virtio_pci crct10dif_pclmul
crct10dif_common floppy libata virtio_ring crc32c_intel serio_raw drm_panel_orientation_quirks virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 9 PID: 14001 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: OpenStack Foundation OpenStack Nova, BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
task: ffff93fed4765230 ti: ffff93ff3ede8000 task.ti: ffff93ff3ede8000
RIP: 0010:[<ffffffff8ed16f26>]  [<ffffffff8ed16f26>] generic_exec_single+0x106/0x1c0
RSP: 0000:ffff93ff3edebb30  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffff93ff3edebb00 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 0000000000000020 RDI: 0000000000000286
RBP: ffff93ff3edebb78 R08: ffffffff8f4172c0 R09: ffffdbe0f14f8840
R10: 00000000000050ad R11: ffff93ff66e7d368 R12: 0000000000000002
R13: ffff9401bffdb008 R14: 0000000000000000 R15: 0000000300000000
FS:  00007f6d297f1b10(0000) GS:ffff94013f040000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559460d55000 CR3: 0000001dcb8c2000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8ec7e730>] ? leave_mm+0x120/0x120
[<ffffffff8ec7e730>] ? leave_mm+0x120/0x120
[<ffffffff8ec7e730>] ? leave_mm+0x120/0x120
[<ffffffff8ed1703f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8ef84fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8ed175eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8ec7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ec7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffff8ee07858>] ptep_clear_flush+0x68/0xa0
[<ffffffff8edef3a5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffff8edf17cb>] do_wp_page+0xfb/0x720
[<ffffffff8ee3e262>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffff8edf5e62>] handle_mm_fault+0xb22/0xfb0
[<ffffffff8f38d653>] __do_page_fault+0x213/0x500
[<ffffffff8f38da26>] trace_do_page_fault+0x56/0x150
[<ffffffff8f38cfa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8f3897a8>] async_page_fault+0x28/0x30
Code: 14 c5 60 10 95 8f 48 89 df e8 17 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #771812
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa6d0779e
Build id: 3.10.0-862.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffffa6c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa6c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa6c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa6d078af
Build id: 3.10.0-862.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa6f4a275
Build id: 3.10.0-862.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa6d07e5b
Build id: 3.10.0-862.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa6c77018
Build id: 3.10.0-862.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffffa6c771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffffa6dd8837
Build id: 3.10.0-862.el7.x86_64 		0x67
11
wp_page_copy.isra.70
vmlinux 0xffffffffa6dc18be
Build id: 3.10.0-862.el7.x86_64 		0x29e
12
do_wp_page
vmlinux 0xffffffffa6dc3abb
Build id: 3.10.0-862.el7.x86_64 		0xfb
13
? __mem_cgroup_commit_charge
vmlinux 0xffffffffa6e0ca92
Build id: 3.10.0-862.el7.x86_64 		0xe2
14
handle_pte_fault
vmlinux 0xffffffffa6dc536d
Build id: 3.10.0-862.el7.x86_64 		0x36d 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [gopls:67909]
Modules linked in: xt_statistic xt_recent veth ip6t_MASQUERADE nf_nat_masquerade_ipv6 xt_nat vxlan ip6_udp_tunnel udp_tunnel xt_mark xt_multiport ip6table_mangle xt_comment ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip_vs_sh ip_vs_wrr ip_vs_rr ip_set ip_vs nf_conntrack_netlink nfnetlink xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) br_netfilter bridge stp llc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus
aesni_intel lrw snd_seq gf128mul glue_helper snd_seq_device ablk_helper cryptd ppdev snd_pcm snd_timer pcspkr snd joydev soundcore nfit sg vmw_vmci shpchp i2c_piix4 libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea crct10dif_pclmul crct10dif_common sysfillrect sysimgblt crc32c_intel fb_sys_fops ttm serio_raw ata_piix drm mptspi e1000 scsi_transport_spi libata mptscsih mptbase i2c_core
CPU: 1 PID: 67909 Comm: gopls Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff97b9197e0000 ti: ffff97b863e94000 task.ti: ffff97b863e94000
RIP: 0010:[<ffffffffa6d0779e>]  [<ffffffffa6d0779e>] generic_exec_single+0xfe/0x1b0
RSP: 0000:ffff97b863e97af0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff97b863e97ac0 RCX: 0000000000000001
RDX: ffffffffa7413170 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff97b863e97b38 R08: ffffffffa7413168 R09: ffffe34f4f2c6a80
R10: ffffe34f4f2c6a80 R11: 00003ffffffff000 R12: 0000000100000000
R13: ffffe34f447c5800 R14: ffffffffa6d9b2ce R15: 000280da00000000
FS:  00007f0a23fff700(0000) GS:ffff97b92d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c0040d0270 CR3: 00000001d36ba000 CR4: 00000000003607e0
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [gopls:67868]
Call Trace:
[<ffffffffa6c76e50>] ? leave_mm+0x110/0x110
[<ffffffffa6c76e50>] ? leave_mm+0x110/0x110
[<ffffffffa6c76e50>] ? leave_mm+0x110/0x110
[<ffffffffa6d078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa6f4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffa6d07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffa6c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa6c771b4>] flush_tlb_page+0x54/0xa0
[<ffffffffa6dd8837>] ptep_clear_flush+0x67/0xa0
[<ffffffffa6dc18be>] wp_page_copy.isra.70+0x29e/0x4f0
[<ffffffffa6dc3abb>] do_wp_page+0xfb/0x710
[<ffffffffa6e0ca92>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
[<ffffffffa6dc536d>] handle_pte_fault+0x36d/0xc30