Info

Permalink
Function
generic_exec_single
First occurrence
2020-09-30
Last occurrence
2026-02-21
State
NEW
Components
kernel
Quality
-41

Statistics

Operating system Count
CentOS 7 717
Architecture Count
x86_64 717
Related packages Count
kernel 717
 0:3.10.0-862.el7 310
 0:3.10.0-1160.59.1.el7 122
 0:3.10.0-1062.12.1.el7 106
 0:3.10.0-957.el7 67
 0:3.10.0-1160.el7 21
 0:3.10.0-1160.71.1.el7 20
 0:3.10.0-1160.45.1.el7 9
 0:3.10.0-1160.66.1.el7 9
 0:3.10.0-1062.el7 8
 0:3.10.0-1160.62.1.el7 7
 0:3.10.0-862.2.3.el7 5
 0:3.10.0-1062.4.1.el7 5
 0:3.10.0-1160.53.1.el7 5
 0:3.10.0-1160.25.1.el7 5
 0:3.10.0-1160.11.1.el7 4
 0:3.10.0-1062.18.1.el7 2
 0:3.10.0-1160.24.1.el7 2
 0:3.10.0-1160.36.2.el7 2
 0:3.10.0-862.11.6.el7 1
 0:3.10.0-862.6.3.el7 1
 0:3.10.0-1160.49.1.el7 1
 0:3.10.0-1062.9.1.el7 1
 0:3.10.0-1160.88.1.el7 1
 0:3.10.0-862.3.3.el7 1
 0:3.10.0-1160.76.1.el7 1
 0:3.10.0-1160.108.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #913850
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9871109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff9867a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9867a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9867a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff987111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff98975465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9871175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9867a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9867a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff987e4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff987e5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffff987e87c2
Build id: 3.10.0-957.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffff987e368d
Build id: 3.10.0-957.el7.x86_64 		0x49d
14
? system_call_after_swapgs
vmlinux 0xffffffff98d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffff98d74d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
16
system_call_fastpath
vmlinux 0xffffffff98d74ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
17
? system_call_after_swapgs
vmlinux 0xffffffff98d74d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 22s! [calico-node:24923]
Modules linked in: xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth iptable_raw xt_CT rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache xt_addrtype xt_set ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipport ip_set_hash_ipportip ip_set dummy ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 devlink tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev sg joydev parport_pc parport i2c_piix4 virtio_balloon pcspkr br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net cirrus drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix libata drm serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 24923 Comm: calico-node Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9496eac85140 ti: ffff94a5b0a04000 task.ti: ffff94a5b0a04000
RIP: 0010:[<ffffffff9871109a>]  [<ffffffff9871109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff94a5b0a07bf0  EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff94a5b0a07bc0 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000282
RBP: ffff94a5b0a07c38 R08: ffffffff98e12d80 R09: 00003ffffffff000
R10: ffff94a6212d1440 R11: 0000000000000246 R12: ffff94a67ff9e000
R13: ffffffff987c0c06 R14: ffff94a5b0a07c00 R15: 0000000000020200
FS:  000000c00007e090(0000) GS:ffff94a63ecc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c4236cb000 CR3: 0000000fb1b20000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff9867a3d0>] ? leave_mm+0x110/0x110
[<ffffffff987111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff98975465>] ? cpumask_next_and+0x35/0x50
[<ffffffff9871175b>] smp_call_function_many+0x22b/0x270
[<ffffffff9867a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9867a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff987e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff987e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff987e87c2>] zap_page_range+0x142/0x190
[<ffffffff987e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff98d74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff98d74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff98d74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff98d74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 34 99 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #826248
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffaed1109a
Build id: 3.10.0-957.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffaec7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffaec7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffaec7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffaed111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffaef75465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffaed1175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaec7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffaec7a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffaede4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffaede5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffffaede87c2
Build id: 3.10.0-957.el7.x86_64 		0x142
13
? handle_mm_fault
vmlinux 0xffffffffaedebc6d
Build id: 3.10.0-957.el7.x86_64 		0x39d
14
SyS_madvise
vmlinux 0xffffffffaede368d
Build id: 3.10.0-957.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffffaf374d21
Build id: 3.10.0-957.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffffaf374d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffffaf374ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
18
? system_call_after_swapgs
vmlinux 0xffffffffaf374d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 21s! [agent_service:18619]
Modules linked in: binfmt_misc sysmon_edr(OE) netfilter_edr(OE) ppdev sg i2c_piix4 joydev virtio_balloon parport_pc parport pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_console virtio_scsi virtio_net serio_raw cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 18619 Comm: agent_service Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff9cfd8e7c1040 ti: ffff9d00d023c000 task.ti: ffff9d00d023c000
RIP: 0010:[<ffffffffaed1109a>]  [<ffffffffaed1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9d00d023fbf0  EFLAGS: 00000202
RAX: ffff9d00d963c600 RBX: 00007fccc9f60000 RCX: ffff9d00d963c610
RDX: ffff9d00dec5b700 RSI: ffff9d00d023fbf0 RDI: ffff9d00d023fbf0
RBP: ffff9d00d023fc38 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9cfd7bb0be60 R11: 0000000000000206 R12: ffff9cfb5079b7a0
R13: ffffffffaedf6e08 R14: ffff9d00d023fb78 R15: ffffffffaedc588e
FS:  00007fccbf24d700(0000) GS:ffff9d00ded00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fccc86413a0 CR3: 000000081075c000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaec7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffaed111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffaef75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffaed1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffaec7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaec7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffaede4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffaede5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffaede87c2>] zap_page_range+0x142/0x190
[<ffffffffaedebc6d>] ? handle_mm_fault+0x39d/0x9b0
[<ffffffffaede368d>] SyS_madvise+0x49d/0xac0
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaf374d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaf374ddb>] system_call_fastpath+0x22/0x27
[<ffffffffaf374d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 94 af 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #746579
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff94b1109e
Build id: 3.10.0-957.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffff94a7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff94a7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff94a7a3d0
Build id: 3.10.0-957.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff94b111af
Build id: 3.10.0-957.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff94d75465
Build id: 3.10.0-957.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff94b1175b
Build id: 3.10.0-957.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff94a7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff94a7a608
Build id: 3.10.0-957.el7.x86_64 		0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff94be4077
Build id: 3.10.0-957.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff94be5975
Build id: 3.10.0-957.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffff94be87c2
Build id: 3.10.0-957.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffff94be368d
Build id: 3.10.0-957.el7.x86_64 		0x49d
14
? system_call_after_swapgs
vmlinux 0xffffffff95174d21
Build id: 3.10.0-957.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffff95174d15
Build id: 3.10.0-957.el7.x86_64 		0xa2
16
system_call_fastpath
vmlinux 0xffffffff95174ddb
Build id: 3.10.0-957.el7.x86_64 		0x22
17
? system_call_after_swapgs
vmlinux 0xffffffff95174d21
Build id: 3.10.0-957.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 29s! [java:26616]
Modules linked in: drbd_transport_tcp(OE) drbd(OE) tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter sunrpc sg pcspkr i2c_piix4 joydev virtio_balloon sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi virtio_gpu drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm virtio_scsi virtio_net drm_panel_orientation_quirks ata_piix libata floppy serio_raw virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 26616 Comm: java Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffff8f661bf20000 ti: ffff8f6a193f0000 task.ti: ffff8f6a193f0000
RIP: 0010:[<ffffffff94b1109e>]  [<ffffffff94b1109e>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff8f6a193f3bf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8f6a193f3bc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8f6a193f3c38 R08: ffffffff95213000 R09: 00003ffffffff000
R10: ffff8f6709b32510 R11: 00007f8b3baf1000 R12: ffff8f6a3ffce000
R13: ffffffff94bc0c06 R14: ffff8f6a193f3c00 R15: 0000000000020200
FS:  00007f8b3bbee700(0000) GS:ffff8f6a3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8b40287a78 CR3: 000000042e5f4000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94a7a3d0>] ? leave_mm+0x110/0x110
[<ffffffff94b111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff94d75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff94b1175b>] smp_call_function_many+0x22b/0x270
[<ffffffff94a7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff94a7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff94be4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff94be5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff94be87c2>] zap_page_range+0x142/0x190
[<ffffffff94be368d>] SyS_madvise+0x49d/0xac0
[<ffffffff95174d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff95174d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff95174ddb>] system_call_fastpath+0x22/0x27
[<ffffffff95174d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 89 de 48 03 14 c5 60 b9 74 95 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87
Complete report #579827
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa7316ae2
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa727ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffa727ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffa727ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffa7316bff
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa7586845
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa73171ab
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa727e258
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa727e2cb
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa73ee6a6
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa73f016f
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa73f0293
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffa73f375a
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xea
14
? kmem_cache_free
vmlinux 0xffffffffa7427022
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x1e2
15
SyS_madvise
vmlinux 0xffffffffa73edc4d
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x49d
16
? system_call_after_swapgs
vmlinux 0xffffffffa7999ed5
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffa7999ec9
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffffa7999f92
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffffa7999ed5
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [mysqld:27149]
Modules linked in: binfmt_misc ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod squashfs loop pcspkr edac_mce_amd joydev sg i2c_piix4 virtio_balloon ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_scsi virtio_net net_failover virtio_console failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix serio_raw libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy fuse
CPU: 1 PID: 27149 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
task: ffff8d70dafe5280 ti: ffff8d710c4b8000 task.ti: ffff8d710c4b8000
RIP: 0010:[<ffffffffa7316ae2>]  [<ffffffffa7316ae2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8d710c4bbbe0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8d710c4bbbb0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8d710c4bbc30 R08: ffffffffa7a13f40 R09: 00003ffffffff000
R10: ffff8d71f5b36000 R11: 00007f17ac9ae000 R12: ffff8d710c4bbcd0
R13: 0000000000000005 R14: ffff8d70ca9c4490 R15: ffffffffa7842c54
FS:  00007f17ac9f6700(0000) GS:ffff8d71ffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f17ac9eeff8 CR3: 0000000134b6a000 CR4: 00000000000007e0
Call Trace:
 [<ffffffffa727ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffa727ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffa727ddd0>] ? leave_mm+0x120/0x120
 [<ffffffffa7316bff>] smp_call_function_single+0x5f/0xa0
 [<ffffffffa7586845>] ? cpumask_next_and+0x35/0x50
 [<ffffffffa73171ab>] smp_call_function_many+0x22b/0x270
 [<ffffffffa727e258>] native_flush_tlb_others+0xb8/0xc0
 [<ffffffffa727e2cb>] flush_tlb_mm_range+0x6b/0x140
 [<ffffffffa73ee6a6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
 [<ffffffffa73f016f>] arch_tlb_finish_mmu+0x3f/0x80
 [<ffffffffa73f0293>] tlb_finish_mmu+0x23/0x30
 [<ffffffffa73f375a>] zap_page_range+0xea/0x130
 [<ffffffffa7427022>] ? kmem_cache_free+0x1e2/0x200
 [<ffffffffa73edc4d>] SyS_madvise+0x49d/0xac0
 [<ffffffffa7999ed5>] ? system_call_after_swapgs+0xa2/0x13a
 [<ffffffffa7999ec9>] ? system_call_after_swapgs+0x96/0x13a
 [<ffffffffa7999f92>] system_call_fastpath+0x25/0x2a
 [<ffffffffa7999ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 00 19 f5 a7 48 89 df e8 d7 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #546629
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa57163d2
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffa567e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa567e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa567e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa57164ef
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa5981685
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa5716a9b
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa567e478
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa567e4eb
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa57ec096
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa57edb5f
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa57edc83
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffa57f098a
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffa57eb69d
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffffa5d8de21
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffffa5d8de15
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffffa5d8dede
Build id: 3.10.0-1062.12.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffffa5d8de21
Build id: 3.10.0-1062.12.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 67s! [java:158508]
Modules linked in: tcp_lp fuse binfmt_misc xt_NFQUEUE nfnetlink_log nfnetlink tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter nf_conntrack_ipv6 nf_defrag_ipv6 rpcsec_gss_krb5 nf_conntrack_ipv4 auth_rpcgss nf_defrag_ipv4 nf_conntrack nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev sg i2c_piix4 parport_pc parport virtio_balloon joydev pcspkr ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci floppy virtio_ring serio_raw drm_panel_orientation_quirks
virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kbase]
CPU: 13 PID: 158508 Comm: java Kdump: loaded Tainted: GF          OEL ------------   3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff939c6af920e0 ti: ffff93aa6cf18000 task.ti: ffff93aa6cf18000
RIP: 0010:[<ffffffffa57163d2>]  [<ffffffffa57163d2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff93aa6cf1bbe0  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff93aa6cf1bbb0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff93aa6cf1bc30 R08: ffffffffa5e137c8 R09: 00003ffffffff000
R10: ffff93aa3e31cbd0 R11: 00007f4bf0861000 R12: ffffffffa58b753a
R13: ffff93aa6cf1bb90 R14: 0000000000001000 R15: ffffffffa57c9512
FS:  00007f4bf095e700(0000) GS:ffff93aa7d140000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4c5f6bb530 CR3: 00000004d6a78000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa567e2b0>] ? leave_mm+0x110/0x110
[<ffffffffa567e2b0>] ? leave_mm+0x110/0x110
[<ffffffffa567e2b0>] ? leave_mm+0x110/0x110
[<ffffffffa57164ef>] smp_call_function_single+0x5f/0xa0
[<ffffffffa5981685>] ? cpumask_next_and+0x35/0x50
[<ffffffffa5716a9b>] smp_call_function_many+0x22b/0x270
[<ffffffffa567e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa567e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa57ec096>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa57edb5f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa57edc83>] tlb_finish_mmu+0x23/0x30
[<ffffffffa57f098a>] zap_page_range+0xea/0x130
[<ffffffffa57eb69d>] SyS_madvise+0x49d/0xac0
[<ffffffffa5d8de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa5d8de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffa5d8dede>] system_call_fastpath+0x25/0x2a
[<ffffffffa5d8de21>] ? system_call_after_swapgs+0xae/0x146
Code: 89 de 48 03 14 c5 20 e8 34 a6 48 89 df e8 37 23 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #467668
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff87316b62
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8727ddd0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8727ddd0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8727ddd0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff87316c7f
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff87586835
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8731722b
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8727e258
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8727e2cb
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff873ee676
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff873f013f
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff873f0263
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff873f372a
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff873edc1d
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffff87999ed5
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xa2
16
? system_call_after_swapgs
vmlinux 0xffffffff87999ec9
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x96
17
system_call_fastpath
vmlinux 0xffffffff87999f92
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffff87999ed5
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [mysqld:3753]
Modules linked in: nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) vboxsf(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod vboxvideo(OE) ttm snd_intel8x0 snd_ac97_codec ext4 drm_kms_helper ac97_bus syscopyarea mbcache snd_seq sysfillrect jbd2 snd_seq_device sysimgblt ppdev joydev snd_pcm fb_sys_fops pcspkr
snd_timer video drm snd sg parport_pc soundcore vboxguest(OE) parport i2c_piix4 drm_panel_orientation_quirks ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi ahci libahci ata_piix crc32c_intel serio_raw libata e1000 fuse
CPU: 3 PID: 3753 Comm: mysqld Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff894778aee300 ti: ffff89464bfec000 task.ti: ffff89464bfec000
RIP: 0010:[<ffffffff87316b62>]  [<ffffffff87316b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff89464bfefbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 0000000000000001 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff89464bfefc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff894778208510 R11: 00007f97ac91b000 R12: ffff894778aee300
R13: ffff8945b4cab480 R14: 0000000000000002 R15: ffff894796d9ad40
FS:  00007f97ac964700(0000) GS:ffff894796d80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4634cb7c70 CR3: 000000020a2e4000 CR4: 00000000000206e0
Call Trace:
[<ffffffff8727ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8727ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8727ddd0>] ? leave_mm+0x120/0x120
[<ffffffff87316c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffff87586835>] ? cpumask_next_and+0x35/0x50
[<ffffffff8731722b>] smp_call_function_many+0x22b/0x270
[<ffffffff8727e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8727e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff873ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff873f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff873f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff873f372a>] zap_page_range+0xea/0x130
[<ffffffff873edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff87999ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff87999ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff87999f92>] system_call_fastpath+0x25/0x2a
[<ffffffff87999ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 f5 87 48 89 df e8 47 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #399761
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa3f07ace
Build id: 3.10.0-862.3.3.el7.x86_64 		0xfe
2
? leave_mm
vmlinux 0xffffffffa3e77110
Build id: 3.10.0-862.3.3.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa3e77110
Build id: 3.10.0-862.3.3.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa3e77110
Build id: 3.10.0-862.3.3.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa3f07bdf
Build id: 3.10.0-862.3.3.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa414a6b5
Build id: 3.10.0-862.3.3.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa3f0818b
Build id: 3.10.0-862.3.3.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa3e772d8
Build id: 3.10.0-862.3.3.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa3e77348
Build id: 3.10.0-862.3.3.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffa3fc07d7
Build id: 3.10.0-862.3.3.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa3fc1fb5
Build id: 3.10.0-862.3.3.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffffa3fc4d42
Build id: 3.10.0-862.3.3.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffffa3fbfe3d
Build id: 3.10.0-862.3.3.el7.x86_64 		0x3cd
14
? system_call_after_swapgs
vmlinux 0xffffffffa452077b
Build id: 3.10.0-862.3.3.el7.x86_64 		0xc8
15
? system_call_after_swapgs
vmlinux 0xffffffffa452076f
Build id: 3.10.0-862.3.3.el7.x86_64 		0xbc
16
system_call_fastpath
vmlinux 0xffffffffa452082f
Build id: 3.10.0-862.3.3.el7.x86_64 		0x1c
17
? system_call_after_swapgs
vmlinux 0xffffffffa452077b
Build id: 3.10.0-862.3.3.el7.x86_64 		0xc8 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [spades-hammer:5712]
Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxsf(OE) sunrpc vboxguest(OE) snd_intel8x0 vmwgfx snd_ac97_codec ttm ac97_bus ppdev drm_kms_helper snd_seq joydev syscopyarea snd_seq_device sysfillrect snd_pcm sysimgblt iosf_mbi fb_sys_fops snd_timer snd drm sg soundcore i2c_piix4 pcspkr parport_pc video parport i2c_core ip_tables
xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod crct10dif_common cdrom ata_generic pata_acpi ata_piix libata crc32c_intel serio_raw e1000 dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 5712 Comm: spades-hammer Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.3.3.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff957741033f40 ti: ffff957757b2c000 task.ti: ffff957757b2c000
RIP: 0010:[<ffffffffa3f07ace>]  [<ffffffffa3f07ace>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff957757b2fbf0  EFLAGS: 00010202
RAX: ffff9576aa145500 RBX: ffff95775ffd5800 RCX: ffff9576aa145510
RDX: ffff95775fc19680 RSI: ffff957757b2fbf0 RDI: ffff957757b2fbf0
RBP: ffff957757b2fc38 R08: 0000000000000001 R09: ffff95770e4d9af8
R10: 0000000000000006 R11: 0000000000000202 R12: 000280da00000000
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007fc9c03fd700(0000) GS:ffff95775fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc9c7c0c000 CR3: 0000000035c94000 CR4: 00000000000206e0
Call Trace:
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3e77110>] ? leave_mm+0x110/0x110
[<ffffffffa3f07bdf>] smp_call_function_single+0x5f/0xa0
[<ffffffffa414a6b5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa3f0818b>] smp_call_function_many+0x22b/0x270
[<ffffffffa3e772d8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa3e77348>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa3fc07d7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa3fc1fb5>] tlb_finish_mmu+0x55/0x60
[<ffffffffa3fc4d42>] zap_page_range+0x142/0x190
[<ffffffffa3fbfe3d>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa452077b>] ? system_call_after_swapgs+0xc8/0x160
[<ffffffffa452076f>] ? system_call_after_swapgs+0xbc/0x160
[<ffffffffa452082f>] system_call_fastpath+0x1c/0x21
[<ffffffffa452077b>] ? system_call_after_swapgs+0xc8/0x160
Code: 48 89 de 48 03 14 c5 e0 52 b3 a4 48 89 df e8 5a 88 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87
Complete report #362073
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8a516a32
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8a47ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8a47ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8a47ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff8a516b4f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8a786a45
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8a5170fb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8a47e258
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8a47e2cb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8a5ee676
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff8a5f013f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff8a5f0263
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff8a5f372a
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff8a5edc1d
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x49d
15
? __switch_to
vmlinux 0xffffffff8a42b981
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x4b1
16
? system_call_after_swapgs
vmlinux 0xffffffff8ab95ed5
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffff8ab95ec9
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffff8ab95f92
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff8ab95ed5
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [Cache2 I/O:4803]
Modules linked in: nls_utf8 isofs vboxsf(OE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_intel8x0 snd_ac97_codec ac97_bus snd_seq ppdev snd_seq_device joydev pcspkr iosf_mbi snd_pcm vboxguest(OE) snd_timer sg parport_pc video parport snd i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic crct10dif_common pata_acpi
hid_multitouch serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm ata_piix libahci drm_panel_orientation_quirks crc32c_intel e1000 libata dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 4803 Comm: Cache2 I/O Kdump: loaded Tainted: G           OE  ------------   3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff90bbd67b6300 ti: ffff90baae164000 task.ti: ffff90baae164000
RIP: 0010:[<ffffffff8a516a32>]  [<ffffffff8a516a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff90baae167be0  EFLAGS: 00010202
RAX: 00000000000008fc RBX: 00000000000000fb RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff90baae167c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff90baa9d40bd0 R11: 0000000000000206 R12: 0000000000000000
R13: 00000000000008fd R14: 0000000000000001 R15: 00003ffffffff000
FS:  00007f5d11b36700(0000) GS:ffff90bbdfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3c06a09000 CR3: 0000000144514000 CR4: 00000000000006e0
Call Trace:
[<ffffffff8a47ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8a47ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8a47ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8a516b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8a786a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff8a5170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff8a47e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8a47e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8a5ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8a5f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8a5f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff8a5f372a>] zap_page_range+0xea/0x130
[<ffffffff8a5edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff8a42b981>] ? __switch_to+0x4b1/0x580
[<ffffffff8ab95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff8ab95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff8ab95f92>] system_call_fastpath+0x25/0x2a
[<ffffffff8ab95ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 15 8b 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #285043
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff86716a32
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8667ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8667ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8667ddd0
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff86716b4f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff86986a45
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff867170fb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8667e258
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8667e2cb
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff867ee676
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff867f013f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff867f0263
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff867f372a
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffff867edc1d
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffff86d95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2
16
? system_call_after_swapgs
vmlinux 0xffffffff86d95ec9
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x96
17
system_call_fastpath
vmlinux 0xffffffff86d95f92
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffff86d95ed5
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [.NET ThreadPool:21362]
Modules linked in: tcp_lp nls_utf8 isofs nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) sunrpc snd_intel8x0 ppdev snd_ac97_codec ac97_bus snd_seq joydev snd_seq_device snd_pcm iosf_mbi snd_timer crc32_pclmul sg snd ghash_clmulni_intel vboxguest(OE) aesni_intel soundcore parport_pc parport lrw gf128mul i2c_piix4 glue_helper video ablk_helper pcspkr cryptd ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ata_piix ttm drm libata crct10dif_pclmul crct10dif_common crc32c_intel drm_panel_orientation_quirks e1000 serio_raw dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 21362 Comm: .NET ThreadPool Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8ad56c191080 ti: ffff8ad0c5818000 task.ti: ffff8ad0c5818000
RIP: 0010:[<ffffffff86716a32>]  [<ffffffff86716a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8ad0c581bbe0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8ad0c581bbb0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8ad0c581bc30 R08: ffffffff86e137c0 R09: 00003ffffffff000
R10: ffff8ad287296288 R11: 00007fea1efff000 R12: ffff8ad56c191080
R13: ffff8ad035d11a80 R14: 0000000000000140 R15: ffff8ad5fbc9ad40
FS:  00007fea1f7fe700(0000) GS:ffff8ad5fbc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1aa5fa4000 CR3: 00000005767a8000 CR4: 00000000000606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8667ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8667ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8667ddd0>] ? leave_mm+0x120/0x120
[<ffffffff86716b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff86986a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff867170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff8667e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8667e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff867ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff867f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff867f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff867f372a>] zap_page_range+0xea/0x130
[<ffffffff867edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff86d95ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff86d95ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff86d95f92>] system_call_fastpath+0x25/0x2a
[<ffffffff86d95ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 35 87 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #65135
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff98116802
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9807e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff9807e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff9807e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff9811691f
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff98385fe5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff98116ecb
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9807e868
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9807e8db
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff981edee6
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff981ef9af
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff981efad3
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff981f2f9a
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xea
14
? handle_mm_fault
vmlinux 0xffffffff981f5fed
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xbfd
15
SyS_madvise
vmlinux 0xffffffff981ed48d
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x49d
16
? system_call_after_swapgs
vmlinux 0xffffffff98793ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffff98793ec9
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffff98793f92
Build id: 3.10.0-1160.11.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff98793ed5
Build id: 3.10.0-1160.11.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 26s! [ossec-agentd:9009]
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix serio_raw mptspi scsi_transport_spi libata mptscsih e1000 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 9009 Comm: ossec-agentd Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1160.11.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff9cf166765280 ti: ffff9cf1664bc000 task.ti: ffff9cf1664bc000
RIP: 0010:[<ffffffff98116802>]  [<ffffffff98116802>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9cf1664bfbe0  EFLAGS: 00000202
RAX: 00000000000008fd RBX: 0000000000000020 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9cf1664bfc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9cf2c3ee76c8 R11: 0000000000000246 R12: ffffffff981c5d1e
R13: ffff9cf1664bfbb8 R14: ffff9cf2fffd6090 R15: fffff61d04de11c0
FS:  00007fd7f7323700(0000) GS:ffff9cf2ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd7ec03e040 CR3: 00000002241ce000 CR4: 00000000000407e0
Call Trace:
[<ffffffff9807e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9807e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9807e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9811691f>] smp_call_function_single+0x5f/0xa0
[<ffffffff98385fe5>] ? cpumask_next_and+0x35/0x50
[<ffffffff98116ecb>] smp_call_function_many+0x22b/0x270
[<ffffffff9807e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9807e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff981edee6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff981ef9af>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff981efad3>] tlb_finish_mmu+0x23/0x30
[<ffffffff981f2f9a>] zap_page_range+0xea/0x130
[<ffffffff981f5fed>] ? handle_mm_fault+0xbfd/0xfb0
[<ffffffff981ed48d>] SyS_madvise+0x49d/0xac0
[<ffffffff98793ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff98793ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff98793f92>] system_call_fastpath+0x25/0x2a
[<ffffffff98793ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 13 d5 98 48 89 df e8 57 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #24806
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffbbf14af6
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x106
2
? leave_mm
vmlinux 0xffffffffbbe7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffbbe7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffbbe7d2b0
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffbbf14c0f
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffbc17fce5
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffbbf151bb
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffbbe7d478
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffbbe7d4eb
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffbbfea756
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffbbfec21f
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffbbfec343
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffbbfef04a
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xea
14
SyS_madvise
vmlinux 0xffffffffbbfe9d5d
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x49d
15
? system_call_after_swapgs
vmlinux 0xffffffffbc58be21
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffffbc58be15
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffffbc58bede
Build id: 3.10.0-1062.4.1.el7.x86_64 		0x25
18
? system_call_after_swapgs
vmlinux 0xffffffffbc58be21
Build id: 3.10.0-1062.4.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [mysqld:121583]
pcc_cpufreq snd_timer snd soundcore i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit drm crc32c_intel libnvdimm ata_piix serio_raw mptspi scsi_transport_spi mptscsih libata e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 121583 Comm: mysqld Kdump: loaded Not tainted 3.10.0-1062.4.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff9f8b8a5cd230 ti: ffff9f8b91e04000 task.ti: ffff9f8b91e04000
RIP: 0010:[<ffffffffbbf14af6>]  [<ffffffffbbf14af6>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff9f8b91e07be0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9f8b91e07bb0 RCX: 0000000000000003
RDX: ffffffffbc613a50 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9f8b91e07c30 R08: ffffffffbc613a48 R09: 00003ffffffff000
R10: ffff9f8cea97bcb0 R11: 0000000000000202 R12: ffff9f8b8a5cd230
R13: 0000000000000002 R14: 0000000000000030 R15: 0000000000000005
FS:  00007faaaa3b4700(0000) GS:ffff9f8cf86c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007faa780c4000 CR3: 00000001733ce000 CR4: 00000000000007e0
Call Trace:
[<ffffffffbbe7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffbbe7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffbbe7d2b0>] ? leave_mm+0x110/0x110
[<ffffffffbbf14c0f>] smp_call_function_single+0x5f/0xa0
[<ffffffffbc17fce5>] ? cpumask_next_and+0x35/0x50
[<ffffffffbbf151bb>] smp_call_function_many+0x22b/0x270
[<ffffffffbbe7d478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbbe7d4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbbfea756>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbbfec21f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbbfec343>] tlb_finish_mmu+0x23/0x30
[<ffffffffbbfef04a>] zap_page_range+0xea/0x130
[<ffffffffbbfe9d5d>] SyS_madvise+0x49d/0xac0
[<ffffffffbc58be21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffbc58be15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffbc58bede>] system_call_fastpath+0x25/0x2a
[<ffffffffbc58be21>] ? system_call_after_swapgs+0xae/0x146
Code: 14 c5 e0 d6 b4 bc 48 89 df e8 77 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f
Complete report #3953
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa270779a
Build id: 3.10.0-862.2.3.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffa2676e60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffa2676e60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffa2676e60
Build id: 3.10.0-862.2.3.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffa27078af
Build id: 3.10.0-862.2.3.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa294a215
Build id: 3.10.0-862.2.3.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffa2707e5b
Build id: 3.10.0-862.2.3.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa2677028
Build id: 3.10.0-862.2.3.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa2677098
Build id: 3.10.0-862.2.3.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffa27c0437
Build id: 3.10.0-862.2.3.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa27c1c15
Build id: 3.10.0-862.2.3.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffffa27c49a2
Build id: 3.10.0-862.2.3.el7.x86_64 		0x142
13
SyS_madvise
vmlinux 0xffffffffa27bfa9d
Build id: 3.10.0-862.2.3.el7.x86_64 		0x3cd
14
? system_call_after_swapgs
vmlinux 0xffffffffa2d1f6e1
Build id: 3.10.0-862.2.3.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffffa2d1f6d5
Build id: 3.10.0-862.2.3.el7.x86_64 		0xa2
16
system_call_fastpath
vmlinux 0xffffffffa2d1f795
Build id: 3.10.0-862.2.3.el7.x86_64 		0x1c
17
? system_call_after_swapgs
vmlinux 0xffffffffa2d1f6e1
Build id: 3.10.0-862.2.3.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [mysqld:23520]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr nfit joydev vmw_balloon sg libnvdimm parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi scsi_transport_spi ata_piix mptscsih libata e1000 crct10dif_pclmul crct10dif_common mptbase crc32c_intel i2c_core serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 23520 Comm: mysqld Kdump: loaded Tainted: G             L ------------   3.10.0-862.2.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8b12ddefbf40 ti: ffff8b12eea8c000 task.ti: ffff8b12eea8c000
RIP: 0010:[<ffffffffa270779a>]  [<ffffffffa270779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8b12eea8fbf0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8b12eea8fbc0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8b12eea8fc38 R08: ffffffffa2e13920 R09: ffff8b12e1259288
R10: ffffdf04c670a3c0 R11: 0000000000000206 R12: ffff8b12ddefbf40
R13: ffff8b12ddefbf40 R14: 0000000000000002 R15: 0000000000000030
FS:  00007ff026e2bb00(0000) GS:ffff8b12ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005631f8c5e000 CR3: 0000000017cb4000 CR4: 00000000000607f0
Call Trace:
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa2676e60>] ? leave_mm+0x110/0x110
[<ffffffffa27078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa294a215>] ? cpumask_next_and+0x35/0x50
[<ffffffffa2707e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffa2677028>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2677098>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa27c0437>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa27c1c15>] tlb_finish_mmu+0x55/0x60
[<ffffffffa27c49a2>] zap_page_range+0x142/0x190
[<ffffffffa27bfa9d>] SyS_madvise+0x3cd/0x9c0
[<ffffffffa2d1f6e1>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffa2d1f6d5>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffa2d1f795>] system_call_fastpath+0x1c/0x21
[<ffffffffa2d1f6e1>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 33 a3 48 89 df e8 ea 86 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #774646
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff85f16a32
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff85e7ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff85e7ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff85e7ddd0
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff85f16b4f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff86186a45
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff85f170fb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff85e7e258
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff85e7e2cb
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff85fee676
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff85ff013f
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff85ff0263
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff85ff372a
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffff85edada8
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffff85fedc1d
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x49d
16
? __switch_to
vmlinux 0xffffffff85e2b621
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x151
17
? system_call_after_swapgs
vmlinux 0xffffffff86595ed5
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffff86595ec9
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x96
19
system_call_fastpath
vmlinux 0xffffffff86595f92
Build id: 3.10.0-1160.53.1.el7.x86_64 		0x25
20
? system_call_after_swapgs
vmlinux 0xffffffff86595ed5
Build id: 3.10.0-1160.53.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [mysqld:56338]
Modules linked in: xt_nat ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) binfmt_misc iptable_filter nls_utf8 isofs vmw_vsock_vmci_transport vsock raid0 coretemp crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod ata_piix crc_t10dif crct10dif_generic drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3
vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 56338 Comm: mysqld Tainted: G             L ------------ T 3.10.0-1160.53.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffff898f5e9da100 ti: ffff8990e44c0000 task.ti: ffff8990e44c0000
RIP: 0010:[<ffffffff85f16a32>]  [<ffffffff85f16a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8990e44c3be0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8990e44c3bb0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8990e44c3c30 R08: ffffffff86613a40 R09: 00003ffffffff000
R10: ffff898edd8115f0 R11: 0000000000000202 R12: ffff898f5e9da100
R13: 0000000300000048 R14: ffff8990e44c3cc0 R15: ffffffff85fc862f
FS:  00007f3b66fcd700(0000) GS:ffff8991ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd87fff8a08 CR3: 0000000330c12000 CR4: 00000000000007e0
Call Trace:
[<ffffffff85e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff85e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff85e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff85f16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff86186a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff85f170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff85e7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85e7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff85fee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff85ff013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff85ff0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff85ff372a>] zap_page_range+0xea/0x130
[<ffffffff85edada8>] ? wake_up_q+0x58/0x80
[<ffffffff85fedc1d>] SyS_madvise+0x49d/0xac0
[<ffffffff85e2b621>] ? __switch_to+0x151/0x580
[<ffffffff86595ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff86595ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff86595f92>] system_call_fastpath+0x25/0x2a
[<ffffffff86595ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 b5 86 48 89 df e8 87 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #525601
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff84116b62
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff8407ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffff8407ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffff8407ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffff84116c7f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff84386865
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff8411722b
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8407e258
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8407e2cb
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff841ee6c6
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff841f018f
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff841f02b3
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffff841f377a
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffff840daed8
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffff841edc6d
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x49d
16
? system_call_after_swapgs
vmlinux 0xffffffff84799ed5
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffff84799ec9
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffff84799f92
Build id: 3.10.0-1160.71.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff84799ed5
Build id: 3.10.0-1160.71.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [docker:5910]
Modules linked in: vhost_net vhost macvtap macvlan hci_vhci bluetooth vfio_iommu_type1 vfio uinput uhid ppp_generic slhc lp loop btrfs raid6_pq xor veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) tcp_lp tcp_diag inet_diag vboxsf(OE) xt_CHECKSUM iptable_mangle cfg80211 ipt_MASQUERADE nf_nat_masquerade_ipv4 rfkill iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_timer snd iosf_mbi joydev vboxguest(OE) sg video pcspkr soundcore parport_pc i2c_piix4 parport sch_fq_codel ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
crct10dif_common vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci serio_raw libahci drm drm_panel_orientation_quirks libata crc32c_intel e1000 dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 5910 Comm: docker Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff88e43f3a0000 ti: ffff88e440904000 task.ti: ffff88e440904000
RIP: 0010:[<ffffffff84116b62>]  [<ffffffff84116b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff88e440907be0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: ffffffff840e108e RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff88e440907c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff88e595a6ad80 R11: 0000000000000202 R12: 000003804e0d5fe7
R13: ffffffff840e495c R14: ffff88e440907b80 R15: 0000000000000087
FS:  00007fd16d54f700(0000) GS:ffff88e59fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f283f3d99d0 CR3: 00000001f5fc4000 CR4: 00000000000206e0
Call Trace:
[<ffffffff8407ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8407ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8407ddd0>] ? leave_mm+0x120/0x120
[<ffffffff84116c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffff84386865>] ? cpumask_next_and+0x35/0x50
[<ffffffff8411722b>] smp_call_function_many+0x22b/0x270
[<ffffffff8407e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8407e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff841ee6c6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff841f018f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff841f02b3>] tlb_finish_mmu+0x23/0x30
[<ffffffff841f377a>] zap_page_range+0xea/0x130
[<ffffffff840daed8>] ? wake_up_q+0x58/0x80
[<ffffffff841edc6d>] SyS_madvise+0x49d/0xac0
[<ffffffff84799ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff84799ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff84799f92>] system_call_fastpath+0x25/0x2a
[<ffffffff84799ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 d5 84 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #432697
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb9116b82
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb907ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb907ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb907ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb9116c9f
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb9386825
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb911724b
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb907e258
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb907e2cb
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb91ee666
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb91f012f
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb91f0253
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffb91f371a
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffffb90daef8
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffffb91edc0d
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x49d
16
? system_call_after_swapgs
vmlinux 0xffffffffb9799ed5
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffb9799ec9
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffffb9799f92
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffffb9799ed5
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [docker:17513]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter overlay(T) sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_timer snd soundcore sg pcspkr parport_pc iosf_mbi video i2c_piix4 parport ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea ahci sysfillrect sysimgblt fb_sys_fops ttm libahci ata_piix drm serio_raw crc32c_intel drm_panel_orientation_quirks libata e1000
dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 17513 Comm: docker Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9e0d6bda5280 ti: ffff9e0d83a60000 task.ti: ffff9e0d83a60000
RIP: 0010:[<ffffffffb9116b82>]  [<ffffffffb9116b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9e0d83a63be0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 00000000000000fb RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9e0d83a63c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9e0d9313c438 R11: 0000000000000202 R12: 0000000000000000
R13: 00000000000008fd R14: 0000000000000001 R15: 00003ffffffff000
FS:  00007fadebfff700(0000) GS:ffff9e0e1fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056034ff5e000 CR3: 00000000d75e6000 CR4: 00000000000206f0
Call Trace:
[<ffffffffb907ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb907ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb907ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb9116c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb9386825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb911724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb907e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb907e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb91ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb91f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb91f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffffb91f371a>] zap_page_range+0xea/0x130
[<ffffffffb90daef8>] ? wake_up_q+0x58/0x80
[<ffffffffb91edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffffb9799ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb9799ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb9799f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb9799ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 d5 b9 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #405345
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb2916b82
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb287ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb287ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb287ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb2916c9f
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb2b86825
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb291724b
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb287e258
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb287e2cb
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb29ee666
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb29f012f
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb29f0253
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffb29f371a
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffffb28daef8
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffffb29edc0d
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x49d
16
? wake_up_state
vmlinux 0xffffffffb28daf40
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x20
17
? system_call_after_swapgs
vmlinux 0xffffffffb2f99ed5
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffffb2f99ec9
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x96
19
tracesys
vmlinux 0xffffffffb2f9a226
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa6 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [containerd:4049]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter overlay(T) tcp_lp xt_owner xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter sunrpc joydev ppdev parport_pc vboxguest(OE) iosf_mbi i2c_piix4 sg video parport pcspkr binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx virtio_net net_failover failover drm_kms_helper ahci virtio_pci ata_piix libahci syscopyarea sysfillrect sysimgblt virtio_ring fb_sys_fops serio_raw ttm drm crc32c_intel virtio libata e1000 drm_panel_orientation_quirks
dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 4049 Comm: containerd Kdump: loaded Tainted: G        W  OE  ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff89e8b8085280 ti: ffff89e8d4f5c000 task.ti: ffff89e8d4f5c000
RIP: 0010:[<ffffffffb2916b82>]  [<ffffffffb2916b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff89e8d4f5fbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: ffff89e8d4f5fbd8 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff89e8d4f5fc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff89e8d4dfaaf8 R11: 0000000000000202 R12: ffff89e8d4f5fb60
R13: 0000000000000001 R14: ffff89e92069acc0 R15: ffffffffb28d3d6a
FS:  00007feb118e7700(0000) GS:ffff89e920680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7060785000 CR3: 0000000481a06000 CR4: 00000000000206e0
Call Trace:
[<ffffffffb287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb2916c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb2b86825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb291724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb287e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb287e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb29ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb29f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb29f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffffb29f371a>] zap_page_range+0xea/0x130
[<ffffffffb28daef8>] ? wake_up_q+0x58/0x80
[<ffffffffb29edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffffb28daf40>] ? wake_up_state+0x20/0x20
[<ffffffffb2f99ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb2f99ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb2f9a226>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 60 18 55 b3 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #397331
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb6916b82
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb687ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb687ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb687ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb6916c9f
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb6b86825
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb691724b
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb687e258
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb687e2cb
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb69ee666
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb69f012f
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb69f0253
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffb69f371a
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffffb68daef8
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffffb69edc0d
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x49d
16
? system_call_after_swapgs
vmlinux 0xffffffffb6f99ed5
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa2
17
? system_call_after_swapgs
vmlinux 0xffffffffb6f99ec9
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x96
18
system_call_fastpath
vmlinux 0xffffffffb6f99f92
Build id: 3.10.0-1160.62.1.el7.x86_64 		0x25
19
? system_call_after_swapgs
vmlinux 0xffffffffb6f99ed5
Build id: 3.10.0-1160.62.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [go:6288]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device ppdev snd_pcm snd_timer iosf_mbi snd pcspkr soundcore vboxguest(OE) sg i2c_piix4 joydev parport_pc parport video ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common vmwgfx drm_kms_helper ahci syscopyarea
sysfillrect sr_mod sysimgblt cdrom fb_sys_fops ttm ata_generic pata_acpi drm crc32c_intel serio_raw libahci ata_piix e1000 libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 6288 Comm: go Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9946443bb180 ti: ffff9946433e8000 task.ti: ffff9946433e8000
RIP: 0010:[<ffffffffb6916b82>]  [<ffffffffb6916b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9946433ebbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: ffffffffb68e10ae RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9946433ebc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9946435b7e58 R11: 0000000000000202 R12: 0000007c13cfc520
R13: ffffffffb68e497c R14: ffff9946433ebb80 R15: ffff99464439b180
FS:  00007fc1eccd3700(0000) GS:ffff99475fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000174a00e31c20 CR3: 000000000323a000 CR4: 00000000000206e0
Call Trace:
[<ffffffffb687ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb687ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb687ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb6916c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb6b86825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb691724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb687e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb687e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb69ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb69f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb69f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffffb69f371a>] zap_page_range+0xea/0x130
[<ffffffffb68daef8>] ? wake_up_q+0x58/0x80
[<ffffffffb69edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffffb6f99ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb6f99ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb6f99f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb6f99ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 55 b7 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #374937
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb8116a32
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffb807ddd0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffb807ddd0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffb807ddd0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffb8116b4f
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb8386a55
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffb81170fb
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb807e258
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb807e2cb
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb81ee676
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb81f013f
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb81f0263
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffb81f372a
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffffb80dada8
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffffb81edc1d
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x49d
16
? wake_up_state
vmlinux 0xffffffffb80dadf0
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x20
17
? system_call_after_swapgs
vmlinux 0xffffffffb8799ed5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffffb8799ec9
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x96
19
system_call_fastpath
vmlinux 0xffffffffb8799f92
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x25
20
? system_call_after_swapgs
vmlinux 0xffffffffb8799ed5
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [prometheus:770]
Modules linked in: tcp_lp vboxsf(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter tun sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer ppdev joydev sg snd parport_pc vboxguest(OE) parport soundcore pcspkr video i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci ttm ata_piix libahci libata drm serio_raw e1000 crc32c_intel drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 770 Comm: prometheus Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8f6291965280 ti: ffff8f628f5d8000 task.ti: ffff8f628f5d8000
RIP: 0010:[<ffffffffb8116a32>]  [<ffffffffb8116a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8f628f5dbbe0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 0000000000000000 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff8f628f5dbc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff8f5fb5409360 R11: 0000000000000202 R12: 0000000000000002
R13: ffff8f629fd1acc0 R14: ffff8f60f83f1c00 R15: ffff8f60f83f3800
FS:  000000c00006c890(0000) GS:ffff8f629fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f267111f268 CR3: 00000003109be000 CR4: 00000000000206f0
Call Trace:
[<ffffffffb807ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb807ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb807ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb8116b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb8386a55>] ? cpumask_next_and+0x35/0x50
[<ffffffffb81170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffb807e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb807e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb81ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb81f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb81f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffb81f372a>] zap_page_range+0xea/0x130
[<ffffffffb80dada8>] ? wake_up_q+0x58/0x80
[<ffffffffb81edc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffb80dadf0>] ? wake_up_state+0x20/0x20
[<ffffffffb8799ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb8799ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb8799f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb8799ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 d5 b8 48 89 df e8 97 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #366707
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffad916562
Build id: 3.10.0-1160.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffffad87e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
3
? leave_mm
vmlinux 0xffffffffad87e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
4
? leave_mm
vmlinux 0xffffffffad87e6a0
Build id: 3.10.0-1160.el7.x86_64 		0x120
5
smp_call_function_single
vmlinux 0xffffffffad91667f
Build id: 3.10.0-1160.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffadb85de5
Build id: 3.10.0-1160.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffad916c2b
Build id: 3.10.0-1160.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffad87e868
Build id: 3.10.0-1160.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffad87e8db
Build id: 3.10.0-1160.el7.x86_64 		0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffad9edd06
Build id: 3.10.0-1160.el7.x86_64 		0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffad9ef7cf
Build id: 3.10.0-1160.el7.x86_64 		0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffad9ef8f3
Build id: 3.10.0-1160.el7.x86_64 		0x23
13
zap_page_range
vmlinux 0xffffffffad9f2dba
Build id: 3.10.0-1160.el7.x86_64 		0xea
14
? wake_up_q
vmlinux 0xffffffffad8daaa8
Build id: 3.10.0-1160.el7.x86_64 		0x58
15
SyS_madvise
vmlinux 0xffffffffad9ed2ad
Build id: 3.10.0-1160.el7.x86_64 		0x49d
16
? wake_up_state
vmlinux 0xffffffffad8daaf0
Build id: 3.10.0-1160.el7.x86_64 		0x20
17
? system_call_after_swapgs
vmlinux 0xffffffffadf93ed5
Build id: 3.10.0-1160.el7.x86_64 		0xa2
18
? system_call_after_swapgs
vmlinux 0xffffffffadf93ec9
Build id: 3.10.0-1160.el7.x86_64 		0x96
19
system_call_fastpath
vmlinux 0xffffffffadf93f92
Build id: 3.10.0-1160.el7.x86_64 		0x25
20
? system_call_after_swapgs
vmlinux 0xffffffffadf93ed5
Build id: 3.10.0-1160.el7.x86_64 		0xa2 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [dockerd:3005]
Modules linked in: veth xt_nat nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc snd_hda_codec_idt snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer ppdev snd parport_pc soundcore joydev sg i2c_piix4 video parport pcspkr ip_tables
xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ahci libahci crc32c_intel libata serio_raw e1000 dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 3005 Comm: dockerd Kdump: loaded Tainted: G               ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9094c363b180 ti: ffff909598340000 task.ti: ffff909598340000
RIP: 0010:[<ffffffffad916562>]  [<ffffffffad916562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff909598343be0  EFLAGS: 00010202
RAX: 00000000000008fd RBX: 00000000000000fb RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff909598343c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9094d5177948 R11: 0000000000000202 R12: 0000000000000000
R13: 00000000000008fd R14: 0000000000000001 R15: 00003ffffffff000
FS:  00007fce567fc700(0000) GS:ffff90959f500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9c66338e40 CR3: 0000000198130000 CR4: 00000000000006e0
Call Trace:
[<ffffffffad87e6a0>] ? leave_mm+0x120/0x120
[<ffffffffad87e6a0>] ? leave_mm+0x120/0x120
[<ffffffffad87e6a0>] ? leave_mm+0x120/0x120
[<ffffffffad91667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffadb85de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffad916c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffad87e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffad87e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffad9edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffad9ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffad9ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffad9f2dba>] zap_page_range+0xea/0x130
[<ffffffffad8daaa8>] ? wake_up_q+0x58/0x80
[<ffffffffad9ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffad8daaf0>] ? wake_up_state+0x20/0x20
[<ffffffffadf93ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffadf93ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffadf93f92>] system_call_fastpath+0x25/0x2a
[<ffffffffadf93ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 13 55 ae 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #199355
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff85d0779a
Build id: 3.10.0-862.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffff85c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff85c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff85c76e50
Build id: 3.10.0-862.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff85d078af
Build id: 3.10.0-862.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff85f4a275
Build id: 3.10.0-862.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff85d07e5b
Build id: 3.10.0-862.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff85c77018
Build id: 3.10.0-862.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff85c77088
Build id: 3.10.0-862.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffff85dc0497
Build id: 3.10.0-862.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffff85dc1c75
Build id: 3.10.0-862.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffff85dc4a02
Build id: 3.10.0-862.el7.x86_64 		0x142
13
? wake_up_q
vmlinux 0xffffffff85ccee38
Build id: 3.10.0-862.el7.x86_64 		0x58
14
SyS_madvise
vmlinux 0xffffffff85dbfafd
Build id: 3.10.0-862.el7.x86_64 		0x3cd
15
? system_call_after_swapgs
vmlinux 0xffffffff8631f721
Build id: 3.10.0-862.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffff8631f715
Build id: 3.10.0-862.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffff8631f7d5
Build id: 3.10.0-862.el7.x86_64 		0x1c
18
? system_call_after_swapgs
vmlinux 0xffffffff8631f721
Build id: 3.10.0-862.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [xfsaild/dm-0:545]
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [containerd-shim:27711]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel nfsv4 dns_resolver nfs ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 fscache rpcsec_gss_krb5 nf_conntrack_netlink br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype xt_set tun bridge stp llc xt_mark ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set nfnetlink dummy xt_comment iptable_nat nf_nat_ipv4 ebtable_filter ebtables nf_nat ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg parport_pc
parport vmw_balloon joydev pcspkr vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci ata_piix mptspi i2c_core libata scsi_transport_spi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptscsih vmxnet3 mptbase dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 27711 Comm: containerd-shim Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff949967f9eeb0 ti: ffff9498af738000 task.ti: ffff9498af738000
RIP: 0010:[<ffffffff85d0779a>]  [<ffffffff85d0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9498af73bbf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9498af73bbc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff9498af73bc38 R08: ffffffff86413b60 R09: ffffdf82d0948340
R10: ffffdf82d0948340 R11: 0000000000000246 R12: ffff949967f9eeb0
R13: ffff949967f9eeb0 R14: ffff9498af73bba0 R15: 0000000000000001
FS:  000000c420031890(0000) GS:ffff94997fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c420091010 CR3: 00000003ee3f8000 CR4: 00000000000007f0
Call Trace:
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85d078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff85f4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff85d07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff85c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85c77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff85dc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff85dc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff85dc4a02>] zap_page_range+0x142/0x190
[<ffffffff85ccee38>] ? wake_up_q+0x58/0x80
[<ffffffff85dbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff8631f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8631f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8631f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff8631f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 93 86 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #16333
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff92916612
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9287e440
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9287e440
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9287e440
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff9291672f
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff92b81845
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff92916cdb
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9287e608
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9287e67b
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x6b
10
zap_page_range
vmlinux 0xffffffff929f0d28
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xd8
11
? wake_up_q
vmlinux 0xffffffff928db5b8
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x58
12
SyS_madvise
vmlinux 0xffffffff929eb22d
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x49d
13
? wake_up_state
vmlinux 0xffffffff928db600
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x20
14
? system_call_after_swapgs
vmlinux 0xffffffff92f8de21
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xae
15
? system_call_after_swapgs
vmlinux 0xffffffff92f8de15
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xa2
16
system_call_fastpath
vmlinux 0xffffffff92f8dede
Build id: 3.10.0-1062.18.1.el7.x86_64 		0x25
17
? system_call_after_swapgs
vmlinux 0xffffffff92f8de21
Build id: 3.10.0-1062.18.1.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kubelet:2073]
Modules linked in: xt_statistic nf_conntrack_netlink nfnetlink xt_nat ipt_REJECT nf_reject_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc vmw_vsock_vmci_transport vsock overlay(T) sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq pcspkr ppdev vmw_balloon snd_seq_device snd_pcm snd_timer snd joydev soundcore sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit
crc32c_intel mptspi libnvdimm scsi_transport_spi mptscsih ata_piix e1000 mptbase serio_raw libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 2073 Comm: kubelet Kdump: loaded Tainted: G             L ------------ T 3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff91029bfce2a0 ti: ffff91029a810000 task.ti: ffff91029a810000
RIP: 0010:[<ffffffff92916612>]  [<ffffffff92916612>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff91029a813c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff91029a813c00 RCX: 0000000000000004
RDX: ffffffff93013cd0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff91029a813c80 R08: ffffffff93013cc8 R09: 00003ffffffff000
R10: ffff9102b60d41b0 R11: 0000000000000246 R12: ffff91029a813bc0
R13: 0000000000000000 R14: ffff9102bb69ac80 R15: ffffffff928de7f5
FS:  00007fd3a12c7700(0000) GS:ffff9102bb680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fce31bafab8 CR3: 0000000078962000 CR4: 00000000000007e0
Call Trace:
[<ffffffff9287e440>] ? leave_mm+0x110/0x110
[<ffffffff9287e440>] ? leave_mm+0x110/0x110
[<ffffffff9287e440>] ? leave_mm+0x110/0x110
[<ffffffff9291672f>] smp_call_function_single+0x5f/0xa0
[<ffffffff92b81845>] ? cpumask_next_and+0x35/0x50
[<ffffffff92916cdb>] smp_call_function_many+0x22b/0x270
[<ffffffff9287e608>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9287e67b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff929f0d28>] zap_page_range+0xd8/0x150
[<ffffffff928db5b8>] ? wake_up_q+0x58/0x80
[<ffffffff929eb22d>] SyS_madvise+0x49d/0xac0
[<ffffffff928db600>] ? wake_up_state+0x20/0x20
[<ffffffff92f8de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff92f8de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff92f8dede>] system_call_fastpath+0x25/0x2a
[<ffffffff92f8de21>] ? system_call_after_swapgs+0xae/0x146
Code: 89 de 48 03 14 c5 60 e8 54 93 48 89 df e8 b7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #12780
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffae70779a
Build id: 3.10.0-862.el7.x86_64 		0xfa
2
? leave_mm
vmlinux 0xffffffffae676e50
Build id: 3.10.0-862.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffffae676e50
Build id: 3.10.0-862.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffffae676e50
Build id: 3.10.0-862.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffffae7078af
Build id: 3.10.0-862.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffffae94a275
Build id: 3.10.0-862.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffffae707e5b
Build id: 3.10.0-862.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffae677018
Build id: 3.10.0-862.el7.x86_64 		0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffae677088
Build id: 3.10.0-862.el7.x86_64 		0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffae7c0497
Build id: 3.10.0-862.el7.x86_64 		0x37
11
tlb_finish_mmu
vmlinux 0xffffffffae7c1c75
Build id: 3.10.0-862.el7.x86_64 		0x55
12
zap_page_range
vmlinux 0xffffffffae7c4a02
Build id: 3.10.0-862.el7.x86_64 		0x142
13
? wake_up_q
vmlinux 0xffffffffae6cee38
Build id: 3.10.0-862.el7.x86_64 		0x58
14
SyS_madvise
vmlinux 0xffffffffae7bfafd
Build id: 3.10.0-862.el7.x86_64 		0x3cd
15
? system_call_after_swapgs
vmlinux 0xffffffffaed1f721
Build id: 3.10.0-862.el7.x86_64 		0xae
16
? system_call_after_swapgs
vmlinux 0xffffffffaed1f715
Build id: 3.10.0-862.el7.x86_64 		0xa2
17
system_call_fastpath
vmlinux 0xffffffffaed1f7d5
Build id: 3.10.0-862.el7.x86_64 		0x1c
18
? system_call_after_swapgs
vmlinux 0xffffffffaed1f721
Build id: 3.10.0-862.el7.x86_64 		0xae 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [dockerd:1566]
Modules linked in: gsch(OE) redirfs(OE) veth tcp_diag udp_diag inet_diag xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink dinet(OE) difile(OE) overlay(T) dsa_filter(POE) sunrpc ppdev sg virtio_balloon joydev parport_pc parport i2c_piix4 pcspkr nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata i2c_core virtio_pci virtio_ring serio_raw floppy virtio dm_mirror dm_region_hash
dm_log dm_mod
CPU: 2 PID: 1566 Comm: dockerd Kdump: loaded Tainted: P        W  OE  ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 1.11.0-2.el7 04/01/2014
task: ffff8becd4209fa0 ti: ffff8beccc2e0000 task.ti: ffff8beccc2e0000
RIP: 0010:[<ffffffffae70779a>]  [<ffffffffae70779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8beccc2e3bf0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8beccc2e3bc0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff8beccc2e3c38 R08: ffffffffaee13160 R09: ffff8bece23866c0
R10: 00007fffffffffff R11: 0000000000000246 R12: 00000000000000d9
R13: ffffffffae6d865c R14: ffff8beccc2e3ba0 R15: 0000000000000001
FS:  00007fd24d7fa700(0000) GS:ffff8bed3fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff861ffafb8 CR3: 00000007e56be000 CR4: 00000000000006e0
Call Trace:
[<ffffffffae676e50>] ? leave_mm+0x110/0x110
[<ffffffffae676e50>] ? leave_mm+0x110/0x110
[<ffffffffae676e50>] ? leave_mm+0x110/0x110
[<ffffffffae7078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffae94a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffae707e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffae677018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffae677088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffae7c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffae7c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffae7c4a02>] zap_page_range+0x142/0x190
[<ffffffffae6cee38>] ? wake_up_q+0x58/0x80
[<ffffffffae7bfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffffaed1f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffaed1f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffaed1f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffffaed1f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 33 af 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00