Info

Permalink
Function
generic_exec_single
First occurrence
2022-09-07
Last occurrence
2026-08-29
State
NEW
Components
kernel
Quality
-39

Statistics

Operating system Count
CentOS 7 26
Architecture Count
x86_64 26
Related packages Count
kernel 26
 0:3.10.0-957.el7 20
 0:3.10.0-514.el7 2
 0:3.10.0-862.el7 2
 0:3.10.0-957.1.3.el7 1
 0:3.10.0-1160.62.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #919838
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8851109a
Build id: 3.10.0-957.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8847a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff8847a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff8847a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff885111af
Build id: 3.10.0-957.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff88775465
Build id: 3.10.0-957.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8851175b
Build id: 3.10.0-957.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8847a598
Build id: 3.10.0-957.el7.x86_64
0xb8
9
flush_tlb_page
vmlinux 0xffffffff8847a734
Build id: 3.10.0-957.el7.x86_64
0x54
10
ptep_clear_flush
vmlinux 0xffffffff885fd587
Build id: 3.10.0-957.el7.x86_64
0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffff885e55bb
Build id: 3.10.0-957.el7.x86_64
0x30b
12
do_wp_page
vmlinux 0xffffffff885e782b
Build id: 3.10.0-957.el7.x86_64
0xfb
13
? __sigqueue_free.part.13
vmlinux 0xffffffff884abe93
Build id: 3.10.0-957.el7.x86_64
0x33
14
handle_pte_fault
vmlinux 0xffffffff885e91bd
Build id: 3.10.0-957.el7.x86_64
0x37d
15
? recalc_sigpending
vmlinux 0xffffffff884ac4ab
Build id: 3.10.0-957.el7.x86_64
0x1b
16
handle_mm_fault
vmlinux 0xffffffff885ebc6d
Build id: 3.10.0-957.el7.x86_64
0x39d
17
__do_page_fault
vmlinux 0xffffffff88b6f5e3
Build id: 3.10.0-957.el7.x86_64
0x203
18
do_page_fault
vmlinux 0xffffffff88b6f915
Build id: 3.10.0-957.el7.x86_64
0x35
19
page_fault
vmlinux 0xffffffff88b6b758
Build id: 3.10.0-957.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kubelet:19510]
Modules linked in: xt_multiport xt_nat veth nf_tables(T) vxlan ip6_udp_tunnel udp_tunnel ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipportnet ip_set_hash_ipport dummy iptable_mangle xt_comment xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_nat_ipv4 ceph libceph dns_resolver overlay(T) toa(OE) vmw_vsock_vmci_transport vsock bnep sunrpc dm_mirror dm_region_hash dm_log dm_mod crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev btusb btrtl btbcm btintel bluetooth rfkill snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus parport_pc parport snd_seq snd_seq_device snd_pcm snd_timer snd soundcore sg vmw_vmci i2c_piix4
nf_conntrack_ipv4 nf_defrag_ipv4 ipip tunnel4 ip_tunnel ipt_REJECT nf_reject_ipv4 ipt_rpfilter xt_set ip_set nfnetlink ip_vs_ftp nf_nat ip_vs_sed ip_vs_nq ip_vs_dh ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs_lblcr ip_vs_lblc ip_vs_lc ip_vs nf_conntrack br_netfilter bridge stp llc ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw nfit libnvdimm ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix e1000 libata mptspi scsi_transport_spi drm_panel_orientation_quirks mptscsih mptbase
CPU: 3 PID: 19510 Comm: kubelet Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff88ca5d079040 ti: ffff88ca72824000 task.ti: ffff88ca72824000
RIP: 0010:[<ffffffff8851109a>]  [<ffffffff8851109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff88ca72827ae0  EFLAGS: 00000202
RAX: ffff88ca58cdde00 RBX: 0000000100000000 RCX: ffff88ca58cdde10
RDX: ffff88caf961b700 RSI: ffff88ca72827ae0 RDI: ffff88ca72827ae0
RBP: ffff88ca72827b30 R08: 0000000000000001 R09: ffffceccc1685280
R10: 000000000000413e R11: 0000000000000001 R12: 0000000200000001
R13: 0000000000000002 R14: ffff88cafffd9008 R15: 0000000000000000
FS:  00007f4c3935a700(0000) GS:ffff88caf96c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4c286b1cf0 CR3: 00000000b0142000 CR4: 0000000000360fe0
Call Trace:
[<ffffffff8847a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8847a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8847a3d0>] ? leave_mm+0x110/0x110
[<ffffffff885111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff88775465>] ? cpumask_next_and+0x35/0x50
[<ffffffff8851175b>] smp_call_function_many+0x22b/0x270
[<ffffffff8847a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8847a734>] flush_tlb_page+0x54/0xa0
[<ffffffff885fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff885e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffff885e782b>] do_wp_page+0xfb/0x720
[<ffffffff884abe93>] ? __sigqueue_free.part.13+0x33/0x40
[<ffffffff885e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffff884ac4ab>] ? recalc_sigpending+0x1b/0x70
[<ffffffff885ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffff88b6f5e3>] __do_page_fault+0x203/0x500
[<ffffffff88b6f915>] do_page_fault+0x35/0x90
[<ffffffff88b6b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 89 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #759680
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb751109a
Build id: 3.10.0-957.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffb747a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffb747a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffb747a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffb75111af
Build id: 3.10.0-957.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb7775465
Build id: 3.10.0-957.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb751175b
Build id: 3.10.0-957.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb747a598
Build id: 3.10.0-957.el7.x86_64
0xb8
9
flush_tlb_page
vmlinux 0xffffffffb747a734
Build id: 3.10.0-957.el7.x86_64
0x54
10
ptep_clear_flush
vmlinux 0xffffffffb75fd587
Build id: 3.10.0-957.el7.x86_64
0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffffb75e55bb
Build id: 3.10.0-957.el7.x86_64
0x30b
12
do_wp_page
vmlinux 0xffffffffb75e782b
Build id: 3.10.0-957.el7.x86_64
0xfb
13
? unlock_page
vmlinux 0xffffffffb75b64fb
Build id: 3.10.0-957.el7.x86_64
0x2b
14
? do_read_fault.isra.61
vmlinux 0xffffffffb75e4879
Build id: 3.10.0-957.el7.x86_64
0x139
15
handle_pte_fault
vmlinux 0xffffffffb75e91bd
Build id: 3.10.0-957.el7.x86_64
0x37d
16
? inode_has_perm.isra.33.constprop.70
vmlinux 0xffffffffb7700d1a
Build id: 3.10.0-957.el7.x86_64
0x2a
17
handle_mm_fault
vmlinux 0xffffffffb75ebc6d
Build id: 3.10.0-957.el7.x86_64
0x39d
18
__do_page_fault
vmlinux 0xffffffffb7b6f5e3
Build id: 3.10.0-957.el7.x86_64
0x203
19
do_page_fault
vmlinux 0xffffffffb7b6f915
Build id: 3.10.0-957.el7.x86_64
0x35
20
? error_swapgs
vmlinux 0xffffffffb7b6ba96
Build id: 3.10.0-957.el7.x86_64
0xa7
21
page_fault
vmlinux 0xffffffffb7b6b758
Build id: 3.10.0-957.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [pgrep:9829]
Modules linked in: tq_base(OE) af_packet_diag netlink_diag vsock_diag unix_diag ipmi_devintf ipmi_msghandler xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables binfmt_misc fuse tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc ppdev coretemp crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr
sg parport_pc parport i2c_piix4 vmw_vmci xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common crc32c_intel serio_raw drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi mptscsih vmxnet3 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 4 PID: 9829 Comm: pgrep Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff981b5d7b1040 ti: ffff981b9b768000 task.ti: ffff981b9b768000
RIP: 0010:[<ffffffffb751109a>]  [<ffffffffb751109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff981b9b76bae0  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff981b9b76bab0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff981b9b76bb30 R08: ffffffffb7c13280 R09: fffffe5a48b90900
R10: 00000000000043b5 R11: 00003ffffffff000 R12: ffff981b873e7050
R13: 0000000000000002 R14: ffff981bbffcf008 R15: 0000000000000000
FS:  00007f8fb536c880(0000) GS:ffff981bbfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8fb40337c0 CR3: 00000003e765c000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb747a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb747a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb747a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb75111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb7775465>] ? cpumask_next_and+0x35/0x50
[<ffffffffb751175b>] smp_call_function_many+0x22b/0x270
[<ffffffffb747a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb747a734>] flush_tlb_page+0x54/0xa0
[<ffffffffb75fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffffb75e55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffffb75e782b>] do_wp_page+0xfb/0x720
[<ffffffffb75b64fb>] ? unlock_page+0x2b/0x30
[<ffffffffb75e4879>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffffb75e91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffffb7700d1a>] ? inode_has_perm.isra.33.constprop.70+0x2a/0x30
[<ffffffffb75ebc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb7b6f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb7b6f915>] do_page_fault+0x35/0x90
[<ffffffffb7b6ba96>] ? error_swapgs+0xa7/0xbd
[<ffffffffb7b6b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 b8 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #546261
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb8316b82
Build id: 3.10.0-1160.62.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffb827ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffffb827ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffffb827ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffffb8316c9f
Build id: 3.10.0-1160.62.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb8586825
Build id: 3.10.0-1160.62.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb831724b
Build id: 3.10.0-1160.62.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb827e258
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb8
9
flush_tlb_page
vmlinux 0xffffffffb827e3f4
Build id: 3.10.0-1160.62.1.el7.x86_64
0x54
10
ptep_clear_flush
vmlinux 0xffffffffb84080f8
Build id: 3.10.0-1160.62.1.el7.x86_64
0x68
11
wp_page_copy.isra.73
vmlinux 0xffffffffb83efbf5
Build id: 3.10.0-1160.62.1.el7.x86_64
0x335
12
do_wp_page
vmlinux 0xffffffffb83f201b
Build id: 3.10.0-1160.62.1.el7.x86_64
0xfb
13
? ext4_file_read
ext4 0xffffffffc04aadc3
Build id: 3.10.0-1160.62.1.el7.x86_64
0x43
14
handle_mm_fault
vmlinux 0xffffffffb83f6692
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb22
15
? do_sync_read
vmlinux 0xffffffffb844d923
Build id: 3.10.0-1160.62.1.el7.x86_64
0x93
16
__do_page_fault
vmlinux 0xffffffffb8994653
Build id: 3.10.0-1160.62.1.el7.x86_64
0x213
17
do_page_fault
vmlinux 0xffffffffb8994975
Build id: 3.10.0-1160.62.1.el7.x86_64
0x35
18
? error_swapgs
vmlinux 0xffffffffb8990ac9
Build id: 3.10.0-1160.62.1.el7.x86_64
0xaa
19
page_fault
vmlinux 0xffffffffb8990778
Build id: 3.10.0-1160.62.1.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [libx2go-server-:6877]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink xt_multiport nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod ppdev parport_pc sg parport i2c_piix4 virtio_balloon pcspkr joydev nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_net
net_failover failover virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci floppy serio_raw virtio_ring virtio drm_panel_orientation_quirks fuse
CPU: 3 PID: 6877 Comm: libx2go-server- Kdump: loaded Not tainted 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5.1-0-g8936dbb-20141113_115728-nilsson.home.kraxel.org 04/01/2014
task: ffff95c52f4fd280 ti: ffff95c4ea794000 task.ti: ffff95c4ea794000
RIP: 0010:[<ffffffffb8316b82>]  [<ffffffffb8316b82>] generic_exec_single+0x102/0x1c0
RSP: 0000:ffff95c4ea797b60  EFLAGS: 00000202
RAX: 0000000000000006 RBX: ffff95c4ea797b30 RCX: 000000000000003a
RDX: 000000000000003f RSI: 0000000000000006 RDI: 0000000000000286
RBP: ffff95c4ea797ba8 R08: ffffffffb8a137c0 R09: ffffe0dd06907800
R10: 0000000000006946 R11: ffff95c52f6380d8 R12: 0000000000000246
R13: ffffffffffffffff R14: 0000000000000002 R15: 0000000000000030
FS:  00007f2303166740(0000) GS:ffff95c57fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001e12538 CR3: 00000001aa774000 CR4: 00000000000006e0
Call Trace:
[<ffffffffb827ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb827ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb827ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb8316c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb8586825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb831724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb827e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb827e3f4>] flush_tlb_page+0x54/0xa0
[<ffffffffb84080f8>] ptep_clear_flush+0x68/0xa0
[<ffffffffb83efbf5>] wp_page_copy.isra.73+0x335/0x5b0
[<ffffffffb83f201b>] do_wp_page+0xfb/0x720
[<ffffffffc04aadc3>] ? ext4_file_read+0x43/0x130 [ext4]
[<ffffffffb83f6692>] handle_mm_fault+0xb22/0xfb0
[<ffffffffb844d923>] ? do_sync_read+0x93/0xe0
[<ffffffffb8994653>] __do_page_fault+0x213/0x500
[<ffffffffb8994975>] do_page_fault+0x35/0x90
[<ffffffffb8990ac9>] ? error_swapgs+0xaa/0xc0
[<ffffffffb8990778>] page_fault+0x28/0x30
Code: 89 de 48 03 14 c5 60 18 f5 b8 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #509600
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb4d1109a
Build id: 3.10.0-957.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffb4c7a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffb4c7a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffb4c7a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffb4d111af
Build id: 3.10.0-957.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb4f75465
Build id: 3.10.0-957.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb4d1175b
Build id: 3.10.0-957.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb4c7a598
Build id: 3.10.0-957.el7.x86_64
0xb8
9
flush_tlb_page
vmlinux 0xffffffffb4c7a734
Build id: 3.10.0-957.el7.x86_64
0x54
10
ptep_clear_flush
vmlinux 0xffffffffb4dfd587
Build id: 3.10.0-957.el7.x86_64
0x67
11
wp_page_copy.isra.71
vmlinux 0xffffffffb4de55bb
Build id: 3.10.0-957.el7.x86_64
0x30b
12
do_wp_page
vmlinux 0xffffffffb4de782b
Build id: 3.10.0-957.el7.x86_64
0xfb
13
? unlock_page
vmlinux 0xffffffffb4db64fb
Build id: 3.10.0-957.el7.x86_64
0x2b
14
? do_read_fault.isra.61
vmlinux 0xffffffffb4de4879
Build id: 3.10.0-957.el7.x86_64
0x139
15
handle_pte_fault
vmlinux 0xffffffffb4de91bd
Build id: 3.10.0-957.el7.x86_64
0x37d
16
? recalc_sigpending
vmlinux 0xffffffffb4cac4ab
Build id: 3.10.0-957.el7.x86_64
0x1b
17
? __set_task_blocked
vmlinux 0xffffffffb4cace21
Build id: 3.10.0-957.el7.x86_64
0x41
18
handle_mm_fault
vmlinux 0xffffffffb4debc6d
Build id: 3.10.0-957.el7.x86_64
0x39d
19
__do_page_fault
vmlinux 0xffffffffb536f5e3
Build id: 3.10.0-957.el7.x86_64
0x203
20
do_page_fault
vmlinux 0xffffffffb536f915
Build id: 3.10.0-957.el7.x86_64
0x35
21
page_fault
vmlinux 0xffffffffb536b758
Build id: 3.10.0-957.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [PassengerHelper:24080]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev vmw_balloon sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ext4 mbcache jbd2 snd_ens1371 joydev snd_rawmidi snd_ac97_codec ac97_bus snd_seq
snd_seq_device pcspkr snd_pcm sg snd_timer snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel nfit libnvdimm ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt serio_raw fb_sys_fops ttm ata_piix mptspi scsi_transport_spi mptscsih drm libata e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 24080 Comm: PassengerHelper Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff89c9b8eb2080 ti: ffff89cd553d4000 task.ti: ffff89cd553d4000
RIP: 0010:[<ffffffffb4d1109a>]  [<ffffffffb4d1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff89cd553d7ae0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff89cd553d7ab0 RCX: 0000000000000002
RDX: ffffffffb5413010 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff89cd553d7b30 R08: ffffffffb5413008 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff89cdbd6559a0
R13: 0000000000000002 R14: ffff89cdfff9f008 R15: 0000000000000000
FS:  00007f67f62ac700(0000) GS:ffff89cdbd680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000911b20 CR3: 0000000bf8a96000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb4c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb4c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb4c7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb4d111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb4f75465>] ? cpumask_next_and+0x35/0x50
[<ffffffffb4d1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffb4c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb4c7a734>] flush_tlb_page+0x54/0xa0
[<ffffffffb4dfd587>] ptep_clear_flush+0x67/0xa0
[<ffffffffb4de55bb>] wp_page_copy.isra.71+0x30b/0x560
[<ffffffffb4de782b>] do_wp_page+0xfb/0x720
[<ffffffffb4db64fb>] ? unlock_page+0x2b/0x30
[<ffffffffb4de4879>] ? do_read_fault.isra.61+0x139/0x1b0
[<ffffffffb4de91bd>] handle_pte_fault+0x37d/0xd10
[<ffffffffb4cac4ab>] ? recalc_sigpending+0x1b/0x70
[<ffffffffb4cace21>] ? __set_task_blocked+0x41/0xa0
[<ffffffffb4debc6d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffb536f5e3>] __do_page_fault+0x203/0x500
[<ffffffffb536f915>] do_page_fault+0x35/0x90
[<ffffffffb536b758>] page_fault+0x28/0x30
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 94 b5 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #1178078
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffaa90779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffaa876e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffaa876e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffaa876e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffaa9078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffaab4a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffaa907e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaa877018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_page
vmlinux 0xffffffffaa8771b4
Build id: 3.10.0-862.el7.x86_64
0x54
10
ptep_clear_flush
vmlinux 0xffffffffaa9d8837
Build id: 3.10.0-862.el7.x86_64
0x67
11
wp_page_copy.isra.70
vmlinux 0xffffffffaa9c18be
Build id: 3.10.0-862.el7.x86_64
0x29e
12
do_wp_page
vmlinux 0xffffffffaa9c3abb
Build id: 3.10.0-862.el7.x86_64
0xfb
13
? inet_recvmsg
vmlinux 0xffffffffaae6e750
Build id: 3.10.0-862.el7.x86_64
0x80
14
? sock_aio_read.part.10
vmlinux 0xffffffffaadcb9bc
Build id: 3.10.0-862.el7.x86_64
0x14c
15
handle_pte_fault
vmlinux 0xffffffffaa9c536d
Build id: 3.10.0-862.el7.x86_64
0x36d
16
? destroy_inode
vmlinux 0xffffffffaaa37f7b
Build id: 3.10.0-862.el7.x86_64
0x3b
17
handle_mm_fault
vmlinux 0xffffffffaa9c747d
Build id: 3.10.0-862.el7.x86_64
0x39d
18
? d_free
vmlinux 0xffffffffaaa32c20
Build id: 3.10.0-862.el7.x86_64
0x60
19
__do_page_fault
vmlinux 0xffffffffaaf1a597
Build id: 3.10.0-862.el7.x86_64
0x197
20
do_page_fault
vmlinux 0xffffffffaaf1a925
Build id: 3.10.0-862.el7.x86_64
0x35
21
page_fault
vmlinux 0xffffffffaaf16768
Build id: 3.10.0-862.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [cpolar:6928]
Modules linked in: tcp_lp nls_utf8 isofs fuse nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel ac97_bus
lrw gf128mul snd_seq glue_helper ablk_helper snd_seq_device ppdev cryptd vmw_balloon snd_pcm pcspkr snd_timer joydev snd sg soundcore i2c_piix4 shpchp vmw_vmci nfit libnvdimm parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic vmwgfx drm_kms_helper pata_acpi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi crct10dif_pclmul crct10dif_common crc32c_intel mptscsih libata i2c_core e1000 mptbase serio_raw
CPU: 3 PID: 6928 Comm: cpolar Kdump: loaded Tainted: G        W      ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8f6d9071af70 ti: ffff8f6d872b4000 task.ti: ffff8f6d872b4000
RIP: 0010:[<ffffffffaa90779a>]  [<ffffffffaa90779a>] generic_exec_single+0xfa/0x1b0
RSP: 0000:ffff8f6d872b7af0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8f6d872b7ac0 RCX: 0000000000000003
RDX: ffffffffab013670 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff8f6d872b7b38 R08: ffffffffab013668 R09: ffffe11480948bc0
R10: ffffe11480948bc0 R11: 00003ffffffff000 R12: 0000000100000000
R13: 000000000000000e R14: ffff8f6db271e200 R15: 000280da00000000
FS:  000000c000080090(0000) GS:ffff8f6db56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc477aefdc8 CR3: 00000001d9604000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffaa876e50>] ? leave_mm+0x110/0x110
[<ffffffffaa876e50>] ? leave_mm+0x110/0x110
[<ffffffffaa876e50>] ? leave_mm+0x110/0x110
[<ffffffffaa9078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffaab4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffaa907e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffaa877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaa8771b4>] flush_tlb_page+0x54/0xa0
[<ffffffffaa9d8837>] ptep_clear_flush+0x67/0xa0
[<ffffffffaa9c18be>] wp_page_copy.isra.70+0x29e/0x4f0
[<ffffffffaa9c3abb>] do_wp_page+0xfb/0x710
[<ffffffffaae6e750>] ? inet_recvmsg+0x80/0xb0
[<ffffffffaadcb9bc>] ? sock_aio_read.part.10+0x14c/0x170
[<ffffffffaa9c536d>] handle_pte_fault+0x36d/0xc30
[<ffffffffaaa37f7b>] ? destroy_inode+0x3b/0x60
[<ffffffffaa9c747d>] handle_mm_fault+0x39d/0x9b0
[<ffffffffaaa32c20>] ? d_free+0x60/0x70
[<ffffffffaaf1a597>] __do_page_fault+0x197/0x4f0
[<ffffffffaaf1a925>] do_page_fault+0x35/0x90
[<ffffffffaaf16768>] page_fault+0x28/0x30
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 53 ab 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #876470
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f925a
Build id: 3.10.0-514.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106c300
Build id: 3.10.0-514.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106c300
Build id: 3.10.0-514.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106c300
Build id: 3.10.0-514.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f935f
Build id: 3.10.0-514.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813179a5
Build id: 3.10.0-514.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f9903
Build id: 3.10.0-514.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106c4c8
Build id: 3.10.0-514.el7.x86_64
0xb8
9
flush_tlb_page
vmlinux 0xffffffff8106c6c5
Build id: 3.10.0-514.el7.x86_64
0x55
10
ptep_clear_flush
vmlinux 0xffffffff811c0d74
Build id: 3.10.0-514.el7.x86_64
0x54
11
wp_page_copy.isra.56
vmlinux 0xffffffff811acffb
Build id: 3.10.0-514.el7.x86_64
0x24b
12
do_wp_page
vmlinux 0xffffffff811aeba5
Build id: 3.10.0-514.el7.x86_64
0xe5
13
handle_mm_fault
vmlinux 0xffffffff811b0e3c
Build id: 3.10.0-514.el7.x86_64
0x72c
14
? d_free
vmlinux 0xffffffff81214d80
Build id: 3.10.0-514.el7.x86_64
0x60
15
__do_page_fault
vmlinux 0xffffffff81691a94
Build id: 3.10.0-514.el7.x86_64
0x154
16
do_page_fault
vmlinux 0xffffffff81691dc5
Build id: 3.10.0-514.el7.x86_64
0x35
17
page_fault
vmlinux 0xffffffff8168e088
Build id: 3.10.0-514.el7.x86_64
0x28
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [qemu-kvm:3724]
Modules linked in: vhost_net vhost macvtap macvlan nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter bridge stp llc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event intel_powerclamp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 snd_rawmidi aesni_intel snd_ac97_codec lrw ac97_bus vmw_balloon gf128mul glue_helper snd_seq ablk_helper cryptd snd_seq_device snd_pcm snd_timer pcspkr sg snd soundcore i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c
sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops serio_raw ttm drm ata_piix mptspi libata scsi_transport_spi i2c_core mptscsih e1000 mptbase fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 3724 Comm: qemu-kvm Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff88032de60000 ti: ffff8802e1bbc000 task.ti: ffff8802e1bbc000
RIP: 0010:[<ffffffff810f925a>]  [<ffffffff810f925a>] generic_exec_single+0xfa/0x1a0
RSP: 0000:ffff8802e1bbfbd0  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8802e1bbfba0 RCX: 0000000000000003
RDX: ffffffff816afeb0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8802e1bbfc18 R08: ffffffff816afea8 R09: ffffea000b6f4ac0
R10: 0000000000003ec4 R11: 8000000001c9d225 R12: fffffff900000000
R13: 0000000000000002 R14: 00000000e1bbfb90 R15: ffff88033ffd9008
FS:  00007fc3c5694880(0000) GS:ffff8803316c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd2594e1a8 CR3: 00000002e17c5000 CR4: 00000000003427e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106c300 ffff8802e1bbfca0
0000000000000003 000000009b7d9762 0000000000000002 ffffffff8106c300
ffffffff8106c300 ffff8802e1bbfc48 ffffffff810f935f ffff8802e1bbfc48
Call Trace:
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff8106c300>] ? leave_mm+0x70/0x70
[<ffffffff810f935f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813179a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f9903>] smp_call_function_many+0x223/0x260
[<ffffffff8106c4c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c6c5>] flush_tlb_page+0x55/0xa0
[<ffffffff811c0d74>] ptep_clear_flush+0x54/0x80
[<ffffffff811acffb>] wp_page_copy.isra.56+0x24b/0x470
[<ffffffff811aeba5>] do_wp_page+0xe5/0x530
[<ffffffff811b0e3c>] handle_mm_fault+0x72c/0xfe0
[<ffffffff81214d80>] ? d_free+0x60/0x70
[<ffffffff81691a94>] __do_page_fault+0x154/0x450
[<ffffffff81691dc5>] do_page_fault+0x35/0x90
[<ffffffff8168e088>] page_fault+0x28/0x30
Code: 80 77 01 00 48 89 de 48 03 14 c5 40 57 ad 81 48 89 df e8 ea 28 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00