Complete report #1407075
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff8b51173a
Build id: 3.10.0-957.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8b47a598
Build id: 3.10.0-957.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8b47a608
Build id: 3.10.0-957.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff8b5e4077
Build id: 3.10.0-957.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8b5e5975
Build id: 3.10.0-957.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff8b5e87c2
Build id: 3.10.0-957.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff8b5e368d
Build id: 3.10.0-957.el7.x86_64
|
0x49d
|
|
8
|
?
poll_select_copy_remaining
|
vmlinux |
0xffffffff8b657044
Build id: 3.10.0-957.el7.x86_64
|
0x144
|
|
9
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8bb74d21
Build id: 3.10.0-957.el7.x86_64
|
0xae
|
|
10
|
tracesys
|
vmlinux |
0xffffffff8bb7505b
Build id: 3.10.0-957.el7.x86_64
|
0xa3
|
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [etcd:20969]
Modules linked in: xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel xt_addrtype xt_set veth ip_set_hash_ipportnet ip_set_hash_ipport ip_set_bitmap_port ip_set_hash_ipportip ip_set rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache dummy iptable_raw xt_CT ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) sunrpc ppdev sg virtio_balloon parport_pc parport i2c_piix4 joydev pcspkr br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack
ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_scsi virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata floppy serio_raw drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 13 PID: 20969 Comm: etcd Kdump: loaded Tainted: G L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff8c7eb4a14100 ti: ffff8c7e1f2e8000 task.ti: ffff8c7e1f2e8000
RIP: 0010:[<ffffffff8b51173a>] [<ffffffff8b51173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8c7e1f2ebc78 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff8c7ebea205a0
RDX: 0000000000000000 RSI: 0000000000000030 RDI: 0000000000000000
RBP: ffff8c7e1f2ebcb0 R08: ffff8c7029526400 R09: ffffffff8b7755c9
R10: ffff8c7ebed5f160 R11: fffff8ac3a74f400 R12: 0000000000000296
R13: ffff8c7e1f2ebc28 R14: 0000000000000001 R15: ffffffff8b61d189
FS: 000000c420072090(0000) GS:ffff8c7ebed40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5d6d40e008 CR3: 0000000f62e22000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8b47a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8b47a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8b5e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff8b5e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff8b5e87c2>] zap_page_range+0x142/0x190
[<ffffffff8b5e368d>] SyS_madvise+0x49d/0xac0
[<ffffffff8b657044>] ? poll_select_copy_remaining+0x144/0x180
[<ffffffff8bb74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8bb7505b>] tracesys+0xa3/0xc9
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 14 8c f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #187245
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
smp_call_function_many
|
vmlinux |
0xffffffff82d07e3a
Build id: 3.10.0-862.el7.x86_64
|
0x20a
|
|
2
|
native_flush_tlb_others
|
vmlinux |
0xffffffff82c77018
Build id: 3.10.0-862.el7.x86_64
|
0xb8
|
|
3
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff82c77088
Build id: 3.10.0-862.el7.x86_64
|
0x68
|
|
4
|
tlb_flush_mmu.part.75
|
vmlinux |
0xffffffff82dc0497
Build id: 3.10.0-862.el7.x86_64
|
0x37
|
|
5
|
tlb_finish_mmu
|
vmlinux |
0xffffffff82dc1c75
Build id: 3.10.0-862.el7.x86_64
|
0x55
|
|
6
|
zap_page_range
|
vmlinux |
0xffffffff82dc4a02
Build id: 3.10.0-862.el7.x86_64
|
0x142
|
|
7
|
SyS_madvise
|
vmlinux |
0xffffffff82dbfafd
Build id: 3.10.0-862.el7.x86_64
|
0x3cd
|
|
8
|
?
poll_select_copy_remaining
|
vmlinux |
0xffffffff82e30801
Build id: 3.10.0-862.el7.x86_64
|
0x121
|
|
9
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8331f721
Build id: 3.10.0-862.el7.x86_64
|
0xae
|
|
10
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8331f715
Build id: 3.10.0-862.el7.x86_64
|
0xa2
|
|
11
|
system_call_fastpath
|
vmlinux |
0xffffffff8331f7d5
Build id: 3.10.0-862.el7.x86_64
|
0x1c
|
|
12
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8331f721
Build id: 3.10.0-862.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 24s! [etcd:1396]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nf_conntrack_netlink br_netfilter xt_addrtype xt_set xt_mark ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport ip_set xt_CHECKSUM iptable_mangle nfnetlink dummy xt_comment ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables overlay(T) ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper parport_pc
parport sg ablk_helper pcspkr cryptd joydev vmw_balloon shpchp i2c_piix4 vmw_vmci sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix mptspi libahci scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 1396 Comm: etcd Kdump: loaded Tainted: G L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8ebce97d8fd0 ti: ffff8eb97a87c000 task.ti: ffff8eb97a87c000
RIP: 0010:[<ffffffff82d07e3a>] [<ffffffff82d07e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8eb97a87fc78 EFLAGS: 00000202
RAX: 0000000000000005 RBX: 00000000000000fc RCX: ffff8ebcffd5cd90
RDX: 0000000000000005 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff8eb97a87fcb0 R08: ffff8eba3a34f000 R09: ffffffff82f4a3d9
R10: ffff8ebcffc9bb00 R11: fffffb004fd18800 R12: 0000000000000296
R13: ffff8eb97a87fc28 R14: 00000000d5936a36 R15: ffffffff82df7ffa
FS: 000000c420062090(0000) GS:ffff8ebcffc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000c4201d7010 CR3: 00000004223e2000 CR4: 00000000000007e0
Call Trace:
[<ffffffff82c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff82c77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff82dc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff82dc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff82dc4a02>] zap_page_range+0x142/0x190
[<ffffffff82dbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff82e30801>] ? poll_select_copy_remaining+0x121/0x150
[<ffffffff8331f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8331f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8331f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff8331f721>] ? system_call_after_swapgs+0xae/0x146
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 93 83 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89