Complete report #1109994
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffffba716562
Build id: 3.10.0-1160.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffffba67e6a0
Build id: 3.10.0-1160.el7.x86_64
|
0x120
|
|
3
|
|
vmlinux |
0xffffffffba67e6a0
Build id: 3.10.0-1160.el7.x86_64
|
0x120
|
|
4
|
|
vmlinux |
0xffffffffba67e6a0
Build id: 3.10.0-1160.el7.x86_64
|
0x120
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffffba71667f
Build id: 3.10.0-1160.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffffba985de5
Build id: 3.10.0-1160.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffffba716c2b
Build id: 3.10.0-1160.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffffba67e868
Build id: 3.10.0-1160.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffba67e8db
Build id: 3.10.0-1160.el7.x86_64
|
0x6b
|
|
10
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffba7edd06
Build id: 3.10.0-1160.el7.x86_64
|
0xa6
|
|
11
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffba7ef7cf
Build id: 3.10.0-1160.el7.x86_64
|
0x3f
|
|
12
|
tlb_finish_mmu
|
vmlinux |
0xffffffffba7ef8f3
Build id: 3.10.0-1160.el7.x86_64
|
0x23
|
|
13
|
zap_page_range
|
vmlinux |
0xffffffffba7f2dba
Build id: 3.10.0-1160.el7.x86_64
|
0xea
|
|
14
|
|
vmlinux |
0xffffffffba7fc9d8
Build id: 3.10.0-1160.el7.x86_64
|
0x378
|
|
15
|
SyS_madvise
|
vmlinux |
0xffffffffba7ed2ad
Build id: 3.10.0-1160.el7.x86_64
|
0x49d
|
|
16
|
|
vmlinux |
0xffffffffba7de1bb
Build id: 3.10.0-1160.el7.x86_64
|
0xeb
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffbad93ed5
Build id: 3.10.0-1160.el7.x86_64
|
0xa2
|
|
18
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffbad93ec9
Build id: 3.10.0-1160.el7.x86_64
|
0x96
|
|
19
|
system_call_fastpath
|
vmlinux |
0xffffffffbad93f92
Build id: 3.10.0-1160.el7.x86_64
|
0x25
|
|
20
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffbad93ed5
Build id: 3.10.0-1160.el7.x86_64
|
0xa2
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 67s! [java:62394]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables uio_ivshmem(OE) uio gsch(OE) redirfs(OE) bridge stp llc ip_set nfnetlink sunrpc ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr virtio_balloon sg i2c_piix4 parport_pc parport xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover virtio_console virtio_blk virtio_scsi failover cirrus drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ip_tables]
CPU: 6 PID: 62394 Comm: java Kdump: loaded Tainted: G W OEL ------------ 3.10.0-1160.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
task: ffff9bd406ef8000 ti: ffff9bd207aa8000 task.ti: ffff9bd207aa8000
RIP: 0010:[<ffffffffba716562>] [<ffffffffba716562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9bd207aabbe0 EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff9bd207aabbb0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff9bd207aabc30 R08: ffffffffbae14408 R09: 00003ffffffff000
R10: ffff9bceb5e5a1b0 R11: 00007fa9c8e5e000 R12: 00000000bac570b5
R13: fffff196d2c04c30 R14: 00007fa9c8e5e000 R15: 0000000180400022
FS: 00007fa9c8f5b700(0000) GS:ffff9bd69cf80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8440c78100 CR3: 00000006f0e36000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffba67e6a0>] ? leave_mm+0x120/0x120
[<ffffffffba67e6a0>] ? leave_mm+0x120/0x120
[<ffffffffba67e6a0>] ? leave_mm+0x120/0x120
[<ffffffffba71667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffba985de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffba716c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffba67e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffba67e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffba7edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffba7ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffba7ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffba7f2dba>] zap_page_range+0xea/0x130
[<ffffffffba7fc9d8>] ? do_mmap+0x378/0x530
[<ffffffffba7ed2ad>] SyS_madvise+0x49d/0xac0
[<ffffffffba7de1bb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffbad93ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffbad93ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffbad93f92>] system_call_fastpath+0x25/0x2a
[<ffffffffbad93ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 13 35 bb 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #451568
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffffb9516b82
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffffb947ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x120
|
|
3
|
|
vmlinux |
0xffffffffb947ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x120
|
|
4
|
|
vmlinux |
0xffffffffb947ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x120
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffffb9516c9f
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffffb9786825
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffffb951724b
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb947e258
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb947e2cb
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x6b
|
|
10
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffb95ee666
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xa6
|
|
11
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffb95f012f
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x3f
|
|
12
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb95f0253
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x23
|
|
13
|
zap_page_range
|
vmlinux |
0xffffffffb95f371a
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xea
|
|
14
|
|
vmlinux |
0xffffffffb95fd35c
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x39c
|
|
15
|
SyS_madvise
|
vmlinux |
0xffffffffb95edc0d
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x49d
|
|
16
|
|
vmlinux |
0xffffffffb95ded9b
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xeb
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb9b99ed5
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xa2
|
|
18
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb9b99ec9
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x96
|
|
19
|
system_call_fastpath
|
vmlinux |
0xffffffffb9b99f92
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0x25
|
|
20
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb9b99ed5
Build id: 3.10.0-1160.62.1.el7.x86_64
|
0xa2
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:30643]
Modules linked in: xt_multiport splxmod(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink scsi_transport_iscsi ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_multipath sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd pcspkr vmw_balloon joydev sg vmw_vmci parport_pc i2c_piix4 parport sch_fq_codel
ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops ttm libata mptspi drm scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih crc32c_intel serio_raw mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: splxmod]
CPU: 1 PID: 30643 Comm: java Kdump: loaded Tainted: G OEL ------------ 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
task: ffffa0e33e47e300 ti: ffffa0e385604000 task.ti: ffffa0e385604000
RIP: 0010:[<ffffffffb9516b82>] [<ffffffffb9516b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffffa0e385607be0 EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffffffffb94e10ae RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffffa0e385607c30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffffa0e53170bd80 R11: 00007f86fce94000 R12: 00000000895b59eb
R13: fffff75308d574b0 R14: 00007f86fce94000 R15: ffffa0e33e47e300
FS: 00007f86fcf91700(0000) GS:ffffa0e53fc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8722fcb380 CR3: 00000002340ba000 CR4: 00000000000407e0
Call Trace:
[<ffffffffb947ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb947ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb947ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb9516c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb9786825>] ? cpumask_next_and+0x35/0x50
[<ffffffffb951724b>] smp_call_function_many+0x22b/0x270
[<ffffffffb947e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb947e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb95ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb95f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb95f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffffb95f371a>] zap_page_range+0xea/0x130
[<ffffffffb95fd35c>] ? do_mmap+0x39c/0x590
[<ffffffffb95edc0d>] SyS_madvise+0x49d/0xac0
[<ffffffffb95ded9b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffb9b99ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb9b99ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb9b99f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb9b99ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 15 ba 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #373581
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff98b0779a
Build id: 3.10.0-862.el7.x86_64
|
0xfa
|
|
2
|
|
vmlinux |
0xffffffff98a76e50
Build id: 3.10.0-862.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffff98a76e50
Build id: 3.10.0-862.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffff98a76e50
Build id: 3.10.0-862.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff98b078af
Build id: 3.10.0-862.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff98d4a275
Build id: 3.10.0-862.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff98b07e5b
Build id: 3.10.0-862.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff98a77018
Build id: 3.10.0-862.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff98a77088
Build id: 3.10.0-862.el7.x86_64
|
0x68
|
|
10
|
tlb_flush_mmu.part.75
|
vmlinux |
0xffffffff98bc0497
Build id: 3.10.0-862.el7.x86_64
|
0x37
|
|
11
|
tlb_finish_mmu
|
vmlinux |
0xffffffff98bc1c75
Build id: 3.10.0-862.el7.x86_64
|
0x55
|
|
12
|
zap_page_range
|
vmlinux |
0xffffffff98bc4a02
Build id: 3.10.0-862.el7.x86_64
|
0x142
|
|
13
|
|
vmlinux |
0xffffffff98bcdf0c
Build id: 3.10.0-862.el7.x86_64
|
0x3ac
|
|
14
|
SyS_madvise
|
vmlinux |
0xffffffff98bbfafd
Build id: 3.10.0-862.el7.x86_64
|
0x3cd
|
|
15
|
|
vmlinux |
0xffffffff98bb35eb
Build id: 3.10.0-862.el7.x86_64
|
0xeb
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9911f721
Build id: 3.10.0-862.el7.x86_64
|
0xae
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9911f715
Build id: 3.10.0-862.el7.x86_64
|
0xa2
|
|
18
|
system_call_fastpath
|
vmlinux |
0xffffffff9911f7d5
Build id: 3.10.0-862.el7.x86_64
|
0x1c
|
|
19
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9911f721
Build id: 3.10.0-862.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [java:55596]
Modules linked in: af_packet_diag netlink_diag tcp_lp nls_utf8 isofs fuse binfmt_misc vsock_diag vsock sctp_diag sctp tcp_diag udp_diag inet_diag unix_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc ext4 mbcache jbd2 virtio_balloon sg ppdev parport_pc parport i2c_piix4 pcspkr joydev sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_scsi virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci i2c_core virtio_ring floppy virtio dm_mirror
dm_region_hash dm_log dm_mod
CPU: 6 PID: 55596 Comm: java Kdump: loaded Tainted: G L ------------ 3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff8d5ef723dee0 ti: ffff8d59ceb84000 task.ti: ffff8d59ceb84000
RIP: 0010:[<ffffffff98b0779a>] [<ffffffff98b0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8d59ceb87bf0 EFLAGS: 00000202
RAX: 0000000000000060 RBX: ffff8d59ceb87bc0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 0000000000000060 RDI: 0000000000000282
RBP: ffff8d59ceb87c38 R08: ffffffff992133e8 R09: ffff8d57b45e3008
R10: 0000000000000008 R11: 00007fd4deedc000 R12: ffff8d5708029000
R13: 00000000f384c646 R14: 00007fd4deedc000 R15: ffff8d5ef723dee0
FS: 00007fd4defd9700(0000) GS:ffff8d5f1df80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559723373dc8 CR3: 0000000010a58000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff98a76e50>] ? leave_mm+0x110/0x110
[<ffffffff98a76e50>] ? leave_mm+0x110/0x110
[<ffffffff98a76e50>] ? leave_mm+0x110/0x110
[<ffffffff98b078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff98d4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff98b07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff98a77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff98a77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff98bc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff98bc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff98bc4a02>] zap_page_range+0x142/0x190
[<ffffffff98bcdf0c>] ? do_mmap+0x3ac/0x4d0
[<ffffffff98bbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff98bb35eb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff9911f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9911f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff9911f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff9911f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 73 99 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #317916
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffffabf1109a
Build id: 3.10.0-957.el7.x86_64
|
0xfa
|
|
2
|
|
vmlinux |
0xffffffffabe7a3d0
Build id: 3.10.0-957.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffffabe7a3d0
Build id: 3.10.0-957.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffffabe7a3d0
Build id: 3.10.0-957.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffffabf111af
Build id: 3.10.0-957.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffffac175465
Build id: 3.10.0-957.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffffabf1175b
Build id: 3.10.0-957.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffffabe7a598
Build id: 3.10.0-957.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffabe7a608
Build id: 3.10.0-957.el7.x86_64
|
0x68
|
|
10
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffffabfe4077
Build id: 3.10.0-957.el7.x86_64
|
0x37
|
|
11
|
tlb_finish_mmu
|
vmlinux |
0xffffffffabfe5975
Build id: 3.10.0-957.el7.x86_64
|
0x55
|
|
12
|
zap_page_range
|
vmlinux |
0xffffffffabfe87c2
Build id: 3.10.0-957.el7.x86_64
|
0x142
|
|
13
|
|
vmlinux |
0xffffffffabff28f8
Build id: 3.10.0-957.el7.x86_64
|
0x378
|
|
14
|
SyS_madvise
|
vmlinux |
0xffffffffabfe368d
Build id: 3.10.0-957.el7.x86_64
|
0x49d
|
|
15
|
|
vmlinux |
0xffffffffabfd612b
Build id: 3.10.0-957.el7.x86_64
|
0xeb
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffac574d21
Build id: 3.10.0-957.el7.x86_64
|
0xae
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffac574d15
Build id: 3.10.0-957.el7.x86_64
|
0xa2
|
|
18
|
system_call_fastpath
|
vmlinux |
0xffffffffac574ddb
Build id: 3.10.0-957.el7.x86_64
|
0x22
|
|
19
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffac574d21
Build id: 3.10.0-957.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [java:22032]
Modules linked in: ipt_REJECT nf_reject_ipv4 xt_comment nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_addrtype nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack iptable_filter sunrpc ext4 mbcache jbd2 ppdev sg pcspkr joydev virtio_balloon i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi serio_raw bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm floppy libahci ata_piix drm_panel_orientation_quirks e1000 libata virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 22032 Comm: java Kdump: loaded Tainted: G L ------------ 3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff93733fef4100 ti: ffff937192b20000 task.ti: ffff937192b20000
RIP: 0010:[<ffffffffabf1109a>] [<ffffffffabf1109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff937192b23bf0 EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff937192b23bc0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff937192b23c38 R08: ffffffffac613500 R09: 00003ffffffff000
R10: ffff937282700870 R11: 00007f3cf2e05000 R12: ffff937283549540
R13: 0000000082733aaa R14: 00007f3cf2e05000 R15: ffff93733fef4100
FS: 00007f3cf2e42700(0000) GS:ffff9373bfc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4d14099000 CR3: 000000009683c000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffabe7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffabe7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffabe7a3d0>] ? leave_mm+0x110/0x110
[<ffffffffabf111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffac175465>] ? cpumask_next_and+0x35/0x50
[<ffffffffabf1175b>] smp_call_function_many+0x22b/0x270
[<ffffffffabe7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffabe7a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffabfe4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffabfe5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffabfe87c2>] zap_page_range+0x142/0x190
[<ffffffffabff28f8>] ? do_mmap+0x378/0x530
[<ffffffffabfe368d>] SyS_madvise+0x49d/0xac0
[<ffffffffabfd612b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffac574d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffac574d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffac574ddb>] system_call_fastpath+0x22/0x27
[<ffffffffac574d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 b4 ac 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #251003
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffffa2b16a32
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffffa2a7ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x120
|
|
3
|
|
vmlinux |
0xffffffffa2a7ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x120
|
|
4
|
|
vmlinux |
0xffffffffa2a7ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x120
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffffa2b16b4f
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffffa2d86a25
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffffa2b170fb
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffffa2a7e258
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffa2a7e2cb
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x6b
|
|
10
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffa2bee676
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xa6
|
|
11
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffa2bf013f
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x3f
|
|
12
|
tlb_finish_mmu
|
vmlinux |
0xffffffffa2bf0263
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x23
|
|
13
|
zap_page_range
|
vmlinux |
0xffffffffa2bf372a
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xea
|
|
14
|
SyS_madvise
|
vmlinux |
0xffffffffa2bedc1d
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x49d
|
|
15
|
|
vmlinux |
0xffffffffa2bdedab
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xeb
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffa3195ed5
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xa2
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffa3195ec9
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x96
|
|
18
|
system_call_fastpath
|
vmlinux |
0xffffffffa3195f92
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0x25
|
|
19
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffa3195ed5
Build id: 3.10.0-1160.41.1.el7.x86_64
|
0xa2
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [qtp2076287037-3:3885]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon joydev pcspkr parport_pc sg parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper e1000 syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel mptspi serio_raw scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks
floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 3885 Comm: qtp2076287037-3 Kdump: loaded Not tainted 3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff962abe37e300 ti: ffff962a2c8d8000 task.ti: ffff962a2c8d8000
RIP: 0010:[<ffffffffa2b16a32>] [<ffffffffa2b16a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff962a2c8dbbe0 EFLAGS: 00000202
RAX: 00000000000008fd RBX: 0000000000000001 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff962a2c8dbc30 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9629647b41b0 R11: 00007f0ab5e3d000 R12: ffff962abe37e300
R13: fffffb31c26072b0 R14: 00007f0ab5e3d000 R15: 0000000100400006
FS: 00007f0ab5e79700(0000) GS:ffff962affc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8768900000 CR3: 00000000a9adc000 CR4: 00000000000007f0
Call Trace:
[<ffffffffa2a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa2a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa2a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa2b16b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa2d86a25>] ? cpumask_next_and+0x35/0x50
[<ffffffffa2b170fb>] smp_call_function_many+0x22b/0x270
[<ffffffffa2a7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2a7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa2bee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa2bf013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa2bf0263>] tlb_finish_mmu+0x23/0x30
[<ffffffffa2bf372a>] zap_page_range+0xea/0x130
[<ffffffffa2bedc1d>] SyS_madvise+0x49d/0xac0
[<ffffffffa2bdedab>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffa3195ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffa3195ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffa3195f92>] system_call_fastpath+0x25/0x2a
[<ffffffffa3195ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 75 a3 48 89 df e8 67 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #198529
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff8151117a
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xfa
|
|
2
|
|
vmlinux |
0xffffffff8147a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffff8147a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffff8147a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff8151128f
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff817756d5
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff8151183b
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8147a598
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8147a608
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x68
|
|
10
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff815e42e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x37
|
|
11
|
tlb_finish_mmu
|
vmlinux |
0xffffffff815e5be5
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x55
|
|
12
|
zap_page_range
|
vmlinux |
0xffffffff815e8a32
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x142
|
|
13
|
SyS_madvise
|
vmlinux |
0xffffffff815e38fd
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x49d
|
|
14
|
|
vmlinux |
0xffffffff815d639b
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xeb
|
|
15
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff81b74d21
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xae
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff81b74d15
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xa2
|
|
17
|
system_call_fastpath
|
vmlinux |
0xffffffff81b74ddb
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22
|
|
18
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff81b74d21
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:14023]
Modules linked in: vsock_diag tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw e1000e ptp pps_core vmwgfx ata_generic pata_acpi drm_kms_helper ahci syscopyarea sysfillrect sysimgblt
libahci fb_sys_fops ttm ata_piix nfit drm vmw_pvscsi libata libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 14023 Comm: java Kdump: loaded Tainted: G L ------------ 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
task: ffff9dc5e0eb9040 ti: ffff9dc5e4e30000 task.ti: ffff9dc5e4e30000
RIP: 0010:[<ffffffff8151117a>] [<ffffffff8151117a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9dc5e4e33bf0 EFLAGS: 00000202
RAX: ffff9dc35427f900 RBX: ffff9dc350412b70 RCX: ffff9dc35427f990
RDX: ffff9dc5ffc1b700 RSI: ffff9dc5e4e33bf0 RDI: ffff9dc5e4e33bf0
RBP: ffff9dc5e4e33c38 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff9dc35064d6c0 R11: 00007f6d456ee000 R12: ffff9dc5e4e33bb0
R13: 0000000000000001 R14: ffff9dc5ffd1ac00 R15: ffff9dc5e0eb90a8
FS: 00007f6d457eb700(0000) GS:ffff9dc5ffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4492a6ae00 CR3: 00000003a62e8000 CR4: 00000000000007e0
Call Trace:
[<ffffffff8147a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8147a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8147a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8151128f>] smp_call_function_single+0x5f/0xa0
[<ffffffff817756d5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8151183b>] smp_call_function_many+0x22b/0x270
[<ffffffff8147a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8147a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff815e42e7>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff815e5be5>] tlb_finish_mmu+0x55/0x60
[<ffffffff815e8a32>] zap_page_range+0x142/0x190
[<ffffffff815e38fd>] SyS_madvise+0x49d/0xac0
[<ffffffff815d639b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff81b74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff81b74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff81b74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff81b74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 82 48 89 df e8 aa b3 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #183463
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffffbad0aa2a
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xfa
|
|
2
|
|
vmlinux |
0xffffffffbac79c90
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffffbac79c90
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffffbac79c90
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffffbad0ab3f
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffffbaf4edb5
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffffbad0b0eb
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffffbac79e58
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffbac79ec8
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x68
|
|
10
|
tlb_flush_mmu.part.75
|
vmlinux |
0xffffffffbadc3487
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x37
|
|
11
|
tlb_finish_mmu
|
vmlinux |
0xffffffffbadc4d85
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x55
|
|
12
|
zap_page_range
|
vmlinux |
0xffffffffbadc7bd2
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x142
|
|
13
|
SyS_madvise
|
vmlinux |
0xffffffffbadc2add
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x3cd
|
|
14
|
|
vmlinux |
0xffffffffbadb63ab
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xeb
|
|
15
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffbb3256e1
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xae
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffbb3256d5
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xa2
|
|
17
|
system_call_fastpath
|
vmlinux |
0xffffffffbb32579b
Build id: 3.10.0-862.14.4.el7.x86_64
|
0x22
|
|
18
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffbb3256e1
Build id: 3.10.0-862.14.4.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [java:41660]
Modules linked in: tcp_lp rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel btusb btrtl btbcm btintel aesni_intel bluetooth lrw snd_ens1371 gf128mul glue_helper snd_rawmidi ablk_helper cryptd snd_ac97_codec ppdev ac97_bus snd_seq vmw_balloon snd_seq_device snd_pcm rfkill snd_timer pcspkr snd sg vmw_vmci joydev soundcore i2c_piix4 nfit shpchp libnvdimm parport_pc parport binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul drm crct10dif_common libata crc32c_intel mptspi e1000 scsi_transport_spi serio_raw mptscsih mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 41660 Comm: java Kdump: loaded Tainted: G L ------------ 3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffffa14698e7af70 ti: ffffa14673a30000 task.ti: ffffa14673a30000
RIP: 0010:[<ffffffffbad0aa2a>] [<ffffffffbad0aa2a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffffa14673a33bf0 EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffffa14673a33bc0 RCX: 0000000000000001
RDX: ffffffffbb4132b0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffffa14673a33c38 R08: ffffffffbb4132a8 R09: 00003ffffffff000
R10: ffffa1456534ea20 R11: 00007f8822269000 R12: ffffa146bffcc000
R13: ffffffffbada1626 R14: ffffa14673a33c00 R15: 0000000000020200
FS: 00007f8822366700(0000) GS:ffffa146ab700000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7df333e000 CR3: 000000009e988000 CR4: 00000000000607e0
Call Trace:
[<ffffffffbac79c90>] ? leave_mm+0x110/0x110
[<ffffffffbac79c90>] ? leave_mm+0x110/0x110
[<ffffffffbac79c90>] ? leave_mm+0x110/0x110
[<ffffffffbad0ab3f>] smp_call_function_single+0x5f/0xa0
[<ffffffffbaf4edb5>] ? cpumask_next_and+0x35/0x50
[<ffffffffbad0b0eb>] smp_call_function_many+0x22b/0x270
[<ffffffffbac79e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbac79ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbadc3487>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffbadc4d85>] tlb_finish_mmu+0x55/0x60
[<ffffffffbadc7bd2>] zap_page_range+0x142/0x190
[<ffffffffbadc2add>] SyS_madvise+0x3cd/0x9c0
[<ffffffffbadb63ab>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffbb3256e1>] ? system_call_after_swapgs+0xae/0x146
[<ffffffffbb3256d5>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffffbb32579b>] system_call_fastpath+0x22/0x27
[<ffffffffbb3256e1>] ? system_call_after_swapgs+0xae/0x146
Code: c0 96 01 00 48 89 de 48 03 14 c5 a0 54 93 bb 48 89 df e8 6a a1 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #15668
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffffb3f16f72
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffffb3e7e730
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x120
|
|
3
|
|
vmlinux |
0xffffffffb3e7e730
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x120
|
|
4
|
|
vmlinux |
0xffffffffb3e7e730
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x120
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffffb3f1708f
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffffb4185005
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffffb3f1763b
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb3e7e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb3e7e96b
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x6b
|
|
10
|
zap_page_range
|
vmlinux |
0xffffffffb3ff2ed8
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xd8
|
|
11
|
|
vmlinux |
0xffffffffb3ffcb28
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x378
|
|
12
|
SyS_madvise
|
vmlinux |
0xffffffffb3fed3ed
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x49d
|
|
13
|
|
vmlinux |
0xffffffffb3fde63b
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xeb
|
|
14
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb4592e15
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xa2
|
|
15
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb4592e09
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x96
|
|
16
|
system_call_fastpath
|
vmlinux |
0xffffffffb4592ed2
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x25
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb4592e15
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xa2
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [qtp366590980-21:17425]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod sg virtio_balloon joydev pcspkr i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_net net_failover failover virtio_blk bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks
floppy
CPU: 0 PID: 17425 Comm: qtp366590980-21 Not tainted 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org 04/01/2014
task: ffff9a94f458e2a0 ti: ffff9a9357924000 task.ti: ffff9a9357924000
RIP: 0010:[<ffffffffb3f16f72>] [<ffffffffb3f16f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9a9357927c30 EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff9a9357927c00 RCX: 0000000000000830
RDX: 0000000000000002 RSI: 0000000000000002 RDI: 0000000000000286
RBP: ffff9a9357927c80 R08: ffffffffb4613940 R09: 00007fd87e58a000
R10: 0000000000000008 R11: 0000000000000206 R12: ffffffffb3fcdd35
R13: ffff9a9357927ba8 R14: ffff9a9357927c88 R15: ffff9a9357927c88
FS: 00007fd87e5c6700(0000) GS:ffff9a94ffc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fafe5c76000 CR3: 00000000b682c000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb3e7e730>] ? leave_mm+0x120/0x120
[<ffffffffb3e7e730>] ? leave_mm+0x120/0x120
[<ffffffffb3e7e730>] ? leave_mm+0x120/0x120
[<ffffffffb3f1708f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb4185005>] ? cpumask_next_and+0x35/0x50
[<ffffffffb3f1763b>] smp_call_function_many+0x22b/0x270
[<ffffffffb3e7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb3e7e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb3ff2ed8>] zap_page_range+0xd8/0x150
[<ffffffffb3ffcb28>] ? do_mmap+0x378/0x530
[<ffffffffb3fed3ed>] SyS_madvise+0x49d/0xab0
[<ffffffffb3fde63b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffffb4592e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb4592e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb4592ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffb4592e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 a0 10 b5 b4 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #6753
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff9e1163c2
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffff9e07e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffff9e07e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffff9e07e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff9e1164df
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff9e381635
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff9e116a8b
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff9e07e478
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9e07e4eb
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x6b
|
|
10
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffff9e1ec026
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xa6
|
|
11
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffff9e1edaef
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x3f
|
|
12
|
tlb_finish_mmu
|
vmlinux |
0xffffffff9e1edc13
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x23
|
|
13
|
zap_page_range
|
vmlinux |
0xffffffff9e1f091a
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xea
|
|
14
|
|
vmlinux |
0xffffffff9e1fad08
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x378
|
|
15
|
SyS_madvise
|
vmlinux |
0xffffffff9e1eb62d
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x49d
|
|
16
|
|
vmlinux |
0xffffffff9e1ddfdb
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xeb
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9e78de21
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xae
|
|
18
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9e78de15
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xa2
|
|
19
|
system_call_fastpath
|
vmlinux |
0xffffffff9e78dede
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0x25
|
|
20
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9e78de21
Build id: 3.10.0-1062.9.1.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [qtp1225197672-3:15107]
Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod ppdev aesni_intel lrw gf128mul sg glue_helper ablk_helper cryptd pcspkr joydev i2c_piix4 virtio_balloon parport_pc parport ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom virtio_net virtio_console virtio_scsi
ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm ata_piix libata drm_panel_orientation_quirks virtio_pci serio_raw virtio_ring virtio floppy
CPU: 2 PID: 15107 Comm: qtp1225197672-3 Tainted: G L ------------ 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
task: ffff9707ef4f5230 ti: ffff9706bbafc000 task.ti: ffff9706bbafc000
RIP: 0010:[<ffffffff9e1163c2>] [<ffffffff9e1163c2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9706bbaffbe0 EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff9706bbaffbb0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff9706bbaffc30 R08: ffffffff9e813540 R09: 00003ffffffff000
R10: ffff970aa2f7a798 R11: 00007fb77eeb1000 R12: 000000007fc1ad00
R13: fffffd88c96e36f0 R14: 00007fb77eeb1000 R15: 000000018040003c
FS: 00007fb77eeed700(0000) GS:ffff97057fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f562fb7c818 CR3: 00000002d669a000 CR4: 00000000000006e0
Call Trace:
[<ffffffff9e07e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9e07e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9e07e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9e1164df>] smp_call_function_single+0x5f/0xa0
[<ffffffff9e381635>] ? cpumask_next_and+0x35/0x50
[<ffffffff9e116a8b>] smp_call_function_many+0x22b/0x270
[<ffffffff9e07e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9e07e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9e1ec026>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9e1edaef>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9e1edc13>] tlb_finish_mmu+0x23/0x30
[<ffffffff9e1f091a>] zap_page_range+0xea/0x130
[<ffffffff9e1fad08>] ? do_mmap+0x378/0x530
[<ffffffff9e1eb62d>] SyS_madvise+0x49d/0xac0
[<ffffffff9e1ddfdb>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff9e78de21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9e78de15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff9e78dede>] system_call_fastpath+0x25/0x2a
[<ffffffff9e78de21>] ? system_call_after_swapgs+0xae/0x146
Code: 89 de 48 03 14 c5 20 e8 d4 9e 48 89 df e8 f7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #2210
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff8c71117a
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xfa
|
|
2
|
|
vmlinux |
0xffffffff8c67a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffff8c67a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffff8c67a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff8c71128f
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff8c9756d5
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff8c71183b
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8c67a598
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8c67a608
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x68
|
|
10
|
tlb_flush_mmu.part.76
|
vmlinux |
0xffffffff8c7e42e7
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x37
|
|
11
|
tlb_finish_mmu
|
vmlinux |
0xffffffff8c7e5be5
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x55
|
|
12
|
zap_page_range
|
vmlinux |
0xffffffff8c7e8a32
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x142
|
|
13
|
|
vmlinux |
0xffffffff8c7f2b68
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x378
|
|
14
|
SyS_madvise
|
vmlinux |
0xffffffff8c7e38fd
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x49d
|
|
15
|
|
vmlinux |
0xffffffff8c7d639b
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xeb
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8cd74d21
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xae
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8cd74d15
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xa2
|
|
18
|
system_call_fastpath
|
vmlinux |
0xffffffff8cd74ddb
Build id: 3.10.0-957.1.3.el7.x86_64
|
0x22
|
|
19
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff8cd74d21
Build id: 3.10.0-957.1.3.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:5110]
Modules linked in: nfring(OE) xt_iprange xt_NFQUEUE nfnetlink_log binfmt_misc xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype br_netfilter ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock ppdev sb_edac coretemp vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport
vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi vmxnet3 mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nfring]
CPU: 1 PID: 5110 Comm: java Kdump: loaded Tainted: G OEL ------------ T 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014
task: ffffa07a41456180 ti: ffffa07bfb1d0000 task.ti: ffffa07bfb1d0000
RIP: 0010:[<ffffffff8c71117a>] [<ffffffff8c71117a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffffa07bfb1d3bf0 EFLAGS: 00000202
RAX: ffffa0807fb5fc00 RBX: ffffa07bc1e1c4b0 RCX: ffffa0807fb5fce0
RDX: ffffa0817fc1b700 RSI: ffffa07bfb1d3bf0 RDI: ffffa07bfb1d3bf0
RBP: ffffa07bfb1d3c38 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffffa08060af36c8 R11: 00007f6edf305000 R12: ffffa07a97112500
R13: 00000000edcab7c0 R14: 00007f6edf305000 R15: ffffa07a41456180
FS: 00007f6edf33a700(0000) GS:ffffa0817fc40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6a40de802f CR3: 000000031eca6000 CR4: 00000000000607e0
Call Trace:
[<ffffffff8c67a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8c67a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8c67a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8c71128f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8c9756d5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8c71183b>] smp_call_function_many+0x22b/0x270
[<ffffffff8c67a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8c67a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8c7e42e7>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff8c7e5be5>] tlb_finish_mmu+0x55/0x60
[<ffffffff8c7e8a32>] zap_page_range+0x142/0x190
[<ffffffff8c7f2b68>] ? do_mmap+0x378/0x530
[<ffffffff8c7e38fd>] SyS_madvise+0x49d/0xac0
[<ffffffff8c7d639b>] ? vm_mmap_pgoff+0xeb/0x120
[<ffffffff8cd74d21>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff8cd74d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff8cd74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff8cd74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 34 8d 48 89 df e8 aa b3 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1088355
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff98116f62
Build id: 3.10.0-1127.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffff9807e730
Build id: 3.10.0-1127.el7.x86_64
|
0x120
|
|
3
|
|
vmlinux |
0xffffffff9807e730
Build id: 3.10.0-1127.el7.x86_64
|
0x120
|
|
4
|
|
vmlinux |
0xffffffff9807e730
Build id: 3.10.0-1127.el7.x86_64
|
0x120
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff9811707f
Build id: 3.10.0-1127.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff98384ee5
Build id: 3.10.0-1127.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff9811762b
Build id: 3.10.0-1127.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff9807e8f8
Build id: 3.10.0-1127.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9807e96b
Build id: 3.10.0-1127.el7.x86_64
|
0x6b
|
|
10
|
zap_page_range
|
vmlinux |
0xffffffff981f2e58
Build id: 3.10.0-1127.el7.x86_64
|
0xd8
|
|
11
|
|
vmlinux |
0xffffffff98112a60
Build id: 3.10.0-1127.el7.x86_64
|
0x90
|
|
12
|
SyS_madvise
|
vmlinux |
0xffffffff981ed36d
Build id: 3.10.0-1127.el7.x86_64
|
0x49d
|
|
13
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff98792e15
Build id: 3.10.0-1127.el7.x86_64
|
0xa2
|
|
14
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff98792e09
Build id: 3.10.0-1127.el7.x86_64
|
0x96
|
|
15
|
system_call_fastpath
|
vmlinux |
0xffffffff98792ed2
Build id: 3.10.0-1127.el7.x86_64
|
0x25
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff98792e15
Build id: 3.10.0-1127.el7.x86_64
|
0xa2
|
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [sdatad:24280]
Modules linked in: tcp_lp fuse sfs(OE) binfmt_misc nls_utf8 isofs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc xfs libcrc32c ppdev pcspkr joydev parport_pc sg parport virtio_balloon i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod cdrom virtio_blk virtio_net virtio_console virtio_scsi net_failover failover ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: dattobd]
CPU: 1 PID: 24280 Comm: sdatad Kdump: loaded Tainted: G OEL ------------ 3.10.0-1127.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9aefb4848000 ti: ffff9af59e86c000 task.ti: ffff9af59e86c000
RIP: 0010:[<ffffffff98116f62>] [<ffffffff98116f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9af59e86fc30 EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff9af59e86fc00 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000286
RBP: ffff9af59e86fc80 R08: ffffffff98813600 R09: 0000000001a4f000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000001
R13: ffff9af1aea5acc0 R14: ffff9af1aea5ad40 R15: ffff9aefb4848068
FS: 00007fbc9b7e6700(0000) GS:ffff9af1aea40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8f7bffe008 CR3: 000000056fdb4000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9807e730>] ? leave_mm+0x120/0x120
[<ffffffff9807e730>] ? leave_mm+0x120/0x120
[<ffffffff9807e730>] ? leave_mm+0x120/0x120
[<ffffffff9811707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff98384ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9811762b>] smp_call_function_many+0x22b/0x270
[<ffffffff9807e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9807e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff981f2e58>] zap_page_range+0xd8/0x150
[<ffffffff98112a60>] ? futex_wake+0x90/0x180
[<ffffffff981ed36d>] SyS_madvise+0x49d/0xab0
[<ffffffff98792e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff98792e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff98792ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff98792e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 0f d5 98 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1072176
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff9ab0779a
Build id: 3.10.0-862.el7.x86_64
|
0xfa
|
|
2
|
|
vmlinux |
0xffffffff9aa76e50
Build id: 3.10.0-862.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffff9aa76e50
Build id: 3.10.0-862.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffff9aa76e50
Build id: 3.10.0-862.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff9ab078af
Build id: 3.10.0-862.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff9ad4a275
Build id: 3.10.0-862.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff9ab07e5b
Build id: 3.10.0-862.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff9aa77018
Build id: 3.10.0-862.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9aa77088
Build id: 3.10.0-862.el7.x86_64
|
0x68
|
|
10
|
tlb_flush_mmu.part.75
|
vmlinux |
0xffffffff9abc0497
Build id: 3.10.0-862.el7.x86_64
|
0x37
|
|
11
|
tlb_finish_mmu
|
vmlinux |
0xffffffff9abc1c75
Build id: 3.10.0-862.el7.x86_64
|
0x55
|
|
12
|
zap_page_range
|
vmlinux |
0xffffffff9abc4a02
Build id: 3.10.0-862.el7.x86_64
|
0x142
|
|
13
|
|
vmlinux |
0xffffffff9aacee38
Build id: 3.10.0-862.el7.x86_64
|
0x58
|
|
14
|
|
vmlinux |
0xffffffff9ab033a0
Build id: 3.10.0-862.el7.x86_64
|
0x90
|
|
15
|
SyS_madvise
|
vmlinux |
0xffffffff9abbfafd
Build id: 3.10.0-862.el7.x86_64
|
0x3cd
|
|
16
|
|
vmlinux |
0xffffffff9aacee80
Build id: 3.10.0-862.el7.x86_64
|
0x20
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9b11f721
Build id: 3.10.0-862.el7.x86_64
|
0xae
|
|
18
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9b11f715
Build id: 3.10.0-862.el7.x86_64
|
0xa2
|
|
19
|
system_call_fastpath
|
vmlinux |
0xffffffff9b11f7d5
Build id: 3.10.0-862.el7.x86_64
|
0x1c
|
|
20
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff9b11f721
Build id: 3.10.0-862.el7.x86_64
|
0xae
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kubelet:1282]
Modules linked in: xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc edac_mce_amd crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg pcspkr joydev vmw_balloon nfit libnvdimm shpchp vmw_vmci parport_pc parport i2c_piix4 nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops crct10dif_pclmul crct10dif_common ttm ahci crc32c_intel drm serio_raw libahci ata_piix libata vmxnet3 vmw_pvscsi i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 1282 Comm: kubelet Tainted: G L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9ac237e26eb0 ti: ffff9ac107c88000 task.ti: ffff9ac107c88000
RIP: 0010:[<ffffffff9ab0779a>] [<ffffffff9ab0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9ac107c8bbf0 EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff9ac107c8bbc0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff9ac107c8bc38 R08: ffffffff9b213160 R09: ffff9ac19da5b1b0
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000091
R13: ffff9ac23fd98b80 R14: ffff9ac1bbb9b780 R15: 0000000000000001
FS: 00007efd8a2e4700(0000) GS:ffff9ac23fd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efd880ad0d8 CR3: 0000000003110000 CR4: 00000000000407e0
Call Trace:
[<ffffffff9aa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9aa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9aa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9ab078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9ad4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff9ab07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff9aa77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9aa77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9abc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff9abc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff9abc4a02>] zap_page_range+0x142/0x190
[<ffffffff9aacee38>] ? wake_up_q+0x58/0x80
[<ffffffff9ab033a0>] ? futex_wake+0x90/0x180
[<ffffffff9abbfafd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff9aacee80>] ? wake_up_state+0x20/0x20
[<ffffffff9b11f721>] ? system_call_after_swapgs+0xae/0x146
[<ffffffff9b11f715>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff9b11f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffff9b11f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 73 9b 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #485268
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffffb0b16b62
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffffb0a7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x120
|
|
3
|
|
vmlinux |
0xffffffffb0a7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x120
|
|
4
|
|
vmlinux |
0xffffffffb0a7ddd0
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x120
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffffb0b16c7f
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffffb0d86865
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffffb0b1722b
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffffb0a7e258
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffffb0a7e2cb
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x6b
|
|
10
|
tlb_flush_mmu_tlbonly
|
vmlinux |
0xffffffffb0bee6c6
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0xa6
|
|
11
|
arch_tlb_finish_mmu
|
vmlinux |
0xffffffffb0bf018f
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x3f
|
|
12
|
tlb_finish_mmu
|
vmlinux |
0xffffffffb0bf02b3
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x23
|
|
13
|
zap_page_range
|
vmlinux |
0xffffffffb0bf377a
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0xea
|
|
14
|
|
vmlinux |
0xffffffffb0b127a0
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x90
|
|
15
|
SyS_madvise
|
vmlinux |
0xffffffffb0bedc6d
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x49d
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb1199ed5
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0xa2
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb1199ec9
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x96
|
|
18
|
system_call_fastpath
|
vmlinux |
0xffffffffb1199f92
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0x25
|
|
19
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffffb1199ed5
Build id: 3.10.0-1160.71.1.el7.x86_64
|
0xa2
|
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [popen.rb:42:13111]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink arc4 md4 nls_utf8 cifs ccm dns_resolver sunrpc ppdev pcspkr virtio_balloon sg i2c_piix4 parport_pc parport joydev ip_tables xfs libcrc32c sr_mod cdrom virtio_net virtio_blk net_failover failover virtio_console virtio_scsi ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata serio_raw virtio_pci drm_panel_orientation_quirks virtio_ring floppy virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 13111 Comm: popen.rb:42 Tainted: G L ------------ 3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff8abe2a11e300 ti: ffff8abd79da0000 task.ti: ffff8abd79da0000
RIP: 0010:[<ffffffffb0b16b62>] [<ffffffffb0b16b62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8abd79da3be0 EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffff8abd79da3bb0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff8abd79da3c30 R08: ffffffffb12137c0 R09: 00003ffffffff000
R10: ffff8abef660f290 R11: 0000000000000206 R12: 0000360ec6e8bfcf
R13: ffffffffb0ae495c R14: ffff8abd79da3b80 R15: ffff8ab4bfd11000
FS: 00007f09626d0700(0000) GS:ffff8abc5aec0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0986fdd000 CR3: 0000000be30fe000 CR4: 00000000000006e0
Call Trace:
[<ffffffffb0a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb0a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb0a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffb0b16c7f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb0d86865>] ? cpumask_next_and+0x35/0x50
[<ffffffffb0b1722b>] smp_call_function_many+0x22b/0x270
[<ffffffffb0a7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb0a7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb0bee6c6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb0bf018f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb0bf02b3>] tlb_finish_mmu+0x23/0x30
[<ffffffffb0bf377a>] zap_page_range+0xea/0x130
[<ffffffffb0b127a0>] ? futex_wake+0x90/0x180
[<ffffffffb0bedc6d>] SyS_madvise+0x49d/0xac0
[<ffffffffb1199ed5>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffffb1199ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb1199f92>] system_call_fastpath+0x25/0x2a
[<ffffffffb1199ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 75 b1 48 89 df e8 77 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #251363
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff92916f62
Build id: 3.10.0-1127.el7.x86_64
|
0x102
|
|
2
|
|
vmlinux |
0xffffffff9287e730
Build id: 3.10.0-1127.el7.x86_64
|
0x120
|
|
3
|
|
vmlinux |
0xffffffff9287e730
Build id: 3.10.0-1127.el7.x86_64
|
0x120
|
|
4
|
|
vmlinux |
0xffffffff9287e730
Build id: 3.10.0-1127.el7.x86_64
|
0x120
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff9291707f
Build id: 3.10.0-1127.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff92b84ee5
Build id: 3.10.0-1127.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff9291762b
Build id: 3.10.0-1127.el7.x86_64
|
0x22b
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff9287e8f8
Build id: 3.10.0-1127.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff9287e96b
Build id: 3.10.0-1127.el7.x86_64
|
0x6b
|
|
10
|
zap_page_range
|
vmlinux |
0xffffffff929f2e58
Build id: 3.10.0-1127.el7.x86_64
|
0xd8
|
|
11
|
|
vmlinux |
0xffffffff92912a60
Build id: 3.10.0-1127.el7.x86_64
|
0x90
|
|
12
|
SyS_madvise
|
vmlinux |
0xffffffff929ed36d
Build id: 3.10.0-1127.el7.x86_64
|
0x49d
|
|
13
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff92f92e15
Build id: 3.10.0-1127.el7.x86_64
|
0xa2
|
|
14
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff92f92e09
Build id: 3.10.0-1127.el7.x86_64
|
0x96
|
|
15
|
system_call_fastpath
|
vmlinux |
0xffffffff92f92ed2
Build id: 3.10.0-1127.el7.x86_64
|
0x25
|
|
16
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff92f92e15
Build id: 3.10.0-1127.el7.x86_64
|
0xa2
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [popen.rb:41:25208]
Modules linked in: tcp_lp btrfs raid6_pq xor vfat fat tcp_diag inet_diag ext4 mbcache jbd2 fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc uas coretemp usb_storage crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper cryptd vmw_balloon
sg pcspkr joydev vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw crct10dif_pclmul crct10dif_common crc32c_intel drm mptspi ata_piix scsi_transport_spi mptscsih libata mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 25208 Comm: popen.rb:41 Kdump: loaded Tainted: G L ------------ 3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
task: ffff9c85815c20e0 ti: ffff9c84ef00c000 task.ti: ffff9c84ef00c000
RIP: 0010:[<ffffffff92916f62>] [<ffffffff92916f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c84ef00fc30 EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffff9c85815c2148 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff9c84ef00fc80 R08: 0000000000000001 R09: 00007f3c1cfac000
R10: 00007f3c39f0a140 R11: 0000000000000206 R12: ffff9c85bfc1acc0
R13: ffffffff928e103c R14: ffff9c84ef00fbb8 R15: ffffffff928e2e89
FS: 00007f3c1d0ab700(0000) GS:ffff9c85bfc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3c2e6f13e0 CR3: 00000000a5918000 CR4: 00000000000007f0
Call Trace:
[<ffffffff9287e730>] ? leave_mm+0x120/0x120
[<ffffffff9287e730>] ? leave_mm+0x120/0x120
[<ffffffff9287e730>] ? leave_mm+0x120/0x120
[<ffffffff9291707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff92b84ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9291762b>] smp_call_function_many+0x22b/0x270
[<ffffffff9287e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9287e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff929f2e58>] zap_page_range+0xd8/0x150
[<ffffffff92912a60>] ? futex_wake+0x90/0x180
[<ffffffff929ed36d>] SyS_madvise+0x49d/0xab0
[<ffffffff92f92e15>] ? system_call_after_swapgs+0xa2/0x13a
[<ffffffff92f92e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff92f92ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff92f92e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 0f 55 93 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #180478
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
generic_exec_single
|
vmlinux |
0xffffffff810fb21a
Build id: 3.10.0-693.17.1.el7.x86_64
|
0xfa
|
|
2
|
|
vmlinux |
0xffffffff8106f340
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x110
|
|
3
|
|
vmlinux |
0xffffffff8106f340
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x110
|
|
4
|
|
vmlinux |
0xffffffff8106f340
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x110
|
|
5
|
smp_call_function_single
|
vmlinux |
0xffffffff810fb31f
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x5f
|
|
6
|
|
vmlinux |
0xffffffff81324bd5
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x35
|
|
7
|
smp_call_function_many
|
vmlinux |
0xffffffff810fb8b3
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x223
|
|
8
|
native_flush_tlb_others
|
vmlinux |
0xffffffff8106f508
Build id: 3.10.0-693.17.1.el7.x86_64
|
0xb8
|
|
9
|
flush_tlb_mm_range
|
vmlinux |
0xffffffff8106f5d8
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x68
|
|
10
|
tlb_flush_mmu.part.63
|
vmlinux |
0xffffffff811af0f3
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x33
|
|
11
|
tlb_finish_mmu
|
vmlinux |
0xffffffff811b0415
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x55
|
|
12
|
zap_page_range
|
vmlinux |
0xffffffff811b299a
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x13a
|
|
13
|
|
vmlinux |
0xffffffff810f6e60
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x90
|
|
14
|
SyS_madvise
|
vmlinux |
0xffffffff811ae7bd
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x3cd
|
|
15
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff816b895a
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x187
|
|
16
|
system_call_fastpath
|
vmlinux |
0xffffffff816b89fd
Build id: 3.10.0-693.17.1.el7.x86_64
|
0x16
|
|
17
|
?
system_call_after_swapgs
|
vmlinux |
0xffffffff816b889d
Build id: 3.10.0-693.17.1.el7.x86_64
|
0xca
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [Plex Media Serv:1571]
Modules linked in: arc4 md4 rpcsec_gss_krb5 nfsv4 nfs fscache nls_utf8 cifs dns_resolver ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ppdev pcspkr sg joydev virtio_balloon shpchp parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net virtio_scsi bochs_drm drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw virtio_pci virtio_ring virtio ata_piix libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 1571 Comm: Plex Media Serv Tainted: G L ------------ 3.10.0-693.17.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff88005989cf10 ti: ffff8800610cc000 task.ti: ffff8800610cc000
RIP: 0010:[<ffffffff810fb21a>] [<ffffffff810fb21a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8800610cfbf0 EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff8800610cfbc0 RCX: 0000000000000830
RDX: 0000000000000002 RSI: 0000000000000002 RDI: 0000000000000282
RBP: ffff8800610cfc40 R08: ffffffff816d69e0 R09: 00007f7adf8e2000
R10: 0000000000000000 R11: 0000000000000206 R12: ffffffff816b9bff
R13: ffffffff816b9c06 R14: ffffffff816b9c0d R15: ffffffff816b9c14
FS: 00007f7ad7ff7700(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7adc1c6138 CR3: 0000000078b56000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8106f340>] ? leave_mm+0x110/0x110
[<ffffffff8106f340>] ? leave_mm+0x110/0x110
[<ffffffff8106f340>] ? leave_mm+0x110/0x110
[<ffffffff810fb31f>] smp_call_function_single+0x5f/0xa0
[<ffffffff81324bd5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810fb8b3>] smp_call_function_many+0x223/0x260
[<ffffffff8106f508>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106f5d8>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811af0f3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b0415>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b299a>] zap_page_range+0x13a/0x180
[<ffffffff810f6e60>] ? futex_wake+0x90/0x180
[<ffffffff811ae7bd>] SyS_madvise+0x3cd/0x9c0
[<ffffffff816b895a>] ? system_call_after_swapgs+0x187/0x214
[<ffffffff816b89fd>] system_call_fastpath+0x16/0x1b
[<ffffffff816b889d>] ? system_call_after_swapgs+0xca/0x214
Code: 40 95 01 00 48 89 de 48 03 14 c5 20 8e b1 81 48 89 df e8 5a e0 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00