Info

Permalink
Function
smp_call_function_many
First occurrence
2020-09-25
Last occurrence
2026-02-08
State
NEW
Components
kernel
Quality
-61

Statistics

Operating system Count
CentOS 7 1,456
Architecture Count
x86_64 1,456
Related packages Count
kernel 1,446
 0:3.10.0-957.el7 427
 0:3.10.0-1160.el7 157
 0:3.10.0-1160.76.1.el7 138
 0:3.10.0-1160.59.1.el7 114
 0:3.10.0-1127.el7 100
 0:3.10.0-1062.18.1.el7 41
 0:3.10.0-1160.118.1.el7 41
 0:3.10.0-1160.49.1.el7 41
 0:3.10.0-1062.12.1.el7 24
 0:3.10.0-1160.36.2.el7 22
 0:3.10.0-1062.el7 20
 0:3.10.0-1160.119.1.el7 19
 0:3.10.0-862.el7 17
 0:3.10.0-957.27.2.el7 16
 0:3.10.0-1160.71.1.el7 15
 0:3.10.0-1160.108.1.el7 13
 0:3.10.0-1160.31.1.el7 13
 0:3.10.0-1160.81.1.el7 12
 0:3.10.0-1160.119.1.el7.tuxcare.els19 11
 0:3.10.0-1160.66.1.el7 11
 0:3.10.0-1160.95.1.el7 11
 0:3.10.0-693.el7 10
 0:3.10.0-1160.119.1.el7.tuxcare.els18 9
 0:3.10.0-1160.15.2.el7 9
 0:3.10.0-1160.62.1.el7 9
 0:3.10.0-1160.83.1.el7 9
 0:3.10.0-1160.90.1.el7 9
 0:3.10.0-957.10.1.el7 9
 0:3.10.0-957.1.3.el7 9
 0:3.10.0-1127.18.2.el7 8
 0:3.10.0-1127.19.1.el7 8
 0:3.10.0-1160.11.1.el7 7
 0:3.10.0-1062.4.3.el7 6
 0:3.10.0-1062.9.1.el7 6
 0:3.10.0-1127.10.1.el7 6
 0:3.10.0-1160.53.1.el7 6
 0:3.10.0-1160.6.1.el7 6
 0:3.10.0-1160.88.1.el7 6
 0:3.10.0-957.12.1.el7 6
 0:3.10.0-1160.102.1.el7 5
 0:3.10.0-1127.8.2.el7 4
 0:3.10.0-957.21.3.el7 4
 0:3.10.0-957.5.1.el7 4
 0:3.10.0-1160.25.1.el7 3
 0:3.10.0-1160.42.2.el7 3
 0:3.10.0-1160.45.1.el7 3
 0:3.10.0-514.21.1.el7 3
 0:3.10.0-1127.13.1.el7 2
 0:3.10.0-1160.114.2.el7 2
 0:3.10.0-1160.24.1.el7 2
 0:3.10.0-1160.80.1.el7 2
 0:3.10.0-327.el7 2
 0:3.10.0-862.14.4.el7 2
 0:3.10.0-1062.1.2.el7 1
 0:3.10.0-1160.105.1.el7 1
 0:3.10.0-1160.99.1.el7 1
 0:3.10.0-957.12.2.el7 1
kernel-lt 9
 0:4.4.15-1.el7.elrepo 9
kernel-plus 1
 0:3.10.0-1160.88.1.el7.centos.plus 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1891
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9e1163c2
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x102
2
? leave_mm
vmlinux 0xffffffff9e07e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
3
? leave_mm
vmlinux 0xffffffff9e07e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
4
? leave_mm
vmlinux 0xffffffff9e07e2b0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
5
smp_call_function_single
vmlinux 0xffffffff9e1164df
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9e381635
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x35
7
smp_call_function_many
vmlinux 0xffffffff9e116a8b
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9e07e478
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xb8
9
flush_tlb_page
vmlinux 0xffffffff9e07e614
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x54
10
ptep_clear_flush
vmlinux 0xffffffff9e2059d8
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x68
11
page_mkclean_one
vmlinux 0xffffffff9e1fef66
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xa6
12
rmap_walk
vmlinux 0xffffffff9e200d93
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xf3
13
page_mkclean
vmlinux 0xffffffff9e2011a6
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x76
14
? page_referenced_one
vmlinux 0xffffffff9e1feec0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x1b0
15
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffff9e1fe5a0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x20
16
clear_page_dirty_for_io
vmlinux 0xffffffff9e1c98e0
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x50
17
mpage_submit_page
ext4 0xffffffffc0498dc4
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x24
18
mpage_process_page_bufs
ext4 0xffffffffc0498f20
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x110
19
mpage_prepare_extent_to_map
ext4 0xffffffffc049a2c5
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x235
20
? jbd2__journal_start
jbd2 0xffffffffc0476a93
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xf3
21
? ext4_writepages
ext4 0xffffffffc049ea5c
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x43c
22
? __ext4_journal_start_sb
ext4 0xffffffffc04ce129
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x69
23
ext4_writepages
ext4 0xffffffffc049ea87
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x467
24
do_writepages
vmlinux 0xffffffff9e1cabc1
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x21
25
__writeback_single_inode
vmlinux 0xffffffff9e279840
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x40
26
? wake_up_bit
vmlinux 0xffffffff9e0c72a5
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x25
27
writeback_sb_inodes
vmlinux 0xffffffff9e27a3d4
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x1c4
28
__writeback_inodes_wb
vmlinux 0xffffffff9e27a6df
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x9f
29
wb_writeback
vmlinux 0xffffffff9e27abc3
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x263
30
? get_nr_inodes
vmlinux 0xffffffff9e266a6c
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x4c
31
bdi_writeback_workfn
vmlinux 0xffffffff9e27b7bb
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2cb
32
process_one_work
vmlinux 0xffffffff9e0be21f
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x17f
33
worker_thread
vmlinux 0xffffffff9e0bf336
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x126
34
? manage_workers.isra.26
vmlinux 0xffffffff9e0bf210
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x2a0
35
kthread
vmlinux 0xffffffff9e0c61f1
Build id: 3.10.0-1062.9.1.el7.x86_64 		0xd1
36
? insert_kthread_work
vmlinux 0xffffffff9e0c6120
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x40
37
ret_from_fork_nospec_begin
vmlinux 0xffffffff9e78dd37
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x21
38
? insert_kthread_work
vmlinux 0xffffffff9e0c6120
Build id: 3.10.0-1062.9.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u10:0:22763]
Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dm_mirror dm_region_hash dm_log dm_mod ppdev aesni_intel lrw gf128mul sg glue_helper ablk_helper cryptd pcspkr joydev i2c_piix4 virtio_balloon parport_pc parport ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom virtio_net virtio_console virtio_scsi
ata_generic pata_acpi bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm ata_piix libata drm_panel_orientation_quirks virtio_pci serio_raw virtio_ring virtio floppy
CPU: 2 PID: 22763 Comm: kworker/u10:0 Not tainted 3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff97056eff41c0 ti: ffff9706bba78000 task.ti: ffff9706bba78000
RIP: 0010:[<ffffffff9e1163c2>]  [<ffffffff9e1163c2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9706bba7b6c0  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff9706bba7b690 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000286
RBP: ffff9706bba7b708 R08: ffffffff9e813540 R09: 0000000562f98000
R10: ffff97086f5747e0 R11: fffffd88c8ac52c0 R12: fffffd88cabb5240
R13: 0000000000000046 R14: ffff9706bba7b6d8 R15: 0000000006c6e5c9
FS:  0000000000000000(0000) GS:ffff97057fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb77b7399c8 CR3: 000000001ee10000 CR4: 00000000000006e0
Call Trace:
[<ffffffff9e07e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9e07e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9e07e2b0>] ? leave_mm+0x110/0x110
[<ffffffff9e1164df>] smp_call_function_single+0x5f/0xa0
[<ffffffff9e381635>] ? cpumask_next_and+0x35/0x50
[<ffffffff9e116a8b>] smp_call_function_many+0x22b/0x270
[<ffffffff9e07e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9e07e614>] flush_tlb_page+0x54/0xa0
[<ffffffff9e2059d8>] ptep_clear_flush+0x68/0xa0
[<ffffffff9e1fef66>] page_mkclean_one+0xa6/0xe0
[<ffffffff9e200d93>] rmap_walk+0xf3/0x360
[<ffffffff9e2011a6>] page_mkclean+0x76/0xb0
[<ffffffff9e1feec0>] ? page_referenced_one+0x1b0/0x1b0
[<ffffffff9e1fe5a0>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffff9e1c98e0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc0498dc4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc0498f20>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc049a2c5>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc0476a93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc049ea5c>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc04ce129>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc049ea87>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffff9e1cabc1>] do_writepages+0x21/0x50
[<ffffffff9e279840>] __writeback_single_inode+0x40/0x260
[<ffffffff9e0c72a5>] ? wake_up_bit+0x25/0x30
[<ffffffff9e27a3d4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9e27a6df>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9e27abc3>] wb_writeback+0x263/0x2f0
[<ffffffff9e266a6c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff9e27b7bb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9e0be21f>] process_one_work+0x17f/0x440
[<ffffffff9e0bf336>] worker_thread+0x126/0x3c0
[<ffffffff9e0bf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9e0c61f1>] kthread+0xd1/0xe0
[<ffffffff9e0c6120>] ? insert_kthread_work+0x40/0x40
[<ffffffff9e78dd37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9e0c6120>] ? insert_kthread_work+0x40/0x40
Code: 89 de 48 03 14 c5 20 e8 d4 9e 48 89 df e8 f7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00
Complete report #1196903
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9451761a
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9447e8f8
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff9447ea94
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff94607888
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x68
5
page_mkclean_one
vmlinux 0xffffffff94600e36
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffff94602c43
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffff94603056
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffff94600d90
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x1b0
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffff94600470
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffff945c9f70
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc05fc1f4
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc05fc350
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc05fd675
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x235
14
? jbd2__journal_start
jbd2 0xffffffffc04dca93
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xf3
15
? ext4_writepages
ext4 0xffffffffc0601dec
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x43c
16
? __ext4_journal_start_sb
ext4 0xffffffffc0631679
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x69
17
ext4_writepages
ext4 0xffffffffc0601e17
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x467
18
do_writepages
vmlinux 0xffffffff945cb261
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffff9467cc10
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffff9467d794
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffff9467da9f
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x9f
22
wb_writeback
vmlinux 0xffffffff9467df73
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x263
23
bdi_writeback_workfn
vmlinux 0xffffffff9467ea6c
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x1cc
24
process_one_work
vmlinux 0xffffffff944be6bf
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x17f
25
worker_thread
vmlinux 0xffffffff944bf7d6
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x126
26
? manage_workers.isra.26
vmlinux 0xffffffff944bf6b0
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x2a0
27
kthread
vmlinux 0xffffffff944c6691
Build id: 3.10.0-1127.18.2.el7.x86_64 		0xd1
28
? insert_kthread_work
vmlinux 0xffffffff944c65c0
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff94b92d37
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x21
30
? insert_kthread_work
vmlinux 0xffffffff944c65c0
Build id: 3.10.0-1127.18.2.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u12:0:11897]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc falcon_lsm_serviceable(PE) falcon_nf_netcontain(E) falcon_kal(E) falcon_lsm_pinned_17607(E) ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul ppdev glue_helper ablk_helper vmw_balloon cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport sch_fq_codel ip_tables ext4 mbcache jbd2 ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sd_mod sysimgblt fb_sys_fops ttm crc_t10dif
crct10dif_generic ahci drm ata_piix libahci nfit crct10dif_pclmul crct10dif_common libnvdimm crc32c_intel serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks
CPU: 2 PID: 11897 Comm: kworker/u12:0 Kdump: loaded Tainted: P            EL ------------   3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff9c4e2c203150 ti: ffff9c4e25254000 task.ti: ffff9c4e25254000
RIP: 0010:[<ffffffff9451761a>]  [<ffffffff9451761a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9c4e25257748  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff9c4fffc60628
RDX: 0000000000000001 RSI: 0000000000000006 RDI: 0000000000000000
RBP: ffff9c4e25257780 R08: ffff9c4d3a372000 R09: ffffffff94785179
R10: ffff9c4fffc9f160 R11: ffffcba3cf773600 R12: 0000000000000296
R13: ffff9c4e252576f8 R14: ffff9c4d3fd70200 R15: 0001122000000002
FS:  0000000000000000(0000) GS:ffff9c4fffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc25a5b8000 CR3: 000000027e410000 CR4: 00000000000607e0
Call Trace:
[<ffffffff9447e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9447ea94>] flush_tlb_page+0x54/0xa0
[<ffffffff94607888>] ptep_clear_flush+0x68/0xa0
[<ffffffff94600e36>] page_mkclean_one+0xa6/0xe0
[<ffffffff94602c43>] rmap_walk+0xf3/0x360
[<ffffffff94603056>] page_mkclean+0x76/0xb0
[<ffffffff94600d90>] ? page_referenced_one+0x1b0/0x1b0
[<ffffffff94600470>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffff945c9f70>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc05fc1f4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc05fc350>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc05fd675>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc04dca93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc0601dec>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc0631679>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc0601e17>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffff945cb261>] do_writepages+0x21/0x50
[<ffffffff9467cc10>] __writeback_single_inode+0x40/0x260
[<ffffffff9467d794>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9467da9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9467df73>] wb_writeback+0x263/0x2f0
[<ffffffff9467ea6c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff944be6bf>] process_one_work+0x17f/0x440
[<ffffffff944bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff944bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff944c6691>] kthread+0xd1/0xe0
[<ffffffff944c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff94b92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff944c65c0>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 ce 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 10 15 95 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 7b c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #1094660
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f99a2
Build id: 3.10.0-693.el7.x86_64 		0x202
2
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106e864
Build id: 3.10.0-693.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff811c2524
Build id: 3.10.0-693.el7.x86_64 		0x54
5
page_mkclean
vmlinux 0xffffffff811bbf2e
Build id: 3.10.0-693.el7.x86_64 		0x12e
6
clear_page_dirty_for_io
vmlinux 0xffffffff8118d9c0
Build id: 3.10.0-693.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffc01df297
Build id: 3.10.0-693.el7.x86_64 		0x47
8
mpage_process_page_bufs
ext4 0xffffffffc01df3e0
Build id: 3.10.0-693.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffc01e02d5
Build id: 3.10.0-693.el7.x86_64 		0x235
10
? jbd2__journal_start
jbd2 0xffffffffc01bda93
Build id: 3.10.0-693.el7.x86_64 		0xf3
11
? ext4_writepages
ext4 0xffffffffc01e4afc
Build id: 3.10.0-693.el7.x86_64 		0x42c
12
? __ext4_journal_start_sb
ext4 0xffffffffc0212ad9
Build id: 3.10.0-693.el7.x86_64 		0x69
13
ext4_writepages
ext4 0xffffffffc01e4b27
Build id: 3.10.0-693.el7.x86_64 		0x457
14
do_writepages
vmlinux 0xffffffff8118effe
Build id: 3.10.0-693.el7.x86_64 		0x1e
15
__writeback_single_inode
vmlinux 0xffffffff8122d7d0
Build id: 3.10.0-693.el7.x86_64 		0x40
16
writeback_sb_inodes
vmlinux 0xffffffff8122e414
Build id: 3.10.0-693.el7.x86_64 		0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff8122e77f
Build id: 3.10.0-693.el7.x86_64 		0x9f
18
wb_writeback
vmlinux 0xffffffff8122efb3
Build id: 3.10.0-693.el7.x86_64 		0x263
19
? bdi_dirty_limit
vmlinux 0xffffffff8118e6b0
Build id: 3.10.0-693.el7.x86_64 		0x40
20
bdi_writeback_workfn
vmlinux 0xffffffff8122f35c
Build id: 3.10.0-693.el7.x86_64 		0x1cc
21
process_one_work
vmlinux 0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64 		0x17a
22
worker_thread
vmlinux 0xffffffff810a94e6
Build id: 3.10.0-693.el7.x86_64 		0x126
23
? manage_workers.isra.24
vmlinux 0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64 		0xcf
25
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40
26
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64 		0x58
27
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/u320:1:81930]
Modules linked in: nls_utf8 isofs xfs qax_tq_base(OE) iptable_filter ip_tables ip_set nfnetlink bridge stp llc libcrc32c tq_base(OE) ppdev sg virtio_balloon parport_pc joydev pcspkr parport i2c_piix4 ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_scsi virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ata_piix ttm drm libata serio_raw virtio_pci virtio_ring virtio i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: qax_tq_base]
CPU: 1 PID: 81930 Comm: kworker/u320:1 Tainted: G           OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8800b8ef1fa0 ti: ffff880ff3284000 task.ti: ffff880ff3284000
RIP: 0010:[<ffffffff810f99a2>]  [<ffffffff810f99a2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff880ff32877e8  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff880ffce1a6c8
RDX: 0000000000000000 RSI: 00000000000000a0 RDI: 0000000000000000
RBP: ffff880ff3287820 R08: ffff8801694c3400 R09: ffffffff813227d9
R10: ffff880ffce59c80 R11: ffffea0008587600 R12: 0000000000000292
R13: ffff880ff3287798 R14: ffffea003fe1ca42 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff880ffce40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f83bab40000 CR3: 0000000823453000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
00000001f87291b0 ffff880ff472ce48 ffff880ff472cb00 00007f0486af3000
0000000000000000 ffff880ff472ce48 ffff88055ac0f798 ffff880ff3287870
ffffffff8106e668 ffff880ff472cb00 00007f0486af3000 0000000000000000
Call Trace:
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e864>] flush_tlb_page+0x54/0xa0
[<ffffffff811c2524>] ptep_clear_flush+0x54/0x80
[<ffffffff811bbf2e>] page_mkclean+0x12e/0x1f0
[<ffffffff8118d9c0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc01df297>] mpage_submit_page+0x47/0x80 [ext4]
[<ffffffffc01df3e0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc01e02d5>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc01bda93>] ? jbd2__journal_start+0xf3/0x1e0 [jbd2]
[<ffffffffc01e4afc>] ? ext4_writepages+0x42c/0xd30 [ext4]
[<ffffffffc0212ad9>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc01e4b27>] ext4_writepages+0x457/0xd30 [ext4]
[<ffffffff8118effe>] do_writepages+0x1e/0x40
[<ffffffff8122d7d0>] __writeback_single_inode+0x40/0x220
[<ffffffff8122e414>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8122e77f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8122efb3>] wb_writeback+0x263/0x2f0
[<ffffffff8118e6b0>] ? bdi_dirty_limit+0x40/0xe0
[<ffffffff8122f35c>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a94e6>] worker_thread+0x126/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 26 45 a2 00 89 c2 39 f0 0f 8d 85 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 fd b0 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 f5 44 a2 00 eb b7 0f b6 4d cc 4c 89
Complete report #1044879
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff85d07e3a
Build id: 3.10.0-862.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff85c77018
Build id: 3.10.0-862.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff85c771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff85dd8837
Build id: 3.10.0-862.el7.x86_64 		0x67
5
page_mkclean
vmlinux 0xffffffff85dd1ba4
Build id: 3.10.0-862.el7.x86_64 		0x144
6
clear_page_dirty_for_io
vmlinux 0xffffffff85d9f7d0
Build id: 3.10.0-862.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffc060dba4
Build id: 3.10.0-862.el7.x86_64 		0x24
8
mpage_process_page_bufs
ext4 0xffffffffc060dd00
Build id: 3.10.0-862.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffc060ec05
Build id: 3.10.0-862.el7.x86_64 		0x235
10
? jbd2__journal_start
jbd2 0xffffffffc05eca93
Build id: 3.10.0-862.el7.x86_64 		0xf3
11
? ext4_writepages
ext4 0xffffffffc061357c
Build id: 3.10.0-862.el7.x86_64 		0x42c
12
? __ext4_journal_start_sb
ext4 0xffffffffc0641659
Build id: 3.10.0-862.el7.x86_64 		0x69
13
ext4_writepages
ext4 0xffffffffc06135a7
Build id: 3.10.0-862.el7.x86_64 		0x457
14
? __schedule
vmlinux 0xffffffff8631291c
Build id: 3.10.0-862.el7.x86_64 		0x41c
15
do_writepages
vmlinux 0xffffffff85da0e21
Build id: 3.10.0-862.el7.x86_64 		0x21
16
__writeback_single_inode
vmlinux 0xffffffff85e48af0
Build id: 3.10.0-862.el7.x86_64 		0x40
17
writeback_sb_inodes
vmlinux 0xffffffff85e49584
Build id: 3.10.0-862.el7.x86_64 		0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffff85e498ef
Build id: 3.10.0-862.el7.x86_64 		0x9f
19
wb_writeback
vmlinux 0xffffffff85e4a123
Build id: 3.10.0-862.el7.x86_64 		0x263
20
bdi_writeback_workfn
vmlinux 0xffffffff85e4aaab
Build id: 3.10.0-862.el7.x86_64 		0x2cb
21
process_one_work
vmlinux 0xffffffff85cb2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffff85cb3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
23
? manage_workers.isra.24
vmlinux 0xffffffff85cb39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffff85cbae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffff85cbad60
Build id: 3.10.0-862.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff8631f637
Build id: 3.10.0-862.el7.x86_64 		0x21
27
? insert_kthread_work
vmlinux 0xffffffff85cbad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 25s! [kworker/u256:1:67]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel snd_ens1371 aesni_intel snd_rawmidi lrw snd_ac97_codec gf128mul ac97_bus glue_helper ppdev ablk_helper snd_seq cryptd snd_seq_device snd_pcm vmw_balloon pcspkr joydev snd_timer snd sg soundcore i2c_piix4 nfit vmw_vmci shpchp libnvdimm parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih ata_piix crc32c_intel serio_raw drm mptbase e1000 libata i2c_core
CPU: 4 PID: 67 Comm: kworker/u256:1 Kdump: loaded Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff89bd35da3f40 ti: ffff89bd35e74000 task.ti: ffff89bd35e74000
RIP: 0010:[<ffffffff85d07e3a>]  [<ffffffff85d07e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff89bd35e777e8  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff89bd7b61cd98
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff89bd35e77820 R08: ffff89bd7a6c4000 R09: ffffffff85f4a3d9
R10: ffff89bd7b71bb00 R11: ffffd6a700d78e00 R12: 0000000000000292
R13: ffff89bd35e77798 R14: ffff89bd35e77788 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff89bd7b700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4029e61980 CR3: 0000000016a0e000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffff85c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85c771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff85dd8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff85dd1ba4>] page_mkclean+0x144/0x210
[<ffffffff85d9f7d0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc060dba4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc060dd00>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc060ec05>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc05eca93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc061357c>] ? ext4_writepages+0x42c/0xd40 [ext4]
[<ffffffffc0641659>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc06135a7>] ext4_writepages+0x457/0xd40 [ext4]
[<ffffffff8631291c>] ? __schedule+0x41c/0xa20
[<ffffffff85da0e21>] do_writepages+0x21/0x50
[<ffffffff85e48af0>] __writeback_single_inode+0x40/0x260
[<ffffffff85e49584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff85e498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff85e4a123>] wb_writeback+0x263/0x2f0
[<ffffffff85e4aaab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff85cb2dff>] process_one_work+0x17f/0x440
[<ffffffff85cb3ac6>] worker_thread+0x126/0x3c0
[<ffffffff85cb39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff85cbae31>] kthread+0xd1/0xe0
[<ffffffff85cbad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff8631f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff85cbad60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 93 86 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #73939
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810f9f72
Build id: 3.10.0-514.21.1.el7.x86_64 		0x202
2
native_flush_tlb_others
vmlinux 0xffffffff8106c5f8
Build id: 3.10.0-514.21.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8106c7f5
Build id: 3.10.0-514.21.1.el7.x86_64 		0x55
4
ptep_clear_flush
vmlinux 0xffffffff811c1464
Build id: 3.10.0-514.21.1.el7.x86_64 		0x54
5
page_mkclean
vmlinux 0xffffffff811bafbe
Build id: 3.10.0-514.21.1.el7.x86_64 		0x16e
6
clear_page_dirty_for_io
vmlinux 0xffffffff8118c2f0
Build id: 3.10.0-514.21.1.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffa02c2287
Build id: 3.10.0-514.21.1.el7.x86_64 		0x47
8
mpage_process_page_bufs
ext4 0xffffffffa02c23d0
Build id: 3.10.0-514.21.1.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffa02c32c5
Build id: 3.10.0-514.21.1.el7.x86_64 		0x235
10
? jbd2__journal_start
jbd2 0xffffffffa02a0a63
Build id: 3.10.0-514.21.1.el7.x86_64 		0xf3
11
? ext4_writepages
ext4 0xffffffffa02c78fc
Build id: 3.10.0-514.21.1.el7.x86_64 		0x42c
12
? __ext4_journal_start_sb
ext4 0xffffffffa02f8b39
Build id: 3.10.0-514.21.1.el7.x86_64 		0x69
13
ext4_writepages
ext4 0xffffffffa02c792b
Build id: 3.10.0-514.21.1.el7.x86_64 		0x45b
14
do_writepages
vmlinux 0xffffffff8118d93e
Build id: 3.10.0-514.21.1.el7.x86_64 		0x1e
15
__writeback_single_inode
vmlinux 0xffffffff81228e40
Build id: 3.10.0-514.21.1.el7.x86_64 		0x40
16
writeback_sb_inodes
vmlinux 0xffffffff81229b2e
Build id: 3.10.0-514.21.1.el7.x86_64 		0x25e
17
__writeback_inodes_wb
vmlinux 0xffffffff81229d8f
Build id: 3.10.0-514.21.1.el7.x86_64 		0x9f
18
wb_writeback
vmlinux 0xffffffff8122a5d3
Build id: 3.10.0-514.21.1.el7.x86_64 		0x263
19
? get_nr_inodes
vmlinux 0xffffffff81218e8c
Build id: 3.10.0-514.21.1.el7.x86_64 		0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff8122c5cb
Build id: 3.10.0-514.21.1.el7.x86_64 		0x2cb
21
process_one_work
vmlinux 0xffffffff810a845b
Build id: 3.10.0-514.21.1.el7.x86_64 		0x17b
22
worker_thread
vmlinux 0xffffffff810a9296
Build id: 3.10.0-514.21.1.el7.x86_64 		0x126
23
? rescuer_thread
vmlinux 0xffffffff810a9170
Build id: 3.10.0-514.21.1.el7.x86_64 		0x460
24
kthread
vmlinux 0xffffffff810b0a4f
Build id: 3.10.0-514.21.1.el7.x86_64 		0xcf
25
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64 		0x140
26
ret_from_fork
vmlinux 0xffffffff81697518
Build id: 3.10.0-514.21.1.el7.x86_64 		0x58
27
? kthread_create_on_node
vmlinux 0xffffffff810b0980
Build id: 3.10.0-514.21.1.el7.x86_64 		0x140 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [kworker/u64:1:28028]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi vmw_vsock_vmci_transport vsock xfs libcrc32c intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr vmw_vmci sg nfit i2c_piix4 shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect crct10dif_pclmul sysimgblt
crct10dif_common fb_sys_fops crc32c_intel ttm serio_raw mptspi scsi_transport_spi drm ata_piix mptscsih libata vmxnet3 mptbase i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 28028 Comm: kworker/u64:1 Tainted: G             L ------------   3.10.0-514.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff88051376bec0 ti: ffff8801424d0000 task.ti: ffff8801424d0000
RIP: 0010:[<ffffffff810f9f72>]  [<ffffffff810f9f72>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff8801424d37e8  EFLAGS: 00000202
RAX: 000000000000000c RBX: 0000000000017780 RCX: ffff88052b11ae20
RDX: 000000000000000c RSI: 0000000000000020 RDI: 0000000000000000
RBP: ffff8801424d3820 R08: ffff880177a75000 R09: ffffffff813182b9
R10: ffff88052af19b40 R11: ffffea0008983800 R12: ffff8801424d3798
R13: 0000000000000000 R14: 0000000000000000 R15: 000000008597b72d
FS:  0000000000000000(0000) GS:ffff88052af00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f063b763008 CR3: 00000005093ba000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000100000001 ffff880480f7a288 ffff880480f79f40 00007fea4d606000
0000000000000000 ffff880480f7a288 ffff880506f3f030 ffff8801424d3870
ffffffff8106c5f8 ffff880480f79f40 00007fea4d606000 0000000000000000
Call Trace:
[<ffffffff8106c5f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c7f5>] flush_tlb_page+0x55/0xa0
[<ffffffff811c1464>] ptep_clear_flush+0x54/0x80
[<ffffffff811bafbe>] page_mkclean+0x16e/0x1f0
[<ffffffff8118c2f0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffa02c2287>] mpage_submit_page+0x47/0x80 [ext4]
[<ffffffffa02c23d0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffa02c32c5>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffa02a0a63>] ? jbd2__journal_start+0xf3/0x1e0 [jbd2]
[<ffffffffa02c78fc>] ? ext4_writepages+0x42c/0xd60 [ext4]
[<ffffffffa02f8b39>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffa02c792b>] ext4_writepages+0x45b/0xd60 [ext4]
[<ffffffff8118d93e>] do_writepages+0x1e/0x40
[<ffffffff81228e40>] __writeback_single_inode+0x40/0x210
[<ffffffff81229b2e>] writeback_sb_inodes+0x25e/0x420
[<ffffffff81229d8f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8122a5d3>] wb_writeback+0x263/0x2f0
[<ffffffff81218e8c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8122c5cb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810a845b>] process_one_work+0x17b/0x470
[<ffffffff810a9296>] worker_thread+0x126/0x410
[<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[<ffffffff810b0a4f>] kthread+0xcf/0xe0
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81697518>] ret_from_fork+0x58/0x90
[<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
Code: 48 63 35 66 da 9e 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 c0 99 ad 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 35 da 9e 00 eb b7 0f b6 4d cc 4c 89
Complete report #11552
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff810e6de2
Build id: 3.10.0-327.el7.x86_64 		0x202
2
native_flush_tlb_others
vmlinux 0xffffffff81065e58
Build id: 3.10.0-327.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff81066055
Build id: 3.10.0-327.el7.x86_64 		0x55
4
ptep_clear_flush
vmlinux 0xffffffff811a77c4
Build id: 3.10.0-327.el7.x86_64 		0x54
5
page_mkclean
vmlinux 0xffffffff811a16b6
Build id: 3.10.0-327.el7.x86_64 		0x166
6
clear_page_dirty_for_io
vmlinux 0xffffffff811742b0
Build id: 3.10.0-327.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffa021bee7
Build id: 3.10.0-327.el7.x86_64 		0x47
8
mpage_process_page_bufs
ext4 0xffffffffa021c030
Build id: 3.10.0-327.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffa021ce75
Build id: 3.10.0-327.el7.x86_64 		0x235
10
? ext4_writepages
ext4 0xffffffffa0220fc4
Build id: 3.10.0-327.el7.x86_64 		0x434
11
? __ext4_journal_start_sb
ext4 0xffffffffa0251eb9
Build id: 3.10.0-327.el7.x86_64 		0x69
12
ext4_writepages
ext4 0xffffffffa0220ff3
Build id: 3.10.0-327.el7.x86_64 		0x463
13
? generic_writepages
vmlinux 0xffffffff81174858
Build id: 3.10.0-327.el7.x86_64 		0x58
14
do_writepages
vmlinux 0xffffffff811758fe
Build id: 3.10.0-327.el7.x86_64 		0x1e
15
__writeback_single_inode
vmlinux 0xffffffff81208410
Build id: 3.10.0-327.el7.x86_64 		0x40
16
writeback_sb_inodes
vmlinux 0xffffffff81208e7e
Build id: 3.10.0-327.el7.x86_64 		0x25e
17
__writeback_inodes_wb
vmlinux 0xffffffff812090df
Build id: 3.10.0-327.el7.x86_64 		0x9f
18
wb_writeback
vmlinux 0xffffffff81209923
Build id: 3.10.0-327.el7.x86_64 		0x263
19
? get_nr_inodes
vmlinux 0xffffffff811f875c
Build id: 3.10.0-327.el7.x86_64 		0x4c
20
bdi_writeback_workfn
vmlinux 0xffffffff8120bbab
Build id: 3.10.0-327.el7.x86_64 		0x2cb
21
process_one_work
vmlinux 0xffffffff8109d5fb
Build id: 3.10.0-327.el7.x86_64 		0x17b
22
worker_thread
vmlinux 0xffffffff8109e3cb
Build id: 3.10.0-327.el7.x86_64 		0x11b
23
? rescuer_thread
vmlinux 0xffffffff8109e2b0
Build id: 3.10.0-327.el7.x86_64 		0x400
24
kthread
vmlinux 0xffffffff810a5aef
Build id: 3.10.0-327.el7.x86_64 		0xcf
25
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64 		0x140
26
ret_from_fork
vmlinux 0xffffffff81645858
Build id: 3.10.0-327.el7.x86_64 		0x58
27
? kthread_create_on_node
vmlinux 0xffffffff810a5a20
Build id: 3.10.0-327.el7.x86_64 		0x140 
BUG: soft lockup - CPU#5 stuck for 22s! [kworker/u256:2:60836]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache bridge stp llc vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon sg pcspkr parport_pc vmw_vmci shpchp parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata vmwgfx vmxnet3 drm_kms_helper ttm e1000 mptspi scsi_transport_spi mptscsih drm mptbase i2c_core floppy [last unloaded: ip_tables]
CPU: 5 PID: 60836 Comm: kworker/u256:2 Not tainted 3.10.0-327.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff880035d84500 ti: ffff88080f934000 task.ti: ffff88080f934000
RIP: 0010:[<ffffffff810e6de2>]  [<ffffffff810e6de2>] smp_call_function_many+0x202/0x260
RSP: 0018:ffff88080f9377e8  EFLAGS: 00000202
RAX: 0000000000000006 RBX: 00000000000000fc RCX: ffff88081e6d87e0
RDX: 0000000000000006 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff88080f937820 R08: ffff880812f03000 R09: ffff88081e6b75e0
R10: ffffea001f1f3000 R11: ffffffff812f2869 R12: 0000000000000292
R13: ffff88080f937798 R14: 0000000000000000 R15: 000000007e41b179
FS:  0000000000000000(0000) GS:ffff88081e6a0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f04d5229000 CR3: 00000004d3967000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000100000001 ffff88080dd7b548 ffff88080dd7b200 00007f9aa4b86000
0000000000000000 ffff88080dd7b548 ffff88080ef58c30 ffff88080f937870
ffffffff81065e58 ffff88080dd7b200 00007f9aa4b86000 0000000000000000
Call Trace:
[<ffffffff81065e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81066055>] flush_tlb_page+0x55/0xa0
[<ffffffff811a77c4>] ptep_clear_flush+0x54/0x80
[<ffffffff811a16b6>] page_mkclean+0x166/0x1e0
[<ffffffff811742b0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffa021bee7>] mpage_submit_page+0x47/0x80 [ext4]
[<ffffffffa021c030>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffa021ce75>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffa0220fc4>] ? ext4_writepages+0x434/0xd60 [ext4]
[<ffffffffa0251eb9>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffa0220ff3>] ext4_writepages+0x463/0xd60 [ext4]
[<ffffffff81174858>] ? generic_writepages+0x58/0x80
[<ffffffff811758fe>] do_writepages+0x1e/0x40
[<ffffffff81208410>] __writeback_single_inode+0x40/0x220
[<ffffffff81208e7e>] writeback_sb_inodes+0x25e/0x420
[<ffffffff812090df>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff81209923>] wb_writeback+0x263/0x2f0
[<ffffffff811f875c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8120bbab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8109d5fb>] process_one_work+0x17b/0x470
[<ffffffff8109e3cb>] worker_thread+0x11b/0x400
[<ffffffff8109e2b0>] ? rescuer_thread+0x400/0x400
[<ffffffff810a5aef>] kthread+0xcf/0xe0
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
[<ffffffff81645858>] ret_from_fork+0x58/0x90
[<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140
Code: 48 63 35 f6 2a 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 e0 b8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 c5 2a 98 00 eb b7 0f b6 4d cc 4c 89
Complete report #1003995
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff99d07e3a
Build id: 3.10.0-862.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff99c77018
Build id: 3.10.0-862.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff99c771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff99dd8837
Build id: 3.10.0-862.el7.x86_64 		0x67
5
page_mkclean
vmlinux 0xffffffff99dd1ba4
Build id: 3.10.0-862.el7.x86_64 		0x144
6
clear_page_dirty_for_io
vmlinux 0xffffffff99d9f7d0
Build id: 3.10.0-862.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffc037dba4
Build id: 3.10.0-862.el7.x86_64 		0x24
8
mpage_process_page_bufs
ext4 0xffffffffc037dd00
Build id: 3.10.0-862.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffc037ec05
Build id: 3.10.0-862.el7.x86_64 		0x235
10
? ext4_writepages
ext4 0xffffffffc038357c
Build id: 3.10.0-862.el7.x86_64 		0x42c
11
? __ext4_journal_start_sb
ext4 0xffffffffc03b1659
Build id: 3.10.0-862.el7.x86_64 		0x69
12
ext4_writepages
ext4 0xffffffffc03835a7
Build id: 3.10.0-862.el7.x86_64 		0x457
13
do_writepages
vmlinux 0xffffffff99da0e21
Build id: 3.10.0-862.el7.x86_64 		0x21
14
__writeback_single_inode
vmlinux 0xffffffff99e48af0
Build id: 3.10.0-862.el7.x86_64 		0x40
15
? __switch_to
vmlinux 0xffffffff99c2959e
Build id: 3.10.0-862.el7.x86_64 		0xce
16
writeback_sb_inodes
vmlinux 0xffffffff99e49584
Build id: 3.10.0-862.el7.x86_64 		0x1c4
17
__writeback_inodes_wb
vmlinux 0xffffffff99e498ef
Build id: 3.10.0-862.el7.x86_64 		0x9f
18
wb_writeback
vmlinux 0xffffffff99e4a123
Build id: 3.10.0-862.el7.x86_64 		0x263
19
? bdi_dirty_limit
vmlinux 0xffffffff99da04d0
Build id: 3.10.0-862.el7.x86_64 		0x40
20
bdi_writeback_workfn
vmlinux 0xffffffff99e4a9ac
Build id: 3.10.0-862.el7.x86_64 		0x1cc
21
process_one_work
vmlinux 0xffffffff99cb2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffff99cb3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
23
? manage_workers.isra.24
vmlinux 0xffffffff99cb39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffff99cbae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffff99cbad60
Build id: 3.10.0-862.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff9a31f61d
Build id: 3.10.0-862.el7.x86_64 		0x7
27
? insert_kthread_work
vmlinux 0xffffffff99cbad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [kworker/u256:1:9320]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev pcspkr i2c_piix4 sg vmw_vmci nfit shpchp libnvdimm parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel
libata e1000 mptspi scsi_transport_spi serio_raw mptscsih i2c_core mptbase
CPU: 7 PID: 9320 Comm: kworker/u256:1 Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff93479eb0cf10 ti: ffff9346fead4000 task.ti: ffff9346fead4000
RIP: 0010:[<ffffffff99d07e3a>]  [<ffffffff99d07e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9346fead77e8  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff9347b6a5cfa0
RDX: 0000000000000001 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9346fead7820 R08: ffff9347b5e2cc00 R09: ffffffff99f4a3d9
R10: ffff9347b6bdbb00 R11: ffffcc7d44c88800 R12: 0000000000000292
R13: ffff9346fead7798 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff9347b6bc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe982307400 CR3: 000000013d60e000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffff99c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff99c771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff99dd8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff99dd1ba4>] page_mkclean+0x144/0x210
[<ffffffff99d9f7d0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc037dba4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc037dd00>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc037ec05>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc038357c>] ? ext4_writepages+0x42c/0xd40 [ext4]
[<ffffffffc03b1659>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc03835a7>] ext4_writepages+0x457/0xd40 [ext4]
[<ffffffff99da0e21>] do_writepages+0x21/0x50
[<ffffffff99e48af0>] __writeback_single_inode+0x40/0x260
[<ffffffff99c2959e>] ? __switch_to+0xce/0x580
[<ffffffff99e49584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff99e498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff99e4a123>] wb_writeback+0x263/0x2f0
[<ffffffff99da04d0>] ? bdi_dirty_limit+0x40/0xe0
[<ffffffff99e4a9ac>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffff99cb2dff>] process_one_work+0x17f/0x440
[<ffffffff99cb3ac6>] worker_thread+0x126/0x3c0
[<ffffffff99cb39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff99cbae31>] kthread+0xd1/0xe0
[<ffffffff99cbad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff9a31f61d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff99cbad60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 93 9a f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #682422
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff93f07e3a
Build id: 3.10.0-862.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff93e77018
Build id: 3.10.0-862.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff93e771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff93fd8837
Build id: 3.10.0-862.el7.x86_64 		0x67
5
page_mkclean
vmlinux 0xffffffff93fd1ba4
Build id: 3.10.0-862.el7.x86_64 		0x144
6
clear_page_dirty_for_io
vmlinux 0xffffffff93f9f7d0
Build id: 3.10.0-862.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffc0566ba4
Build id: 3.10.0-862.el7.x86_64 		0x24
8
mpage_process_page_bufs
ext4 0xffffffffc0566d00
Build id: 3.10.0-862.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffc0567c05
Build id: 3.10.0-862.el7.x86_64 		0x235
10
? ext4_writepages
ext4 0xffffffffc056c57c
Build id: 3.10.0-862.el7.x86_64 		0x42c
11
? __ext4_journal_start_sb
ext4 0xffffffffc059a659
Build id: 3.10.0-862.el7.x86_64 		0x69
12
ext4_writepages
ext4 0xffffffffc056c5a7
Build id: 3.10.0-862.el7.x86_64 		0x457
13
? update_curr
vmlinux 0xffffffff93ed816c
Build id: 3.10.0-862.el7.x86_64 		0x14c
14
? __schedule
vmlinux 0xffffffff9451291c
Build id: 3.10.0-862.el7.x86_64 		0x41c
15
do_writepages
vmlinux 0xffffffff93fa0e21
Build id: 3.10.0-862.el7.x86_64 		0x21
16
__writeback_single_inode
vmlinux 0xffffffff94048af0
Build id: 3.10.0-862.el7.x86_64 		0x40
17
writeback_sb_inodes
vmlinux 0xffffffff94049584
Build id: 3.10.0-862.el7.x86_64 		0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffff940498ef
Build id: 3.10.0-862.el7.x86_64 		0x9f
19
wb_writeback
vmlinux 0xffffffff9404a123
Build id: 3.10.0-862.el7.x86_64 		0x263
20
bdi_writeback_workfn
vmlinux 0xffffffff9404aaab
Build id: 3.10.0-862.el7.x86_64 		0x2cb
21
process_one_work
vmlinux 0xffffffff93eb2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
22
worker_thread
vmlinux 0xffffffff93eb3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
23
? manage_workers.isra.24
vmlinux 0xffffffff93eb39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
24
kthread
vmlinux 0xffffffff93ebae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
25
? insert_kthread_work
vmlinux 0xffffffff93ebad60
Build id: 3.10.0-862.el7.x86_64 		0x40
26
ret_from_fork_nospec_begin
vmlinux 0xffffffff9451f637
Build id: 3.10.0-862.el7.x86_64 		0x21
27
? insert_kthread_work
vmlinux 0xffffffff93ebad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [kworker/u256:2:14383]
Modules linked in: tcp_lp rfcomm fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink bnep vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel btusb btrtl btbcm aesni_intel btintel snd_ens1371 lrw gf128mul snd_rawmidi glue_helper bluetooth ablk_helper cryptd snd_ac97_codec ppdev ac97_bus snd_seq snd_seq_device snd_pcm rfkill snd_timer joydev snd pcspkr sg soundcore i2c_piix4 vmw_vmci shpchp nfit libnvdimm parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod cdrom
sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata mptspi scsi_transport_spi mptscsih e1000 serio_raw mptbase i2c_core
CPU: 7 PID: 14383 Comm: kworker/u256:2 Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff8b477b690fd0 ti: ffff8b4835ac4000 task.ti: ffff8b4835ac4000
RIP: 0010:[<ffffffff93f07e3a>]  [<ffffffff93f07e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8b4835ac77e8  EFLAGS: 00000202
RAX: 0000000000000003 RBX: 00000000000000fc RCX: ffff8b48396dcfa0
RDX: 0000000000000003 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8b4835ac7820 R08: ffff8b483762cc00 R09: ffffffff9414a3d9
R10: ffff8b48397dbb00 R11: ffffe82f41f1de00 R12: 0000000000000292
R13: ffff8b4835ac7798 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8b48397c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f289bdab8d0 CR3: 000000001780e000 CR4: 00000000003607e0
Call Trace:
[<ffffffff93e77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff93e771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff93fd8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff93fd1ba4>] page_mkclean+0x144/0x210
[<ffffffff93f9f7d0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc0566ba4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc0566d00>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc0567c05>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc056c57c>] ? ext4_writepages+0x42c/0xd40 [ext4]
[<ffffffffc059a659>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc056c5a7>] ext4_writepages+0x457/0xd40 [ext4]
[<ffffffff93ed816c>] ? update_curr+0x14c/0x1e0
[<ffffffff9451291c>] ? __schedule+0x41c/0xa20
[<ffffffff93fa0e21>] do_writepages+0x21/0x50
[<ffffffff94048af0>] __writeback_single_inode+0x40/0x260
[<ffffffff94049584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff940498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9404a123>] wb_writeback+0x263/0x2f0
[<ffffffff9404aaab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff93eb2dff>] process_one_work+0x17f/0x440
[<ffffffff93eb3ac6>] worker_thread+0x126/0x3c0
[<ffffffff93eb39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff93ebae31>] kthread+0xd1/0xe0
[<ffffffff93ebad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff9451f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff93ebad60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 b3 94 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #509788
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffba907e3a
Build id: 3.10.0-862.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffba877018
Build id: 3.10.0-862.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffba8771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffba9d8837
Build id: 3.10.0-862.el7.x86_64 		0x67
5
page_mkclean
vmlinux 0xffffffffba9d1ba4
Build id: 3.10.0-862.el7.x86_64 		0x144
6
clear_page_dirty_for_io
vmlinux 0xffffffffba99f7d0
Build id: 3.10.0-862.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffc03ddba4
Build id: 3.10.0-862.el7.x86_64 		0x24
8
mpage_process_page_bufs
ext4 0xffffffffc03ddd00
Build id: 3.10.0-862.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffc03dec05
Build id: 3.10.0-862.el7.x86_64 		0x235
10
? jbd2__journal_start
jbd2 0xffffffffc03bca93
Build id: 3.10.0-862.el7.x86_64 		0xf3
11
? ext4_writepages
ext4 0xffffffffc03e357c
Build id: 3.10.0-862.el7.x86_64 		0x42c
12
? __ext4_journal_start_sb
ext4 0xffffffffc0411659
Build id: 3.10.0-862.el7.x86_64 		0x69
13
ext4_writepages
ext4 0xffffffffc03e35a7
Build id: 3.10.0-862.el7.x86_64 		0x457
14
? submit_bio
vmlinux 0xffffffffbab1a940
Build id: 3.10.0-862.el7.x86_64 		0x70
15
do_writepages
vmlinux 0xffffffffba9a0e21
Build id: 3.10.0-862.el7.x86_64 		0x21
16
__writeback_single_inode
vmlinux 0xffffffffbaa48af0
Build id: 3.10.0-862.el7.x86_64 		0x40
17
writeback_sb_inodes
vmlinux 0xffffffffbaa49584
Build id: 3.10.0-862.el7.x86_64 		0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffffbaa498ef
Build id: 3.10.0-862.el7.x86_64 		0x9f
19
wb_writeback
vmlinux 0xffffffffbaa4a123
Build id: 3.10.0-862.el7.x86_64 		0x263
20
? get_nr_inodes
vmlinux 0xffffffffbaa3665c
Build id: 3.10.0-862.el7.x86_64 		0x4c
21
bdi_writeback_workfn
vmlinux 0xffffffffbaa4aaab
Build id: 3.10.0-862.el7.x86_64 		0x2cb
22
process_one_work
vmlinux 0xffffffffba8b2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
23
worker_thread
vmlinux 0xffffffffba8b3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
24
? manage_workers.isra.24
vmlinux 0xffffffffba8b39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
25
kthread
vmlinux 0xffffffffba8bae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
26
? insert_kthread_work
vmlinux 0xffffffffba8bad60
Build id: 3.10.0-862.el7.x86_64 		0x40
27
ret_from_fork_nospec_begin
vmlinux 0xffffffffbaf1f637
Build id: 3.10.0-862.el7.x86_64 		0x21
28
? insert_kthread_work
vmlinux 0xffffffffba8bad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [kworker/u32:4:30095]
Modules linked in: xt_statistic vxlan ip6_udp_tunnel udp_tunnel ipt_REJECT nf_reject_ipv4 nf_tables(T) ip_set ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs ip6table_mangle ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_filter iptable_mangle ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables xt_comment xt_mark xt_nat veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter bridge stp llc overlay(T) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ext4 mbcache jbd2 sunrpc sg 8139too virtio_balloon pcspkr joydev ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi ata_piix libata 8139cp virtio_pci virtio_ring mii serio_raw virtio dm_mirror
dm_region_hash dm_log dm_mod
CPU: 14 PID: 30095 Comm: kworker/u32:4 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff885be52dbf40 ti: ffff886653be4000 task.ti: ffff886653be4000
RIP: 0010:[<ffffffffba907e3a>]  [<ffffffffba907e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff886653be77e8  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0000000000019680 RCX: ffff886d6f21ceb8
RDX: 0000000000000000 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff886653be7820 R08: ffff885adf737400 R09: ffffffffbab4a3d9
R10: ffff886d6f59bb00 R11: fffff3d9b7508000 R12: ffff886653be7798
R13: 0000000069e30000 R14: ffffffffba9f5299 R15: ffffffffba996afe
FS:  0000000000000000(0000) GS:ffff886d6f580000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00005557d1b5d000 CR3: 0000000da2e72000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffba877018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffba8771b4>] flush_tlb_page+0x54/0xa0
[<ffffffffba9d8837>] ptep_clear_flush+0x67/0xa0
[<ffffffffba9d1ba4>] page_mkclean+0x144/0x210
[<ffffffffba99f7d0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc03ddba4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc03ddd00>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc03dec05>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc03bca93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc03e357c>] ? ext4_writepages+0x42c/0xd40 [ext4]
[<ffffffffc0411659>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc03e35a7>] ext4_writepages+0x457/0xd40 [ext4]
[<ffffffffbab1a940>] ? submit_bio+0x70/0x150
[<ffffffffba9a0e21>] do_writepages+0x21/0x50
[<ffffffffbaa48af0>] __writeback_single_inode+0x40/0x260
[<ffffffffbaa49584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbaa498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbaa4a123>] wb_writeback+0x263/0x2f0
[<ffffffffbaa3665c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffbaa4aaab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffba8b2dff>] process_one_work+0x17f/0x440
[<ffffffffba8b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffba8b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffba8bae31>] kthread+0xd1/0xe0
[<ffffffffba8bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffbaf1f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffba8bad60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 53 bb f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #475207
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff97707e3a
Build id: 3.10.0-862.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff97677018
Build id: 3.10.0-862.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff976771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff977d8837
Build id: 3.10.0-862.el7.x86_64 		0x67
5
page_mkclean
vmlinux 0xffffffff977d1ba4
Build id: 3.10.0-862.el7.x86_64 		0x144
6
clear_page_dirty_for_io
vmlinux 0xffffffff9779f7d0
Build id: 3.10.0-862.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffc04eeba4
Build id: 3.10.0-862.el7.x86_64 		0x24
8
mpage_process_page_bufs
ext4 0xffffffffc04eed00
Build id: 3.10.0-862.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffc04efc05
Build id: 3.10.0-862.el7.x86_64 		0x235
10
? jbd2__journal_start
jbd2 0xffffffffc04cda93
Build id: 3.10.0-862.el7.x86_64 		0xf3
11
? ext4_writepages
ext4 0xffffffffc04f457c
Build id: 3.10.0-862.el7.x86_64 		0x42c
12
? __ext4_journal_start_sb
ext4 0xffffffffc0522659
Build id: 3.10.0-862.el7.x86_64 		0x69
13
ext4_writepages
ext4 0xffffffffc04f45a7
Build id: 3.10.0-862.el7.x86_64 		0x457
14
? __schedule
vmlinux 0xffffffff97d12b08
Build id: 3.10.0-862.el7.x86_64 		0x608
15
? __schedule
vmlinux 0xffffffff97d12b14
Build id: 3.10.0-862.el7.x86_64 		0x614
16
? __schedule
vmlinux 0xffffffff97d12b08
Build id: 3.10.0-862.el7.x86_64 		0x608
17
do_writepages
vmlinux 0xffffffff977a0e21
Build id: 3.10.0-862.el7.x86_64 		0x21
18
__writeback_single_inode
vmlinux 0xffffffff97848af0
Build id: 3.10.0-862.el7.x86_64 		0x40
19
writeback_sb_inodes
vmlinux 0xffffffff97849584
Build id: 3.10.0-862.el7.x86_64 		0x1c4
20
__writeback_inodes_wb
vmlinux 0xffffffff978498ef
Build id: 3.10.0-862.el7.x86_64 		0x9f
21
wb_writeback
vmlinux 0xffffffff9784a123
Build id: 3.10.0-862.el7.x86_64 		0x263
22
? get_nr_inodes
vmlinux 0xffffffff9783665c
Build id: 3.10.0-862.el7.x86_64 		0x4c
23
bdi_writeback_workfn
vmlinux 0xffffffff9784aaab
Build id: 3.10.0-862.el7.x86_64 		0x2cb
24
process_one_work
vmlinux 0xffffffff976b2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
25
worker_thread
vmlinux 0xffffffff976b3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
26
? manage_workers.isra.24
vmlinux 0xffffffff976b39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
27
kthread
vmlinux 0xffffffff976bae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
28
? insert_kthread_work
vmlinux 0xffffffff976bad60
Build id: 3.10.0-862.el7.x86_64 		0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffff97d1f637
Build id: 3.10.0-862.el7.x86_64 		0x21
30
? insert_kthread_work
vmlinux 0xffffffff976bad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [kworker/u256:1:5097]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus crc32_pclmul snd_seq snd_seq_device ghash_clmulni_intel snd_pcm ppdev aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper cryptd snd_timer snd pcspkr joydev nfit sg soundcore vmw_vmci libnvdimm i2c_piix4 shpchp parport_pc parport ip_tables ext4 mbcache
jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix libata e1000 i2c_core mptspi serio_raw scsi_transport_spi mptscsih mptbase
CPU: 3 PID: 5097 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff8d874a343f40 ti: ffff8d8775150000 task.ti: ffff8d8775150000
RIP: 0010:[<ffffffff97707e3a>]  [<ffffffff97707e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8d87751537e8  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 00000000000000fc RCX: ffff8d87b565cf00
RDX: 0000000000000001 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff8d8775153820 R08: ffff8d86fc853c00 R09: ffffffff9794a3d9
R10: ffff8d87b56dbb00 R11: ffffe47e46c9d400 R12: 0000000000000292
R13: ffff8d8775153798 R14: ffff8d87b56d8b00 R15: 0000000000000003
FS:  0000000000000000(0000) GS:ffff8d87b56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa70fdc7850 CR3: 00000001ecb76000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffff97677018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff976771b4>] flush_tlb_page+0x54/0xa0
[<ffffffff977d8837>] ptep_clear_flush+0x67/0xa0
[<ffffffff977d1ba4>] page_mkclean+0x144/0x210
[<ffffffff9779f7d0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc04eeba4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc04eed00>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc04efc05>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc04cda93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc04f457c>] ? ext4_writepages+0x42c/0xd40 [ext4]
[<ffffffffc0522659>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc04f45a7>] ext4_writepages+0x457/0xd40 [ext4]
[<ffffffff97d12b08>] ? __schedule+0x608/0xa20
[<ffffffff97d12b14>] ? __schedule+0x614/0xa20
[<ffffffff97d12b08>] ? __schedule+0x608/0xa20
[<ffffffff977a0e21>] do_writepages+0x21/0x50
[<ffffffff97848af0>] __writeback_single_inode+0x40/0x260
[<ffffffff97849584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff978498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9784a123>] wb_writeback+0x263/0x2f0
[<ffffffff9783665c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff9784aaab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff976b2dff>] process_one_work+0x17f/0x440
[<ffffffff976b3ac6>] worker_thread+0x126/0x3c0
[<ffffffff976b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff976bae31>] kthread+0xd1/0xe0
[<ffffffff976bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffff97d1f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff976bad60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 33 98 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #189936
Frame # Function Binary Source or offset Line
1
queued_spin_lock_slowpath
vmlinux 0xffffffffa5d0dc5f
Build id: 3.10.0-862.14.4.el7.x86_64 		0xb
2
smp_call_function_many
vmlinux 0xffffffffa570b0eb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x22b
3
_raw_spin_lock
vmlinux 0xffffffffa5d1b750
Build id: 3.10.0-862.14.4.el7.x86_64 		0x20
4
native_flush_tlb_others
vmlinux 0xffffffffa5679e58
Build id: 3.10.0-862.14.4.el7.x86_64 		0xb8
5
flush_tlb_page
vmlinux 0xffffffffa5679ff4
Build id: 3.10.0-862.14.4.el7.x86_64 		0x54
6
tcp_v4_rcv
vmlinux 0xffffffffa5c5d92a
Build id: 3.10.0-862.14.4.el7.x86_64 		0x76a
7
ip_local_deliver_finish
vmlinux 0xffffffffa5c3685d
Build id: 3.10.0-862.14.4.el7.x86_64 		0xbd
8
ptep_clear_flush
vmlinux 0xffffffffa57dc0f7
Build id: 3.10.0-862.14.4.el7.x86_64 		0x67
9
page_mkclean
vmlinux 0xffffffffa57d53c4
Build id: 3.10.0-862.14.4.el7.x86_64 		0x144
10
clear_page_dirty_for_io
vmlinux 0xffffffffa57a2530
Build id: 3.10.0-862.14.4.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc0562ba4
Build id: 3.10.0-862.14.4.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc0562d00
Build id: 3.10.0-862.14.4.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc0563c05
Build id: 3.10.0-862.14.4.el7.x86_64 		0x235
14
? jbd2__journal_start
jbd2 0xffffffffc0541a93
Build id: 3.10.0-862.14.4.el7.x86_64 		0xf3
15
? ext4_writepages
ext4 0xffffffffc056857c
Build id: 3.10.0-862.14.4.el7.x86_64 		0x42c
16
? __ext4_journal_start_sb
ext4 0xffffffffc05966a9
Build id: 3.10.0-862.14.4.el7.x86_64 		0x69
17
ext4_writepages
ext4 0xffffffffc05685a7
Build id: 3.10.0-862.14.4.el7.x86_64 		0x457
18
? update_curr
vmlinux 0xffffffffa56db2fc
Build id: 3.10.0-862.14.4.el7.x86_64 		0x14c
19
do_writepages
vmlinux 0xffffffffa57a3b81
Build id: 3.10.0-862.14.4.el7.x86_64 		0x21
20
__writeback_single_inode
vmlinux 0xffffffffa584cfc0
Build id: 3.10.0-862.14.4.el7.x86_64 		0x40
21
writeback_sb_inodes
vmlinux 0xffffffffa584da54
Build id: 3.10.0-862.14.4.el7.x86_64 		0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffffa584ddbf
Build id: 3.10.0-862.14.4.el7.x86_64 		0x9f
23
wb_writeback
vmlinux 0xffffffffa584e5f3
Build id: 3.10.0-862.14.4.el7.x86_64 		0x263
24
? get_nr_inodes
vmlinux 0xffffffffa583ab0c
Build id: 3.10.0-862.14.4.el7.x86_64 		0x4c
25
bdi_writeback_workfn
vmlinux 0xffffffffa584ef7b
Build id: 3.10.0-862.14.4.el7.x86_64 		0x2cb
26
process_one_work
vmlinux 0xffffffffa56b613f
Build id: 3.10.0-862.14.4.el7.x86_64 		0x17f
27
worker_thread
vmlinux 0xffffffffa56b71d6
Build id: 3.10.0-862.14.4.el7.x86_64 		0x126
28
? manage_workers.isra.24
vmlinux 0xffffffffa56b70b0
Build id: 3.10.0-862.14.4.el7.x86_64 		0x2a0
29
kthread
vmlinux 0xffffffffa56bdf21
Build id: 3.10.0-862.14.4.el7.x86_64 		0xd1
30
? insert_kthread_work
vmlinux 0xffffffffa56bde50
Build id: 3.10.0-862.14.4.el7.x86_64 		0x40
31
ret_from_fork_nospec_begin
vmlinux 0xffffffffa5d255f7
Build id: 3.10.0-862.14.4.el7.x86_64 		0x21
32
? insert_kthread_work
vmlinux 0xffffffffa56bde50
Build id: 3.10.0-862.14.4.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 30s! [httpd:9868]
<IRQ> 
[<ffffffffa5d0dc5f>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffa570b0eb>] smp_call_function_many+0x22b/0x270
[<ffffffffa5d1b750>] _raw_spin_lock+0x20/0x30
[<ffffffffa5679e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa5679ff4>] flush_tlb_page+0x54/0xa0
[<ffffffffa5c5d92a>] tcp_v4_rcv+0x76a/0x9c0
[<ffffffffa5c3685d>] ip_local_deliver_finish+0xbd/0x200
[<ffffffffa57dc0f7>] ptep_clear_flush+0x67/0xa0
[<ffffffffa57d53c4>] page_mkclean+0x144/0x210
[<ffffffffa57a2530>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc0562ba4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc0562d00>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc0563c05>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc0541a93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc056857c>] ? ext4_writepages+0x42c/0xd40 [ext4]
[<ffffffffc05966a9>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc05685a7>] ext4_writepages+0x457/0xd40 [ext4]
[<ffffffffa56db2fc>] ? update_curr+0x14c/0x1e0
[<ffffffffa57a3b81>] do_writepages+0x21/0x50
[<ffffffffa584cfc0>] __writeback_single_inode+0x40/0x260
[<ffffffffa584da54>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa584ddbf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa584e5f3>] wb_writeback+0x263/0x2f0
[<ffffffffa583ab0c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa584ef7b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa56b613f>] process_one_work+0x17f/0x440
[<ffffffffa56b71d6>] worker_thread+0x126/0x3c0
[<ffffffffa56b70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffa56bdf21>] kthread+0xd1/0xe0
[<ffffffffa56bde50>] ? insert_kthread_work+0x40/0x40
[<ffffffffa5d255f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa56bde50>] ? insert_kthread_work+0x40/0x40
Complete report #21886
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa8507e3a
Build id: 3.10.0-862.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa8477018
Build id: 3.10.0-862.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffa84771b4
Build id: 3.10.0-862.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffa85d8837
Build id: 3.10.0-862.el7.x86_64 		0x67
5
page_mkclean
vmlinux 0xffffffffa85d1ba4
Build id: 3.10.0-862.el7.x86_64 		0x144
6
clear_page_dirty_for_io
vmlinux 0xffffffffa859f7d0
Build id: 3.10.0-862.el7.x86_64 		0x50
7
mpage_submit_page
ext4 0xffffffffc0263ba4
Build id: 3.10.0-862.el7.x86_64 		0x24
8
mpage_process_page_bufs
ext4 0xffffffffc0263d00
Build id: 3.10.0-862.el7.x86_64 		0x110
9
mpage_prepare_extent_to_map
ext4 0xffffffffc0264c05
Build id: 3.10.0-862.el7.x86_64 		0x235
10
? jbd2__journal_start
jbd2 0xffffffffc0242a93
Build id: 3.10.0-862.el7.x86_64 		0xf3
11
? ext4_writepages
ext4 0xffffffffc026957c
Build id: 3.10.0-862.el7.x86_64 		0x42c
12
? __ext4_journal_start_sb
ext4 0xffffffffc0297659
Build id: 3.10.0-862.el7.x86_64 		0x69
13
ext4_writepages
ext4 0xffffffffc02695a7
Build id: 3.10.0-862.el7.x86_64 		0x457
14
? generic_writepages
vmlinux 0xffffffffa859fd88
Build id: 3.10.0-862.el7.x86_64 		0x58
15
do_writepages
vmlinux 0xffffffffa85a0e21
Build id: 3.10.0-862.el7.x86_64 		0x21
16
__writeback_single_inode
vmlinux 0xffffffffa8648af0
Build id: 3.10.0-862.el7.x86_64 		0x40
17
writeback_sb_inodes
vmlinux 0xffffffffa8649584
Build id: 3.10.0-862.el7.x86_64 		0x1c4
18
__writeback_inodes_wb
vmlinux 0xffffffffa86498ef
Build id: 3.10.0-862.el7.x86_64 		0x9f
19
wb_writeback
vmlinux 0xffffffffa864a123
Build id: 3.10.0-862.el7.x86_64 		0x263
20
? get_nr_inodes
vmlinux 0xffffffffa863665c
Build id: 3.10.0-862.el7.x86_64 		0x4c
21
bdi_writeback_workfn
vmlinux 0xffffffffa864aaab
Build id: 3.10.0-862.el7.x86_64 		0x2cb
22
process_one_work
vmlinux 0xffffffffa84b2dff
Build id: 3.10.0-862.el7.x86_64 		0x17f
23
worker_thread
vmlinux 0xffffffffa84b3ac6
Build id: 3.10.0-862.el7.x86_64 		0x126
24
? manage_workers.isra.24
vmlinux 0xffffffffa84b39a0
Build id: 3.10.0-862.el7.x86_64 		0x2a0
25
kthread
vmlinux 0xffffffffa84bae31
Build id: 3.10.0-862.el7.x86_64 		0xd1
26
? insert_kthread_work
vmlinux 0xffffffffa84bad60
Build id: 3.10.0-862.el7.x86_64 		0x40
27
ret_from_fork_nospec_begin
vmlinux 0xffffffffa8b1f637
Build id: 3.10.0-862.el7.x86_64 		0x21
28
? insert_kthread_work
vmlinux 0xffffffffa84bad60
Build id: 3.10.0-862.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u256:2:5605]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec aesni_intel lrw gf128mul glue_helper ac97_bus ablk_helper snd_seq cryptd snd_seq_device snd_pcm snd_timer snd soundcore nfit libnvdimm ppdev vmw_balloon pcspkr sg joydev parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi
vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common crc32c_intel libata e1000 mptspi scsi_transport_spi serio_raw mptscsih mptbase i2c_core
CPU: 3 PID: 5605 Comm: kworker/u256:2 Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff881cd41ddee0 ti: ffff881d75420000 task.ti: ffff881d75420000
RIP: 0010:[<ffffffffa8507e3a>]  [<ffffffffa8507e3a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff881d754237e8  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000000fc RCX: ffff881d7961cf00
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff881d75423820 R08: ffff881d7762bc00 R09: ffffffffa874a3d9
R10: ffff881d796dbb00 R11: fffffde2002a5e00 R12: 0000000000000292
R13: ffff881d75423798 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff881d796c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5873e35050 CR3: 000000006ea0e000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa8477018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa84771b4>] flush_tlb_page+0x54/0xa0
[<ffffffffa85d8837>] ptep_clear_flush+0x67/0xa0
[<ffffffffa85d1ba4>] page_mkclean+0x144/0x210
[<ffffffffa859f7d0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc0263ba4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc0263d00>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc0264c05>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc0242a93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc026957c>] ? ext4_writepages+0x42c/0xd40 [ext4]
[<ffffffffc0297659>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc02695a7>] ext4_writepages+0x457/0xd40 [ext4]
[<ffffffffa859fd88>] ? generic_writepages+0x58/0x80
[<ffffffffa85a0e21>] do_writepages+0x21/0x50
[<ffffffffa8648af0>] __writeback_single_inode+0x40/0x260
[<ffffffffa8649584>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa86498ef>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa864a123>] wb_writeback+0x263/0x2f0
[<ffffffffa863665c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa864aaab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa84b2dff>] process_one_work+0x17f/0x440
[<ffffffffa84b3ac6>] worker_thread+0x126/0x3c0
[<ffffffffa84b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffffa84bae31>] kthread+0xd1/0xe0
[<ffffffffa84bad60>] ? insert_kthread_work+0x40/0x40
[<ffffffffa8b1f637>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa84bad60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 8e b6 c3 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 53 13 a9 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 5d b6 c3 00 eb b7 0f b6 4d cc 4c 89
Complete report #922307
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8eb1173a
Build id: 3.10.0-957.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8ea7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8ea7a734
Build id: 3.10.0-957.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff8ebfd587
Build id: 3.10.0-957.el7.x86_64 		0x67
5
page_mkclean_one
vmlinux 0xffffffff8ebf6b66
Build id: 3.10.0-957.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffff8ebf8983
Build id: 3.10.0-957.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffff8ebf8d96
Build id: 3.10.0-957.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffff8ebf6ac0
Build id: 3.10.0-957.el7.x86_64 		0x180
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffff8ebf6200
Build id: 3.10.0-957.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffff8ebc1d60
Build id: 3.10.0-957.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc03bfd54
Build id: 3.10.0-957.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc03bfeb0
Build id: 3.10.0-957.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc03c1255
Build id: 3.10.0-957.el7.x86_64 		0x235
14
? jbd2__journal_start
jbd2 0xffffffffc02f5a93
Build id: 3.10.0-957.el7.x86_64 		0xf3
15
? ext4_writepages
ext4 0xffffffffc03c591c
Build id: 3.10.0-957.el7.x86_64 		0x43c
16
? __ext4_journal_start_sb
ext4 0xffffffffc03f4c59
Build id: 3.10.0-957.el7.x86_64 		0x69
17
ext4_writepages
ext4 0xffffffffc03c5947
Build id: 3.10.0-957.el7.x86_64 		0x467
18
do_writepages
vmlinux 0xffffffff8ebc3181
Build id: 3.10.0-957.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffff8ec6f930
Build id: 3.10.0-957.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffff8ec703c4
Build id: 3.10.0-957.el7.x86_64 		0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffff8ec7072f
Build id: 3.10.0-957.el7.x86_64 		0x9f
22
wb_writeback
vmlinux 0xffffffff8ec70f63
Build id: 3.10.0-957.el7.x86_64 		0x263
23
? get_nr_inodes
vmlinux 0xffffffff8ec5cd7c
Build id: 3.10.0-957.el7.x86_64 		0x4c
24
bdi_writeback_workfn
vmlinux 0xffffffff8ec718eb
Build id: 3.10.0-957.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffff8eab9d4f
Build id: 3.10.0-957.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffff8eabade6
Build id: 3.10.0-957.el7.x86_64 		0x126
27
? manage_workers.isra.25
vmlinux 0xffffffff8eabacc0
Build id: 3.10.0-957.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffff8eac1c31
Build id: 3.10.0-957.el7.x86_64 		0xd1
29
? insert_kthread_work
vmlinux 0xffffffff8eac1b60
Build id: 3.10.0-957.el7.x86_64 		0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffff8f174c37
Build id: 3.10.0-957.el7.x86_64 		0x21
31
? insert_kthread_work
vmlinux 0xffffffff8eac1b60
Build id: 3.10.0-957.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 23s! [kworker/u51:2:677]
Modules linked in: nfsv3 nfs_acl tcp_diag inet_diag bmhook(OE) dsa_filter(POE) dsa_filter_hook(OE) bmsensor(OE) tmhook(OE) iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi ppdev crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci parport_pc i2c_piix4 parport ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect crc32c_intel sysimgblt fb_sys_fops serio_raw ttm ahci ata_piix libahci drm vmxnet3 libata nfit vmw_pvscsi libnvdimm drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 13 PID: 677 Comm: kworker/u51:2 Tainted: P           OEL ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff9de014fcd140 ti: ffff9ddf602ec000 task.ti: ffff9ddf602ec000
RIP: 0010:[<ffffffff8eb1173a>]  [<ffffffff8eb1173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9ddf602ef748  EFLAGS: 00000202
RAX: 000000000000000c RBX: 00000000000000fc RCX: ffff9de02fc208c8
RDX: 000000000000000c RSI: 0000000000000018 RDI: 0000000000000000
RBP: ffff9ddf602ef780 R08: ffff9ddcbfc0ac00 R09: ffffffff8ed755c9
R10: ffff9de02fc5f160 R11: fffff220ed71bc00 R12: 0000000000000296
R13: ffff9ddf602ef6f8 R14: ffff9de014fcd140 R15: ffff9de02e53d0c0
FS:  0000000000000000(0000) GS:ffff9de02fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f24bb1733e0 CR3: 0000000c2e5ca000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8ea7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ea7a734>] flush_tlb_page+0x54/0xa0
[<ffffffff8ebfd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff8ebf6b66>] page_mkclean_one+0xa6/0xe0
[<ffffffff8ebf8983>] rmap_walk+0xf3/0x360
[<ffffffff8ebf8d96>] page_mkclean+0x76/0xb0
[<ffffffff8ebf6ac0>] ? page_referenced_one+0x180/0x180
[<ffffffff8ebf6200>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffff8ebc1d60>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc03bfd54>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc03bfeb0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc03c1255>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc02f5a93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc03c591c>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc03f4c59>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc03c5947>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffff8ebc3181>] do_writepages+0x21/0x50
[<ffffffff8ec6f930>] __writeback_single_inode+0x40/0x260
[<ffffffff8ec703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8ec7072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8ec70f63>] wb_writeback+0x263/0x2f0
[<ffffffff8ec5cd7c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8ec718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8eab9d4f>] process_one_work+0x17f/0x440
[<ffffffff8eabade6>] worker_thread+0x126/0x3c0
[<ffffffff8eabacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff8eac1c31>] kthread+0xd1/0xe0
[<ffffffff8eac1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff8f174c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff8eac1b60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 74 8f f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #387175
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa8b1185a
Build id: 3.10.0-957.10.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa8a7a5c8
Build id: 3.10.0-957.10.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffa8a7a764
Build id: 3.10.0-957.10.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffa8bfda17
Build id: 3.10.0-957.10.1.el7.x86_64 		0x67
5
page_mkclean_one
vmlinux 0xffffffffa8bf6ff6
Build id: 3.10.0-957.10.1.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffffa8bf8e13
Build id: 3.10.0-957.10.1.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffffa8bf9226
Build id: 3.10.0-957.10.1.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffffa8bf6f50
Build id: 3.10.0-957.10.1.el7.x86_64 		0x180
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffffa8bf6690
Build id: 3.10.0-957.10.1.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffffa8bc20f0
Build id: 3.10.0-957.10.1.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc06add54
Build id: 3.10.0-957.10.1.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc06adeb0
Build id: 3.10.0-957.10.1.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc06af255
Build id: 3.10.0-957.10.1.el7.x86_64 		0x235
14
? ext4_writepages
ext4 0xffffffffc06b391c
Build id: 3.10.0-957.10.1.el7.x86_64 		0x43c
15
? __ext4_journal_start_sb
ext4 0xffffffffc06e2c59
Build id: 3.10.0-957.10.1.el7.x86_64 		0x69
16
ext4_writepages
ext4 0xffffffffc06b3947
Build id: 3.10.0-957.10.1.el7.x86_64 		0x467
17
do_writepages
vmlinux 0xffffffffa8bc3511
Build id: 3.10.0-957.10.1.el7.x86_64 		0x21
18
__writeback_single_inode
vmlinux 0xffffffffa8c6ffa0
Build id: 3.10.0-957.10.1.el7.x86_64 		0x40
19
writeback_sb_inodes
vmlinux 0xffffffffa8c70a34
Build id: 3.10.0-957.10.1.el7.x86_64 		0x1c4
20
__writeback_inodes_wb
vmlinux 0xffffffffa8c70d9f
Build id: 3.10.0-957.10.1.el7.x86_64 		0x9f
21
wb_writeback
vmlinux 0xffffffffa8c715d3
Build id: 3.10.0-957.10.1.el7.x86_64 		0x263
22
? get_nr_inodes
vmlinux 0xffffffffa8c5d3ec
Build id: 3.10.0-957.10.1.el7.x86_64 		0x4c
23
bdi_writeback_workfn
vmlinux 0xffffffffa8c71f5b
Build id: 3.10.0-957.10.1.el7.x86_64 		0x2cb
24
process_one_work
vmlinux 0xffffffffa8ab9d8f
Build id: 3.10.0-957.10.1.el7.x86_64 		0x17f
25
worker_thread
vmlinux 0xffffffffa8abae26
Build id: 3.10.0-957.10.1.el7.x86_64 		0x126
26
? manage_workers.isra.25
vmlinux 0xffffffffa8abad00
Build id: 3.10.0-957.10.1.el7.x86_64 		0x2a0
27
kthread
vmlinux 0xffffffffa8ac1c71
Build id: 3.10.0-957.10.1.el7.x86_64 		0xd1
28
? insert_kthread_work
vmlinux 0xffffffffa8ac1ba0
Build id: 3.10.0-957.10.1.el7.x86_64 		0x40
29
ret_from_fork_nospec_begin
vmlinux 0xffffffffa9175c37
Build id: 3.10.0-957.10.1.el7.x86_64 		0x21
30
? insert_kthread_work
vmlinux 0xffffffffa8ac1ba0
Build id: 3.10.0-957.10.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 67s! [kworker/u16:0:6]
Modules linked in: xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) ext4 mbcache jbd2 ppdev sg pcspkr virtio_balloon i2c_piix4 parport_pc parport joydev ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom virtio_net virtio_scsi ata_generic pata_acpi bochs_drm drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix serio_raw drm libata virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 6 Comm: kworker/u16:0 Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.10.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
Workqueue: writeback bdi_writeback_workfn (flush-8:16)
task: ffff8e05b4855140 ti: ffff8e05b4890000 task.ti: ffff8e05b4890000
RIP: 0010:[<ffffffffa8b1185a>]  [<ffffffffa8b1185a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8e05b4893748  EFLAGS: 00000202
RAX: 0000000000000007 RBX: 00000000000000fc RCX: ffff8e0c4fde0348
RDX: 0000000000000007 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff8e05b4893780 R08: ffff8e05b49a0800 R09: ffffffffa8d75c99
R10: ffff8e0c4fd5f160 R11: ffffea9d9ec9e200 R12: 0000000000000296
R13: ffff8e05b48936f8 R14: ffffffffa8bc0f96 R15: ffff8e05b4893748
FS:  0000000000000000(0000) GS:ffff8e0c4fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4bab9a9080 CR3: 000000064da10000 CR4: 00000000000006e0
Call Trace:
[<ffffffffa8a7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa8a7a764>] flush_tlb_page+0x54/0xa0
[<ffffffffa8bfda17>] ptep_clear_flush+0x67/0xa0
[<ffffffffa8bf6ff6>] page_mkclean_one+0xa6/0xe0
[<ffffffffa8bf8e13>] rmap_walk+0xf3/0x360
[<ffffffffa8bf9226>] page_mkclean+0x76/0xb0
[<ffffffffa8bf6f50>] ? page_referenced_one+0x180/0x180
[<ffffffffa8bf6690>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffffa8bc20f0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc06add54>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc06adeb0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc06af255>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc06b391c>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc06e2c59>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc06b3947>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffffa8bc3511>] do_writepages+0x21/0x50
[<ffffffffa8c6ffa0>] __writeback_single_inode+0x40/0x260
[<ffffffffa8c70a34>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffa8c70d9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa8c715d3>] wb_writeback+0x263/0x2f0
[<ffffffffa8c5d3ec>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa8c71f5b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa8ab9d8f>] process_one_work+0x17f/0x440
[<ffffffffa8abae26>] worker_thread+0x126/0x3c0
[<ffffffffa8abad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffa8ac1c71>] kthread+0xd1/0xe0
[<ffffffffa8ac1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa9175c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa8ac1ba0>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 6e 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 b9 74 a9 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 82 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #298454
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8691173a
Build id: 3.10.0-957.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8687a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8687a734
Build id: 3.10.0-957.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff869fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
5
page_mkclean_one
vmlinux 0xffffffff869f6b66
Build id: 3.10.0-957.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffff869f8983
Build id: 3.10.0-957.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffff869f8d96
Build id: 3.10.0-957.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffff869f6ac0
Build id: 3.10.0-957.el7.x86_64 		0x180
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffff869f6200
Build id: 3.10.0-957.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffff869c1d60
Build id: 3.10.0-957.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc04edd54
Build id: 3.10.0-957.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc04edeb0
Build id: 3.10.0-957.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc04ef255
Build id: 3.10.0-957.el7.x86_64 		0x235
14
? jbd2__journal_start
jbd2 0xffffffffc04cba93
Build id: 3.10.0-957.el7.x86_64 		0xf3
15
? ext4_writepages
ext4 0xffffffffc04f391c
Build id: 3.10.0-957.el7.x86_64 		0x43c
16
? __ext4_journal_start_sb
ext4 0xffffffffc0522c59
Build id: 3.10.0-957.el7.x86_64 		0x69
17
ext4_writepages
ext4 0xffffffffc04f3947
Build id: 3.10.0-957.el7.x86_64 		0x467
18
do_writepages
vmlinux 0xffffffff869c3181
Build id: 3.10.0-957.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffff86a6f930
Build id: 3.10.0-957.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffff86a703c4
Build id: 3.10.0-957.el7.x86_64 		0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffff86a7072f
Build id: 3.10.0-957.el7.x86_64 		0x9f
22
wb_writeback
vmlinux 0xffffffff86a70f63
Build id: 3.10.0-957.el7.x86_64 		0x263
23
? get_nr_inodes
vmlinux 0xffffffff86a5cd7c
Build id: 3.10.0-957.el7.x86_64 		0x4c
24
bdi_writeback_workfn
vmlinux 0xffffffff86a718eb
Build id: 3.10.0-957.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffff868b9d4f
Build id: 3.10.0-957.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffff868bade6
Build id: 3.10.0-957.el7.x86_64 		0x126
27
? manage_workers.isra.25
vmlinux 0xffffffff868bacc0
Build id: 3.10.0-957.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffff868c1c31
Build id: 3.10.0-957.el7.x86_64 		0xd1 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [java:30110]
Modules linked in: kbase(OE) syshook_linux(OE) binfmt_misc udp_diag tcp_diag inet_diag fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel virtio_balloon aesni_intel parport_pc lrw parport gf128mul glue_helper i2c_piix4 ablk_helper cryptd joydev pcspkr ip_tables ext4 mbcache jbd2 virtio_console
virtio_blk virtio_net ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kbase]
CPU: 6 PID: 30110 Comm: java Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
task: ffff9b4fc2fe4100 ti: ffff9b50c3978000 task.ti: ffff9b50c3978000
RIP: 0010:[<ffffffff8691173a>]  [<ffffffff8691173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9b50c397bb78  EFLAGS: 00000202
RAX: 0000000000000007 RBX: 00000000000000fc RCX: ffff9b5fbf3e0398
NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [kworker/u32:0:4853]
Modules linked in: kbase(OE) syshook_linux(OE) binfmt_misc udp_diag tcp_diag inet_diag fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel virtio_balloon aesni_intel parport_pc lrw parport gf128mul glue_helper i2c_piix4 ablk_helper cryptd joydev pcspkr ip_tables ext4 mbcache jbd2 virtio_console
virtio_blk virtio_net ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy ata_piix drm crct10dif_pclmul crct10dif_common crc32c_intel libata serio_raw drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod [last unloaded: kbase]
CPU: 15 PID: 4853 Comm: kworker/u32:0 Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9b4fc5b830c0 ti: ffff9b4fc2de4000 task.ti: ffff9b4fc2de4000
RIP: 0010:[<ffffffff8691173a>]  [<ffffffff8691173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9b4fc2de7748  EFLAGS: 00000202
RAX: 0000000000000007 RBX: 00000000000000fc RCX: ffff9b5fbf3e0618
RDX: 0000000000000007 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9b4fc2de7780 R08: ffff9b51294f6800 R09: ffffffff86b755c9
R10: ffff9b5fbf5df160 R11: fffffa0bc020da00 R12: 0000000000000296
R13: ffff9b4fc2de76f8 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff9b5fbf5c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd2a34ffa08 CR3: 0000000fbcbe8000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8687a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8687a734>] flush_tlb_page+0x54/0xa0
[<ffffffff869fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff869f6b66>] page_mkclean_one+0xa6/0xe0
[<ffffffff869f8983>] rmap_walk+0xf3/0x360
[<ffffffff869f8d96>] page_mkclean+0x76/0xb0
[<ffffffff869f6ac0>] ? page_referenced_one+0x180/0x180
[<ffffffff869f6200>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffff869c1d60>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc04edd54>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc04edeb0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc04ef255>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc04cba93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc04f391c>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc0522c59>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc04f3947>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffff869c3181>] do_writepages+0x21/0x50
[<ffffffff86a6f930>] __writeback_single_inode+0x40/0x260
[<ffffffff86a703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff86a7072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff86a70f63>] wb_writeback+0x263/0x2f0
[<ffffffff86a5cd7c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff86a718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff868b9d4f>] process_one_work+0x17f/0x440
[<ffffffff868bade6>] worker_thread+0x126/0x3c0
[<ffffffff868bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff868c1c31>] kthread+0xd1/0xe0
Complete report #188032
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff8751181a
Build id: 3.10.0-957.1.3.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff8747a598
Build id: 3.10.0-957.1.3.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff8747a734
Build id: 3.10.0-957.1.3.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff875fd7f7
Build id: 3.10.0-957.1.3.el7.x86_64 		0x67
5
page_mkclean_one
vmlinux 0xffffffff875f6dd6
Build id: 3.10.0-957.1.3.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffff875f8bf3
Build id: 3.10.0-957.1.3.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffff875f9006
Build id: 3.10.0-957.1.3.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffff875f6d30
Build id: 3.10.0-957.1.3.el7.x86_64 		0x180
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffff875f6470
Build id: 3.10.0-957.1.3.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffff875c1fd0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc06acd54
Build id: 3.10.0-957.1.3.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc06aceb0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc06ae255
Build id: 3.10.0-957.1.3.el7.x86_64 		0x235
14
? ext4_journal_check_start
ext4 0xffffffffc06e1a82
Build id: 3.10.0-957.1.3.el7.x86_64 		0x12
15
? __ext4_journal_start_sb
ext4 0xffffffffc06e1cc5
Build id: 3.10.0-957.1.3.el7.x86_64 		0xd5
16
ext4_writepages
ext4 0xffffffffc06b2947
Build id: 3.10.0-957.1.3.el7.x86_64 		0x467
17
? submit_bio
vmlinux 0xffffffff877451a0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x70
18
? generic_writepages
vmlinux 0xffffffff875c2588
Build id: 3.10.0-957.1.3.el7.x86_64 		0x58
19
do_writepages
vmlinux 0xffffffff875c33f1
Build id: 3.10.0-957.1.3.el7.x86_64 		0x21
20
__writeback_single_inode
vmlinux 0xffffffff8766fba0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40
21
writeback_sb_inodes
vmlinux 0xffffffff87670634
Build id: 3.10.0-957.1.3.el7.x86_64 		0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffff8767099f
Build id: 3.10.0-957.1.3.el7.x86_64 		0x9f
23
wb_writeback
vmlinux 0xffffffff876711d3
Build id: 3.10.0-957.1.3.el7.x86_64 		0x263
24
? get_nr_inodes
vmlinux 0xffffffff8765cfec
Build id: 3.10.0-957.1.3.el7.x86_64 		0x4c
25
bdi_writeback_workfn
vmlinux 0xffffffff87671b5b
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2cb
26
process_one_work
vmlinux 0xffffffff874b9d4f
Build id: 3.10.0-957.1.3.el7.x86_64 		0x17f
27
worker_thread
vmlinux 0xffffffff874bade6
Build id: 3.10.0-957.1.3.el7.x86_64 		0x126
28
? manage_workers.isra.25
vmlinux 0xffffffff874bacc0
Build id: 3.10.0-957.1.3.el7.x86_64 		0x2a0
29
kthread
vmlinux 0xffffffff874c1c31
Build id: 3.10.0-957.1.3.el7.x86_64 		0xd1
30
? insert_kthread_work
vmlinux 0xffffffff874c1b60
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40
31
ret_from_fork_nospec_begin
vmlinux 0xffffffff87b74c37
Build id: 3.10.0-957.1.3.el7.x86_64 		0x21
32
? insert_kthread_work
vmlinux 0xffffffff874c1b60
Build id: 3.10.0-957.1.3.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [kworker/u16:2:19187]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac loop coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod sr_mod
cdrom crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic pata_acpi vmxnet3 vmwgfx drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 19187 Comm: kworker/u16:2 Kdump: loaded Tainted: G             L ------------   3.10.0-957.1.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Workqueue: writeback bdi_writeback_workfn (flush-7:1)
task: ffff8d1cecef5140 ti: ffff8d1b04ab4000 task.ti: ffff8d1b04ab4000
RIP: 0010:[<ffffffff8751181a>]  [<ffffffff8751181a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff8d1b04ab7748  EFLAGS: 00000202
RAX: 0000000000000001 RBX: 000000000001b700 RCX: ffff8d1e3fc606a0
RDX: 0000000000000001 RSI: 0000000000000008 RDI: 0000000000000000
RBP: ffff8d1b04ab7780 R08: ffff8d1b79c62400 R09: ffffffff87775839
R10: ffff8d1e3fddf160 R11: ffffc22c8bbef600 R12: ffffffff8777578f
R13: 000000000001c000 R14: 0000000000015ab9 R15: ffffffff884e9c00
FS:  0000000000000000(0000) GS:ffff8d1e3fdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f360e30f000 CR3: 0000000143810000 CR4: 00000000000407e0
Call Trace:
[<ffffffff8747a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8747a734>] flush_tlb_page+0x54/0xa0
[<ffffffff875fd7f7>] ptep_clear_flush+0x67/0xa0
[<ffffffff875f6dd6>] page_mkclean_one+0xa6/0xe0
[<ffffffff875f8bf3>] rmap_walk+0xf3/0x360
[<ffffffff875f9006>] page_mkclean+0x76/0xb0
[<ffffffff875f6d30>] ? page_referenced_one+0x180/0x180
[<ffffffff875f6470>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffff875c1fd0>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc06acd54>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc06aceb0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc06ae255>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc06e1a82>] ? ext4_journal_check_start+0x12/0x90 [ext4]
[<ffffffffc06e1cc5>] ? __ext4_journal_start_sb+0xd5/0xe0 [ext4]
[<ffffffffc06b2947>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffff877451a0>] ? submit_bio+0x70/0x150
[<ffffffff875c2588>] ? generic_writepages+0x58/0x80
[<ffffffff875c33f1>] do_writepages+0x21/0x50
[<ffffffff8766fba0>] __writeback_single_inode+0x40/0x260
[<ffffffff87670634>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8767099f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff876711d3>] wb_writeback+0x263/0x2f0
[<ffffffff8765cfec>] ? get_nr_inodes+0x4c/0x70
[<ffffffff87671b5b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff874b9d4f>] process_one_work+0x17f/0x440
[<ffffffff874bade6>] worker_thread+0x126/0x3c0
[<ffffffff874bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff874c1c31>] kthread+0xd1/0xe0
[<ffffffff874c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff87b74c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff874c1b60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 6e 82 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 14 88 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 3d 82 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #184165
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff9791173a
Build id: 3.10.0-957.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffff9787a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff9787a734
Build id: 3.10.0-957.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff979fd587
Build id: 3.10.0-957.el7.x86_64 		0x67
5
page_mkclean_one
vmlinux 0xffffffff979f6b66
Build id: 3.10.0-957.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffff979f8983
Build id: 3.10.0-957.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffff979f8d96
Build id: 3.10.0-957.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffff979f6ac0
Build id: 3.10.0-957.el7.x86_64 		0x180
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffff979f6200
Build id: 3.10.0-957.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffff979c1d60
Build id: 3.10.0-957.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc046dd54
Build id: 3.10.0-957.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc046deb0
Build id: 3.10.0-957.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc046f255
Build id: 3.10.0-957.el7.x86_64 		0x235
14
? jbd2__journal_start
jbd2 0xffffffffc044ba93
Build id: 3.10.0-957.el7.x86_64 		0xf3
15
? ext4_writepages
ext4 0xffffffffc047391c
Build id: 3.10.0-957.el7.x86_64 		0x43c
16
? __ext4_journal_start_sb
ext4 0xffffffffc04a2c59
Build id: 3.10.0-957.el7.x86_64 		0x69
17
ext4_writepages
ext4 0xffffffffc0473947
Build id: 3.10.0-957.el7.x86_64 		0x467
18
? __schedule
vmlinux 0xffffffff97f6770f
Build id: 3.10.0-957.el7.x86_64 		0x3ff
19
do_writepages
vmlinux 0xffffffff979c3181
Build id: 3.10.0-957.el7.x86_64 		0x21
20
__writeback_single_inode
vmlinux 0xffffffff97a6f930
Build id: 3.10.0-957.el7.x86_64 		0x40
21
writeback_sb_inodes
vmlinux 0xffffffff97a703c4
Build id: 3.10.0-957.el7.x86_64 		0x1c4
22
__writeback_inodes_wb
vmlinux 0xffffffff97a7072f
Build id: 3.10.0-957.el7.x86_64 		0x9f
23
wb_writeback
vmlinux 0xffffffff97a70f63
Build id: 3.10.0-957.el7.x86_64 		0x263
24
bdi_writeback_workfn
vmlinux 0xffffffff97a718eb
Build id: 3.10.0-957.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffff978b9d4f
Build id: 3.10.0-957.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffff978bade6
Build id: 3.10.0-957.el7.x86_64 		0x126
27
? manage_workers.isra.25
vmlinux 0xffffffff978bacc0
Build id: 3.10.0-957.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffff978c1c31
Build id: 3.10.0-957.el7.x86_64 		0xd1
29
? insert_kthread_work
vmlinux 0xffffffff978c1b60
Build id: 3.10.0-957.el7.x86_64 		0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffff97f74c1d
Build id: 3.10.0-957.el7.x86_64 		0x7
31
? insert_kthread_work
vmlinux 0xffffffff978c1b60
Build id: 3.10.0-957.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 28s! [kworker/u256:2:25662]
Modules linked in: tcp_lp fuse xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter devlink ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec joydev ac97_bus pcspkr snd_seq snd_seq_device snd_pcm sg snd_timer snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sd_mod sr_mod cdrom crc_t10dif crct10dif_generic
ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm e1000 mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks ata_piix libata nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 25662 Comm: kworker/u256:2 Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff9223a80e2080 ti: ffff9223b9958000 task.ti: ffff9223b9958000
RIP: 0010:[<ffffffff9791173a>]  [<ffffffff9791173a>] smp_call_function_many+0x20a/0x270
RSP: 0018:ffff9223b995b748  EFLAGS: 00000202
RAX: 0000000000000002 RBX: 00000000000000fc RCX: ffff9225b369f610
RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000000000000000
RBP: ffff9223b995b780 R08: ffff92247fd8d000 R09: ffffffff97b755c9
R10: ffff9225b361f160 R11: ffffe0a8c2ba4400 R12: 0000000000000296
R13: 0000000000000018 R14: ffff9223b995b6b8 R15: 0000000000000296
FS:  0000000000000000(0000) GS:ffff9225b3600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6446621000 CR3: 0000000012c10000 CR4: 00000000003607f0
Call Trace:
[<ffffffff9787a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9787a734>] flush_tlb_page+0x54/0xa0
[<ffffffff979fd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff979f6b66>] page_mkclean_one+0xa6/0xe0
[<ffffffff979f8983>] rmap_walk+0xf3/0x360
[<ffffffff979f8d96>] page_mkclean+0x76/0xb0
[<ffffffff979f6ac0>] ? page_referenced_one+0x180/0x180
[<ffffffff979f6200>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffff979c1d60>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc046dd54>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc046deb0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc046f255>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc044ba93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc047391c>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc04a2c59>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc0473947>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffff97f6770f>] ? __schedule+0x3ff/0x890
[<ffffffff979c3181>] do_writepages+0x21/0x50
[<ffffffff97a6f930>] __writeback_single_inode+0x40/0x260
[<ffffffff97a703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff97a7072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff97a70f63>] wb_writeback+0x263/0x2f0
[<ffffffff97a718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff978b9d4f>] process_one_work+0x17f/0x440
[<ffffffff978bade6>] worker_thread+0x126/0x3c0
[<ffffffff978bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff978c1c31>] kthread+0xd1/0xe0
[<ffffffff978c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff97f74c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff978c1b60>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 4e 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 54 98 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89
Complete report #168331
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffff82d1173e
Build id: 3.10.0-957.el7.x86_64 		0x20e
2
native_flush_tlb_others
vmlinux 0xffffffff82c7a598
Build id: 3.10.0-957.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffff82c7a734
Build id: 3.10.0-957.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffff82dfd587
Build id: 3.10.0-957.el7.x86_64 		0x67
5
page_mkclean_one
vmlinux 0xffffffff82df6b66
Build id: 3.10.0-957.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffff82df8983
Build id: 3.10.0-957.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffff82df8d96
Build id: 3.10.0-957.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffff82df6ac0
Build id: 3.10.0-957.el7.x86_64 		0x180
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffff82df6200
Build id: 3.10.0-957.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffff82dc1d60
Build id: 3.10.0-957.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc03c3d54
Build id: 3.10.0-957.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc03c3eb0
Build id: 3.10.0-957.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc03c5255
Build id: 3.10.0-957.el7.x86_64 		0x235
14
? ext4_writepages
ext4 0xffffffffc03c991c
Build id: 3.10.0-957.el7.x86_64 		0x43c
15
? __ext4_journal_start_sb
ext4 0xffffffffc03f8c59
Build id: 3.10.0-957.el7.x86_64 		0x69
16
ext4_writepages
ext4 0xffffffffc03c9947
Build id: 3.10.0-957.el7.x86_64 		0x467
17
? account_entity_dequeue
vmlinux 0xffffffff82cdca2e
Build id: 3.10.0-957.el7.x86_64 		0xae
18
do_writepages
vmlinux 0xffffffff82dc3181
Build id: 3.10.0-957.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffff82e6f930
Build id: 3.10.0-957.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffff82e703c4
Build id: 3.10.0-957.el7.x86_64 		0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffff82e7072f
Build id: 3.10.0-957.el7.x86_64 		0x9f
22
wb_writeback
vmlinux 0xffffffff82e70f63
Build id: 3.10.0-957.el7.x86_64 		0x263
23
? get_nr_inodes
vmlinux 0xffffffff82e5cd7c
Build id: 3.10.0-957.el7.x86_64 		0x4c
24
bdi_writeback_workfn
vmlinux 0xffffffff82e718eb
Build id: 3.10.0-957.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffff82cb9d4f
Build id: 3.10.0-957.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffff82cbade6
Build id: 3.10.0-957.el7.x86_64 		0x126
27
? manage_workers.isra.25
vmlinux 0xffffffff82cbacc0
Build id: 3.10.0-957.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffff82cc1c31
Build id: 3.10.0-957.el7.x86_64 		0xd1
29
? insert_kthread_work
vmlinux 0xffffffff82cc1b60
Build id: 3.10.0-957.el7.x86_64 		0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffff83374c37
Build id: 3.10.0-957.el7.x86_64 		0x21
31
? insert_kthread_work
vmlinux 0xffffffff82cc1b60
Build id: 3.10.0-957.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [kworker/u32:2:23967]
Modules linked in: kbase(OE) udp_diag tcp_diag inet_diag binfmt_misc fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd parport_pc parport virtio_balloon joydev i2c_piix4 pcspkr ip_tables ext4 mbcache jbd2 ata_generic pata_acpi
virtio_blk virtio_console virtio_net cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix floppy libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 23967 Comm: kworker/u32:2 Kdump: loaded Tainted: G           OEL ------------   3.10.0-957.el7.x86_64 #1
Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9c0363b41040 ti: ffff9bfeb4ef8000 task.ti: ffff9bfeb4ef8000
RIP: 0010:[<ffffffff82d1173e>]  [<ffffffff82d1173e>] smp_call_function_many+0x20e/0x270
RSP: 0018:ffff9bfeb4efb748  EFLAGS: 00000202
RAX: 000000000000000e RBX: 00000000000000fc RCX: ffff9c0e3f5a0398
RDX: 000000000000000e RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9bfeb4efb780 R08: ffff9bffa94f4400 R09: ffffffff82f755c9
R10: ffff9c0e3f39f160 R11: fffff81ed48ace00 R12: 0000000000000296
R13: ffff9bfeb4efb6f8 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff9c0e3f380000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5138b2b000 CR3: 00000005a06f8000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff82c7a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff82c7a734>] flush_tlb_page+0x54/0xa0
[<ffffffff82dfd587>] ptep_clear_flush+0x67/0xa0
[<ffffffff82df6b66>] page_mkclean_one+0xa6/0xe0
[<ffffffff82df8983>] rmap_walk+0xf3/0x360
[<ffffffff82df8d96>] page_mkclean+0x76/0xb0
[<ffffffff82df6ac0>] ? page_referenced_one+0x180/0x180
[<ffffffff82df6200>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffff82dc1d60>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc03c3d54>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc03c3eb0>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc03c5255>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc03c991c>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc03f8c59>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc03c9947>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffff82cdca2e>] ? account_entity_dequeue+0xae/0xd0
[<ffffffff82dc3181>] do_writepages+0x21/0x50
[<ffffffff82e6f930>] __writeback_single_inode+0x40/0x260
[<ffffffff82e703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff82e7072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff82e70f63>] wb_writeback+0x263/0x2f0
[<ffffffff82e5cd7c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff82e718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff82cb9d4f>] process_one_work+0x17f/0x440
[<ffffffff82cbade6>] worker_thread+0x126/0x3c0
[<ffffffff82cbacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff82cc1c31>] kthread+0xd1/0xe0
[<ffffffff82cc1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffff83374c37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff82cc1b60>] ? insert_kthread_work+0x40/0x40
Code: 83 c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 60 b9 94 83 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 1d 83 c4 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee
Complete report #8190
Frame # Function Binary Source or offset Line
1
smp_call_function_many
vmlinux 0xffffffffa3d1761a
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x20a
2
native_flush_tlb_others
vmlinux 0xffffffffa3c7e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xb8
3
flush_tlb_page
vmlinux 0xffffffffa3c7ea94
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x54
4
ptep_clear_flush
vmlinux 0xffffffffa3e07888
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x68
5
page_mkclean_one
vmlinux 0xffffffffa3e00e36
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xa6
6
rmap_walk
vmlinux 0xffffffffa3e02c43
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xf3
7
page_mkclean
vmlinux 0xffffffffa3e03056
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x76
8
? page_referenced_one
vmlinux 0xffffffffa3e00d90
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x1b0
9
? percpu_flush_tlb_batch_pages
vmlinux 0xffffffffa3e00470
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x20
10
clear_page_dirty_for_io
vmlinux 0xffffffffa3dc9f70
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x50
11
mpage_submit_page
ext4 0xffffffffc054f1f4
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x24
12
mpage_process_page_bufs
ext4 0xffffffffc054f350
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x110
13
mpage_prepare_extent_to_map
ext4 0xffffffffc0550675
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x235
14
? jbd2__journal_start
jbd2 0xffffffffc052ca93
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xf3
15
? ext4_writepages
ext4 0xffffffffc0554dec
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x43c
16
? __ext4_journal_start_sb
ext4 0xffffffffc0584679
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x69
17
ext4_writepages
ext4 0xffffffffc0554e17
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x467
18
do_writepages
vmlinux 0xffffffffa3dcb261
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
19
__writeback_single_inode
vmlinux 0xffffffffa3e7cc40
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
20
writeback_sb_inodes
vmlinux 0xffffffffa3e7d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x1c4
21
__writeback_inodes_wb
vmlinux 0xffffffffa3e7dacf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x9f
22
wb_writeback
vmlinux 0xffffffffa3e7dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x263
23
? get_nr_inodes
vmlinux 0xffffffffa3e69c5c
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x4c
24
bdi_writeback_workfn
vmlinux 0xffffffffa3e7eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2cb
25
process_one_work
vmlinux 0xffffffffa3cbe6bf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x17f
26
worker_thread
vmlinux 0xffffffffa3cbf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x126
27
? manage_workers.isra.26
vmlinux 0xffffffffa3cbf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2a0
28
kthread
vmlinux 0xffffffffa3cc6691
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xd1
29
? insert_kthread_work
vmlinux 0xffffffffa3cc65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
30
ret_from_fork_nospec_begin
vmlinux 0xffffffffa4392d37
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
31
? insert_kthread_work
vmlinux 0xffffffffa3cc65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [kworker/u36:0:1164]
Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon sg joydev pcspkr parport_pc parport vmw_vmci i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix serio_raw e1000 libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks floppy
CPU: 6 PID: 1164 Comm: kworker/u36:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff9c3a76ad5230 ti: ffff9c3c76850000 task.ti: ffff9c3c76850000
RIP: 0010:[<ffffffffa3d1761a>]  [<ffffffffa3d1761a>] smp_call_function_many+0x20a/0x270
RSP: 0000:ffff9c3c76853748  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 000000000001b840 RCX: ffff9c3477c209a0
RDX: 0000000000000000 RSI: 0000000000000010 RDI: 0000000000000000
RBP: ffff9c3c76853780 R08: ffff9c38ffc08c00 R09: ffffffffa3f851b9
R10: ffff9c3a77c1f160 R11: ffffdb75d2f45c00 R12: ffff9c3c768536f8
R13: ffff9c3477c37810 R14: 0000000000000002 R15: ffff9c4077904a28
FS:  0000000000000000(0000) GS:ffff9c3a77c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f29d9dc326f CR3: 0000000ae2c10000 CR4: 00000000000407e0
Call Trace:
[<ffffffffa3c7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa3c7ea94>] flush_tlb_page+0x54/0xa0
[<ffffffffa3e07888>] ptep_clear_flush+0x68/0xa0
[<ffffffffa3e00e36>] page_mkclean_one+0xa6/0xe0
[<ffffffffa3e02c43>] rmap_walk+0xf3/0x360
[<ffffffffa3e03056>] page_mkclean+0x76/0xb0
[<ffffffffa3e00d90>] ? page_referenced_one+0x1b0/0x1b0
[<ffffffffa3e00470>] ? percpu_flush_tlb_batch_pages+0x20/0x20
[<ffffffffa3dc9f70>] clear_page_dirty_for_io+0x50/0xd0
[<ffffffffc054f1f4>] mpage_submit_page+0x24/0x70 [ext4]
[<ffffffffc054f350>] mpage_process_page_bufs+0x110/0x130 [ext4]
[<ffffffffc0550675>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
[<ffffffffc052ca93>] ? jbd2__journal_start+0xf3/0x1f0 [jbd2]
[<ffffffffc0554dec>] ? ext4_writepages+0x43c/0xcf0 [ext4]
[<ffffffffc0584679>] ? __ext4_journal_start_sb+0x69/0xe0 [ext4]
[<ffffffffc0554e17>] ext4_writepages+0x467/0xcf0 [ext4]
[<ffffffffa3dcb261>] do_writepages+0x21/0x50
[<ffffffffa3e7cc40>] __writeback_single_inode+0x40/0x260
[<ffffffffa3e7d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa3e7dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa3e7dfa3>] wb_writeback+0x263/0x2f0
[<ffffffffa3e69c5c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa3e7eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa3cbe6bf>] process_one_work+0x17f/0x440
[<ffffffffa3cbf7d6>] worker_thread+0x126/0x3c0
[<ffffffffa3cbf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa3cc6691>] kthread+0xd1/0xe0
[<ffffffffa3cc65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa4392d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa3cc65c0>] ? insert_kthread_work+0x40/0x40
Code: 48 63 35 ce 7b c4 00 89 c2 39 f0 0f 8d 7d fe ff ff 48 98 49 8b 0f 48 03 0c c5 a0 10 95 a4 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 <f6> 41 20 01 75 f8 48 63 35 9d 7b c4 00 eb b7 0f b6 4d cc 4c 89