Info

Permalink
Function
ktime_get_ts64
First occurrence
2020-10-18
Last occurrence
2026-09-11
State
NEW
Components
kernel
Quality
-27

Statistics

Operating system Count
CentOS 7 23
Architecture Count
x86_64 23
Related packages Count
kernel 23
 0:3.10.0-1127.13.1.el7 12
 0:3.10.0-1160.el7 6
 0:3.10.0-1127.el7 1
 0:3.10.0-1160.71.1.el7 1
 0:3.10.0-957.5.1.el7 1
 0:3.10.0-1160.81.1.el7 1
 0:3.10.0-1160.119.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

No reports in the last 20 days.

Report backtrace


Complete report #1422100
Frame # Function Binary Source or offset Line
1
? bit_wait
vmlinux 0xffffffffa8db5ec0
Build id: 3.10.0-1160.119.1.el7.x86_64
0x60
2
io_schedule_timeout
vmlinux 0xffffffffa8db754d
Build id: 3.10.0-1160.119.1.el7.x86_64
0xad
3
io_schedule
vmlinux 0xffffffffa8db75f8
Build id: 3.10.0-1160.119.1.el7.x86_64
0x18
4
bit_wait_io
vmlinux 0xffffffffa8db5ed1
Build id: 3.10.0-1160.119.1.el7.x86_64
0x11
5
__wait_on_bit
vmlinux 0xffffffffa8db59d7
Build id: 3.10.0-1160.119.1.el7.x86_64
0x67
6
wait_on_page_bit
vmlinux 0xffffffffa87c8071
Build id: 3.10.0-1160.119.1.el7.x86_64
0x81
7
? wake_bit_function
vmlinux 0xffffffffa86cc860
Build id: 3.10.0-1160.119.1.el7.x86_64
0x40
8
__filemap_fdatawait_range
vmlinux 0xffffffffa87c81a1
Build id: 3.10.0-1160.119.1.el7.x86_64
0x111
9
? do_writepages
vmlinux 0xffffffffa87d6b01
Build id: 3.10.0-1160.119.1.el7.x86_64
0x21
10
filemap_fdatawait_range
vmlinux 0xffffffffa87c8234
Build id: 3.10.0-1160.119.1.el7.x86_64
0x14
11
filemap_write_and_wait_range
vmlinux 0xffffffffa87cad26
Build id: 3.10.0-1160.119.1.el7.x86_64
0x56
12
xfs_file_fsync
xfs 0xffffffffc04f0486
Build id: 3.10.0-1160.119.1.el7.x86_64
0x66
13
do_fsync
vmlinux 0xffffffffa8893195
Build id: 3.10.0-1160.119.1.el7.x86_64
0x55
14
? __audit_syscall_exit
vmlinux 0xffffffffa8746986
Build id: 3.10.0-1160.119.1.el7.x86_64
0x1f6
15
SyS_fsync
vmlinux 0xffffffffa8893460
Build id: 3.10.0-1160.119.1.el7.x86_64
0x10
16
system_call_fastpath
vmlinux 0xffffffffa8dc539a
Build id: 3.10.0-1160.119.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 1366s! [timesync:1429]
[<ffffffffa8db5ec0>] ? bit_wait+0x60/0x60
[<ffffffffa8db754d>] io_schedule_timeout+0xad/0x140
[<ffffffffa8db75f8>] io_schedule+0x18/0x20
[<ffffffffa8db5ed1>] bit_wait_io+0x11/0x60
[<ffffffffa8db59d7>] __wait_on_bit+0x67/0xa0
[<ffffffffa87c8071>] wait_on_page_bit+0x81/0xa0
[<ffffffffa86cc860>] ? wake_bit_function+0x40/0x40
[<ffffffffa87c81a1>] __filemap_fdatawait_range+0x111/0x190
[<ffffffffa87d6b01>] ? do_writepages+0x21/0x50
[<ffffffffa87c8234>] filemap_fdatawait_range+0x14/0x30
[<ffffffffa87cad26>] filemap_write_and_wait_range+0x56/0xa0
[<ffffffffc04f0486>] xfs_file_fsync+0x66/0x1c0 [xfs]
[<ffffffffa8893195>] do_fsync+0x55/0x80
[<ffffffffa8746986>] ? __audit_syscall_exit+0x1f6/0x2b0
[<ffffffffa8893460>] SyS_fsync+0x10/0x20
[<ffffffffa8dc539a>] system_call_fastpath+0x25/0x2a
Complete report #1165881
Frame # Function Binary Source or offset Line
1
? bit_wait
vmlinux 0xffffffffad9b5ec0
Build id: 3.10.0-1160.81.1.el7.x86_64
0x60
2
io_schedule_timeout
vmlinux 0xffffffffad9b754d
Build id: 3.10.0-1160.81.1.el7.x86_64
0xad
3
io_schedule
vmlinux 0xffffffffad9b75f8
Build id: 3.10.0-1160.81.1.el7.x86_64
0x18
4
bit_wait_io
vmlinux 0xffffffffad9b5ed1
Build id: 3.10.0-1160.81.1.el7.x86_64
0x11
5
__wait_on_bit
vmlinux 0xffffffffad9b59d7
Build id: 3.10.0-1160.81.1.el7.x86_64
0x67
6
wait_on_page_bit
vmlinux 0xffffffffad3c7e51
Build id: 3.10.0-1160.81.1.el7.x86_64
0x81
7
? wake_bit_function
vmlinux 0xffffffffad2cc710
Build id: 3.10.0-1160.81.1.el7.x86_64
0x40
8
__filemap_fdatawait_range
vmlinux 0xffffffffad3c7f81
Build id: 3.10.0-1160.81.1.el7.x86_64
0x111
9
? do_writepages
vmlinux 0xffffffffad3d68e1
Build id: 3.10.0-1160.81.1.el7.x86_64
0x21
10
filemap_fdatawait_range
vmlinux 0xffffffffad3c8014
Build id: 3.10.0-1160.81.1.el7.x86_64
0x14
11
filemap_write_and_wait_range
vmlinux 0xffffffffad3cab06
Build id: 3.10.0-1160.81.1.el7.x86_64
0x56
12
xfs_file_fsync
xfs 0xffffffffc0613486
Build id: 3.10.0-1160.81.1.el7.x86_64
0x66
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [rtkit-daemon:816]
[<ffffffffad9b5ec0>] ? bit_wait+0x60/0x60
[<ffffffffad9b754d>] io_schedule_timeout+0xad/0x140
[<ffffffffad9b75f8>] io_schedule+0x18/0x20
[<ffffffffad9b5ed1>] bit_wait_io+0x11/0x60
[<ffffffffad9b59d7>] __wait_on_bit+0x67/0xa0
[<ffffffffad3c7e51>] wait_on_page_bit+0x81/0xa0
[<ffffffffad2cc710>] ? wake_bit_function+0x40/0x40
[<ffffffffad3c7f81>] __filemap_fdatawait_range+0x111/0x190
[<ffffffffad3d68e1>] ? do_writepages+0x21/0x50
[<ffffffffad3c8014>] filemap_fdatawait_range+0x14/0x30
[<ffffffffad3cab06>] filemap_write_and_wait_range+0x56/0xa0
[<ffffffffc0613486>] xfs_file_fsync+0x66/0x1c0 [xfs]
Complete report #1138985
Frame # Function Binary Source or offset Line
1
ktime_get_ts64
vmlinux 0xffffffff833010e0
Build id: 3.10.0-957.5.1.el7.x86_64
0xa0
2
__delayacct_blkio_start
vmlinux 0xffffffff83358312
Build id: 3.10.0-957.5.1.el7.x86_64
0x22
3
io_schedule_timeout
vmlinux 0xffffffff839672cb
Build id: 3.10.0-957.5.1.el7.x86_64
0x8b
4
io_schedule
vmlinux 0xffffffff83967388
Build id: 3.10.0-957.5.1.el7.x86_64
0x18
5
bit_wait_io
vmlinux 0xffffffff83965d71
Build id: 3.10.0-957.5.1.el7.x86_64
0x11
6
__wait_on_bit
vmlinux 0xffffffff83965897
Build id: 3.10.0-957.5.1.el7.x86_64
0x67
7
wait_on_page_bit
vmlinux 0xffffffff833b57a1
Build id: 3.10.0-957.5.1.el7.x86_64
0x81
8
? wake_bit_function
vmlinux 0xffffffff832c2e00
Build id: 3.10.0-957.5.1.el7.x86_64
0x40
9
__filemap_fdatawait_range
vmlinux 0xffffffff833b58d1
Build id: 3.10.0-957.5.1.el7.x86_64
0x111
10
? do_writepages
vmlinux 0xffffffff833c3431
Build id: 3.10.0-957.5.1.el7.x86_64
0x21
11
filemap_fdatawait_range
vmlinux 0xffffffff833b5964
Build id: 3.10.0-957.5.1.el7.x86_64
0x14
12
filemap_write_and_wait_range
vmlinux 0xffffffff833b7dc6
Build id: 3.10.0-957.5.1.el7.x86_64
0x56
13
xfs_file_fsync
xfs 0xffffffffc0369156
Build id: 3.10.0-957.5.1.el7.x86_64
0x66
14
do_fsync
vmlinux 0xffffffff83476227
Build id: 3.10.0-957.5.1.el7.x86_64
0x67
15
? system_call_after_swapgs
vmlinux 0xffffffff83974d15
Build id: 3.10.0-957.5.1.el7.x86_64
0xa2
16
SyS_fsync
vmlinux 0xffffffff83476510
Build id: 3.10.0-957.5.1.el7.x86_64
0x10
17
system_call_fastpath
vmlinux 0xffffffff83974ddb
Build id: 3.10.0-957.5.1.el7.x86_64
0x22
18
? system_call_after_swapgs
vmlinux 0xffffffff83974d21
Build id: 3.10.0-957.5.1.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 6157s! [postdrop:5799]
Modules linked in: devlink nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support edac_mce_amd joydev kvm_amd kvm irqbypass pcspkr sg snd_hda_codec_generic virtio_gpu snd_hda_intel ttm snd_hda_codec i2c_i801 drm_kms_helper snd_hda_core lpc_ich snd_hwdep snd_seq syscopyarea virtio_rng sysfillrect sysimgblt fb_sys_fops snd_seq_device drm snd_pcm
snd_timer snd drm_panel_orientation_quirks soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common virtio_console ahci libahci libata e1000 virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 5799 Comm: postdrop Kdump: loaded Tainted: G             L ------------   3.10.0-957.5.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org 04/01/2014
task: ffff9f4878378000 ti: ffff9f4738fdc000 task.ti: ffff9f4738fdc000
RIP: 0010:[<ffffffff833010e0>]  [<ffffffff833010e0>] ktime_get_ts64+0xa0/0xf0
RSP: 0018:ffff9f4738fdfc58  EFLAGS: 00000296
RAX: abb3fc0245d02b32 RBX: ffff9f4735fb8000 RCX: 000000000008c6ad
RDX: 000000006a0d7286 RSI: 000000002d41fc2a RDI: ffffffff83e2ea80
RBP: ffff9f4738fdfc78 R08: ffff9f487ff962c0 R09: be2c11f1b8000000
R10: 41d3dd994c7c6e00 R11: ffff9f487331f700 R12: ffff9f4735fb8000
R13: 0000000000000000 R14: ffff9f487331f700 R15: ffffffffc02d8a40
FS:  00007f04a29a5880(0000) GS:ffff9f487fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f04a052a0f0 CR3: 000000005b2b4000 CR4: 00000000000006f0
Call Trace:
[<ffffffff83358312>] __delayacct_blkio_start+0x22/0x30
[<ffffffff839672cb>] io_schedule_timeout+0x8b/0x130
[<ffffffff83967388>] io_schedule+0x18/0x20
[<ffffffff83965d71>] bit_wait_io+0x11/0x50
[<ffffffff83965897>] __wait_on_bit+0x67/0x90
[<ffffffff833b57a1>] wait_on_page_bit+0x81/0xa0
[<ffffffff832c2e00>] ? wake_bit_function+0x40/0x40
[<ffffffff833b58d1>] __filemap_fdatawait_range+0x111/0x190
[<ffffffff833c3431>] ? do_writepages+0x21/0x50
[<ffffffff833b5964>] filemap_fdatawait_range+0x14/0x30
[<ffffffff833b7dc6>] filemap_write_and_wait_range+0x56/0x90
[<ffffffffc0369156>] xfs_file_fsync+0x66/0x1c0 [xfs]
[<ffffffff83476227>] do_fsync+0x67/0xb0
[<ffffffff83974d15>] ? system_call_after_swapgs+0xa2/0x146
[<ffffffff83476510>] SyS_fsync+0x10/0x20
[<ffffffff83974ddb>] system_call_fastpath+0x22/0x27
[<ffffffff83974d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 03 05 fc 80 ff 00 48 d3 f8 48 8b 0d 32 81 ff 00 44 3b 35 cb 80 ff 00 75 95 48 01 f0 49 03 0c 24 48 3d ff c9 9a 3b 76 28 31 d2 <48> 2d 00 ca 9a 3b 83 c2 01 48 3d ff c9 9a 3b 77 ef 49 89 44 24 
Complete report #592705
Frame # Function Binary Source or offset Line
1
read_tsc
vmlinux 0xffffffff96435479
Build id: 3.10.0-1160.71.1.el7.x86_64
0x9
2
ktime_get_ts64
vmlinux 0xffffffff96506ac2
Build id: 3.10.0-1160.71.1.el7.x86_64
0x52
3
__delayacct_blkio_start
vmlinux 0xffffffff9655e262
Build id: 3.10.0-1160.71.1.el7.x86_64
0x22
4
io_schedule_timeout
vmlinux 0xffffffff96b8bc8b
Build id: 3.10.0-1160.71.1.el7.x86_64
0x8b
5
io_schedule
vmlinux 0xffffffff96b8bd48
Build id: 3.10.0-1160.71.1.el7.x86_64
0x18
6
bit_wait_io
vmlinux 0xffffffff96b8a711
Build id: 3.10.0-1160.71.1.el7.x86_64
0x11
7
__wait_on_bit
vmlinux 0xffffffff96b8a237
Build id: 3.10.0-1160.71.1.el7.x86_64
0x67
8
wait_on_page_bit
vmlinux 0xffffffff965bd431
Build id: 3.10.0-1160.71.1.el7.x86_64
0x81
9
? wake_bit_function
vmlinux 0xffffffff964c7140
Build id: 3.10.0-1160.71.1.el7.x86_64
0x40
10
__filemap_fdatawait_range
vmlinux 0xffffffff965bd561
Build id: 3.10.0-1160.71.1.el7.x86_64
0x111
11
? do_writepages
vmlinux 0xffffffff965cba11
Build id: 3.10.0-1160.71.1.el7.x86_64
0x21
12
filemap_fdatawait_range
vmlinux 0xffffffff965bd5f4
Build id: 3.10.0-1160.71.1.el7.x86_64
0x14
13
filemap_write_and_wait_range
vmlinux 0xffffffff965bfff6
Build id: 3.10.0-1160.71.1.el7.x86_64
0x56
14
xfs_file_fsync
xfs 0xffffffffc0345d66
Build id: 3.10.0-1160.71.1.el7.x86_64
0x66
15
do_fsync
vmlinux 0xffffffff96684255
Build id: 3.10.0-1160.71.1.el7.x86_64
0x55
16
? __audit_syscall_exit
vmlinux 0xffffffff9653ecd6
Build id: 3.10.0-1160.71.1.el7.x86_64
0x1f6
17
SyS_fdatasync
vmlinux 0xffffffff96684523
Build id: 3.10.0-1160.71.1.el7.x86_64
0x13
18
system_call_fastpath
vmlinux 0xffffffff96b99f92
Build id: 3.10.0-1160.71.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [postgres:2873]
Modules linked in: rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 bridge stp llc ebtable_filter ebtables devlink tun ipmi_devintf ipmi_msghandler ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 vmw_vsock_vmci_transport xt_conntrack vsock bnep ip6table_filter ip6_tables iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_ens1371 snd_rawmidi snd_ac97_codec lrw gf128mul glue_helper ablk_helper ac97_bus snd_seq snd_seq_device snd_pcm btusb btrtl btbcm btintel ppdev snd_timer cryptd snd bluetooth rfkill soundcore vmw_balloon joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 sch_fq_codel ip_tables
xfs libcrc32c sd_mod crc_t10dif sr_mod cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel libata nfit mptspi scsi_transport_spi serio_raw libnvdimm mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 1 PID: 2873 Comm: postgres Kdump: loaded Tainted: G             L ------------   3.10.0-1160.71.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff99b6a4103180 ti: ffff99b79b504000 task.ti: ffff99b79b504000
RIP: 0010:[<ffffffff96435479>]  [<ffffffff96435479>] read_tsc+0x9/0x20
RSP: 0018:ffff99b79b507c58  EFLAGS: 00000246
RAX: 00000000bf2bf111 RBX: ffff99b635148000 RCX: 000000000000000d
RDX: 0000000000052aac RSI: 0000000000000002 RDI: ffffffff9702f200
RBP: ffff99b79b507c58 R08: ffff99b7bffa38a8 R09: 963d13ba1a000000
R10: 69c2d1d52aee8680 R11: ffff99b7b5416bc0 R12: ffff99b79b507be8
R13: ffff99b7b5416bc0 R14: ffff99b635dd2048 R15: ffff99b635f41000
FS:  00007efe88f93880(0000) GS:ffff99b7b7640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000562e8efff060 CR3: 000000019b508000 CR4: 00000000003607e0
Call Trace:
[<ffffffff96506ac2>] ktime_get_ts64+0x52/0xf0
[<ffffffff9655e262>] __delayacct_blkio_start+0x22/0x30
[<ffffffff96b8bc8b>] io_schedule_timeout+0x8b/0x130
[<ffffffff96b8bd48>] io_schedule+0x18/0x20
[<ffffffff96b8a711>] bit_wait_io+0x11/0x50
[<ffffffff96b8a237>] __wait_on_bit+0x67/0x90
[<ffffffff965bd431>] wait_on_page_bit+0x81/0xa0
[<ffffffff964c7140>] ? wake_bit_function+0x40/0x40
[<ffffffff965bd561>] __filemap_fdatawait_range+0x111/0x190
[<ffffffff965cba11>] ? do_writepages+0x21/0x50
[<ffffffff965bd5f4>] filemap_fdatawait_range+0x14/0x30
[<ffffffff965bfff6>] filemap_write_and_wait_range+0x56/0x90
[<ffffffffc0345d66>] xfs_file_fsync+0x66/0x1c0 [xfs]
[<ffffffff96684255>] do_fsync+0x55/0x80
[<ffffffff9653ecd6>] ? __audit_syscall_exit+0x1f6/0x2b0
[<ffffffff96684523>] SyS_fdatasync+0x13/0x20
[<ffffffff96b99f92>] system_call_fastpath+0x25/0x2a
Code: 40 00 55 8b 05 c9 c0 d1 00 48 89 e5 5d c3 0f 1f 40 00 55 b8 ed ff ff ff 48 89 e5 5d c3 0f 1f 44 00 00 55 48 89 e5 0f ae e8 0f 31 <89> c0 48 c1 e2 20 48 09 c2 48 89 d0 5d c3 66 0f 1f 84 00 00 00 
Complete report #371252
Frame # Function Binary Source or offset Line
1
ktime_get_ts64
vmlinux 0xffffffffa3706d30
Build id: 3.10.0-1127.13.1.el7.x86_64
0xa0
2
__delayacct_blkio_start
vmlinux 0xffffffffa375e522
Build id: 3.10.0-1127.13.1.el7.x86_64
0x22
3
io_schedule_timeout
vmlinux 0xffffffffa3d8547b
Build id: 3.10.0-1127.13.1.el7.x86_64
0x8b
4
io_schedule
vmlinux 0xffffffffa3d85538
Build id: 3.10.0-1127.13.1.el7.x86_64
0x18
5
bit_wait_io
vmlinux 0xffffffffa3d83f01
Build id: 3.10.0-1127.13.1.el7.x86_64
0x11
6
__wait_on_bit
vmlinux 0xffffffffa3d83a27
Build id: 3.10.0-1127.13.1.el7.x86_64
0x67
7
wait_on_page_bit
vmlinux 0xffffffffa37bd6f1
Build id: 3.10.0-1127.13.1.el7.x86_64
0x81
8
? wake_bit_function
vmlinux 0xffffffffa36c7840
Build id: 3.10.0-1127.13.1.el7.x86_64
0x40
9
__filemap_fdatawait_range
vmlinux 0xffffffffa37bd821
Build id: 3.10.0-1127.13.1.el7.x86_64
0x111
10
? do_writepages
vmlinux 0xffffffffa37cb211
Build id: 3.10.0-1127.13.1.el7.x86_64
0x21
11
filemap_fdatawait_range
vmlinux 0xffffffffa37bd8b4
Build id: 3.10.0-1127.13.1.el7.x86_64
0x14
12
filemap_write_and_wait_range
vmlinux 0xffffffffa37bfd96
Build id: 3.10.0-1127.13.1.el7.x86_64
0x56
13
xfs_file_fsync
xfs 0xffffffffc05bcdd6
Build id: 3.10.0-1127.13.1.el7.x86_64
0x66
14
do_fsync
vmlinux 0xffffffffa3882f67
Build id: 3.10.0-1127.13.1.el7.x86_64
0x67
15
SyS_fsync
vmlinux 0xffffffffa3883250
Build id: 3.10.0-1127.13.1.el7.x86_64
0x10
16
system_call_fastpath
vmlinux 0xffffffffa3d92ed2
Build id: 3.10.0-1127.13.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [zsd:1575]
Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event coretemp snd_ens1371 iosf_mbi snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel ppdev ac97_bus snd_seq aesni_intel snd_seq_device lrw snd_pcm gf128mul glue_helper
ablk_helper cryptd snd_timer vmw_balloon snd soundcore pcspkr sg joydev i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect ata_piix sysimgblt fb_sys_fops crc32c_intel ttm libata mptspi drm scsi_transport_spi mptscsih serio_raw nfit libnvdimm mptbase e1000 drm_panel_orientation_quirks
CPU: 2 PID: 1575 Comm: zsd Kdump: loaded Not tainted 3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff8bb66615b150 ti: ffff8bb65beb4000 task.ti: ffff8bb65beb4000
RIP: 0010:[<ffffffffa3706d30>]  [<ffffffffa3706d30>] ktime_get_ts64+0xa0/0xf0
RSP: 0018:ffff8bb65beb7c58  EFLAGS: 00000216
RAX: 34ed196937f10c87 RBX: ffff8bb6f1118000 RCX: 000000000001986c
RDX: 0000000068320801 RSI: 0000000031ea0437 RDI: ffffffffa422f200
RBP: ffff8bb65beb7c78 R08: ffff8bb6fffbab08 R09: abf704ec01000000
R10: 5408e611c03b0040 R11: ffff8bb67b89dc00 R12: ffff8bb6f1118000
R13: ffffffffc04422e0 R14: ffff8bb65beb7bd8 R15: ffff8bb67b89dc00
FS:  00007f9956a8a700(0000) GS:ffff8bb6f9680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9966c73000 CR3: 00000000a62f4000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa375e522>] __delayacct_blkio_start+0x22/0x30
[<ffffffffa3d8547b>] io_schedule_timeout+0x8b/0x130
[<ffffffffa3d85538>] io_schedule+0x18/0x20
[<ffffffffa3d83f01>] bit_wait_io+0x11/0x50
[<ffffffffa3d83a27>] __wait_on_bit+0x67/0x90
[<ffffffffa37bd6f1>] wait_on_page_bit+0x81/0xa0
[<ffffffffa36c7840>] ? wake_bit_function+0x40/0x40
[<ffffffffa37bd821>] __filemap_fdatawait_range+0x111/0x190
[<ffffffffa37cb211>] ? do_writepages+0x21/0x50
[<ffffffffa37bd8b4>] filemap_fdatawait_range+0x14/0x30
[<ffffffffa37bfd96>] filemap_write_and_wait_range+0x56/0x90
[<ffffffffc05bcdd6>] xfs_file_fsync+0x66/0x1c0 [xfs]
[<ffffffffa3882f67>] do_fsync+0x67/0xb0
[<ffffffffa3883250>] SyS_fsync+0x10/0x20
[<ffffffffa3d92ed2>] system_call_fastpath+0x25/0x2a
Code: 48 03 05 ec 26 01 01 48 d3 f8 48 8b 0d 22 27 01 01 44 3b 35 bb 26 01 01 75 95 48 01 f0 49 03 0c 24 48 3d ff c9 9a 3b 76 28 31 d2 <48> 2d 00 ca 9a 3b 83 c2 01 48 3d ff c9 9a 3b 77 ef 49 89 44 24 
Complete report #251048
Frame # Function Binary Source or offset Line
1
ktime_get_ts64
vmlinux 0xffffffff843063b0
Build id: 3.10.0-1160.el7.x86_64
0xa0
2
__delayacct_blkio_start
vmlinux 0xffffffff8435dc62
Build id: 3.10.0-1160.el7.x86_64
0x22
3
io_schedule_timeout
vmlinux 0xffffffff8498681b
Build id: 3.10.0-1160.el7.x86_64
0x8b
4
io_schedule
vmlinux 0xffffffff849868d8
Build id: 3.10.0-1160.el7.x86_64
0x18
5
bit_wait_io
vmlinux 0xffffffff849852a1
Build id: 3.10.0-1160.el7.x86_64
0x11
6
__wait_on_bit
vmlinux 0xffffffff84984dc7
Build id: 3.10.0-1160.el7.x86_64
0x67
7
wait_on_page_bit
vmlinux 0xffffffff843bcf01
Build id: 3.10.0-1160.el7.x86_64
0x81
8
? wake_bit_function
vmlinux 0xffffffff842c6dd0
Build id: 3.10.0-1160.el7.x86_64
0x40
9
__filemap_fdatawait_range
vmlinux 0xffffffff843bd031
Build id: 3.10.0-1160.el7.x86_64
0x111
10
? do_writepages
vmlinux 0xffffffff843cade1
Build id: 3.10.0-1160.el7.x86_64
0x21
11
filemap_fdatawait_range
vmlinux 0xffffffff843bd0c4
Build id: 3.10.0-1160.el7.x86_64
0x14
12
filemap_write_and_wait_range
vmlinux 0xffffffff843bfab6
Build id: 3.10.0-1160.el7.x86_64
0x56
13
ext4_sync_file
ext4 0xffffffffc02669fa
Build id: 3.10.0-1160.el7.x86_64
0xba
14
do_fsync
vmlinux 0xffffffff84483a07
Build id: 3.10.0-1160.el7.x86_64
0x67
15
SyS_fsync
vmlinux 0xffffffff84483cf0
Build id: 3.10.0-1160.el7.x86_64
0x10
16
system_call_fastpath
vmlinux 0xffffffff84993f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [alertmanager:23946]
Modules linked in: xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter usblp overlay(T) prl_fs_freeze(POE) prl_fs(POE) prl_eth(POE) sunrpc snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq gpio_ich coretemp snd_seq_device ppdev snd_pcm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_timer lrw snd gf128mul glue_helper ablk_helper cryptd soundcore sg pcspkr i2c_i801 joydev lpc_ich sbs sbshc virtio_balloon parport_pc parport ip_tables ext4 mbcache jbd2
sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover failover ahci crct10dif_pclmul crct10dif_common libahci ata_piix libata crc32c_intel prl_vid(POE) virtio_pci serio_raw virtio_ring virtio prl_tg(POE) dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 23946 Comm: alertmanager Kdump: loaded Tainted: P           OEL ------------ T 3.10.0-1160.el7.x86_64 #1
Hardware name: Parallels Software International Inc. Parallels Virtual Platform/Parallels Virtual Platform, BIOS 16.1.2 (49151) 12/18/2020
task: ffff8ca0cfa9c200 ti: ffff8ca10e0d8000 task.ti: ffff8ca10e0d8000
RIP: 0010:[<ffffffff843063b0>]  [<ffffffff843063b0>] ktime_get_ts64+0xa0/0xf0
RSP: 0018:ffff8ca10e0dbc68  EFLAGS: 00000212
RAX: 6018e9b361684ac1 RBX: ffff8ca0b6122848 RCX: 00000000000ec995
RDX: 00000000aec6db80 RSI: 000000003209d020 RDI: ffff8ca191785040
RBP: ffff8ca10e0dbc88 R08: 00000000a55fead5 R09: 6ec400f3d1000000
R10: 913bdabd2f3cf440 R11: ffff8ca12aa04e00 R12: 0000000000000000
R13: 0000000000000000 R14: ffffffff846ecc48 R15: ffff8ca10e0dbc20
FS:  000000c000475490(0000) GS:ffff8ca18d440000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006e6bd4 CR3: 00000000a77be000 CR4: 00000000003606e0
Call Trace:
[<ffffffff8435dc62>] __delayacct_blkio_start+0x22/0x30
[<ffffffff8498681b>] io_schedule_timeout+0x8b/0x130
[<ffffffff849868d8>] io_schedule+0x18/0x20
[<ffffffff849852a1>] bit_wait_io+0x11/0x50
[<ffffffff84984dc7>] __wait_on_bit+0x67/0x90
[<ffffffff843bcf01>] wait_on_page_bit+0x81/0xa0
[<ffffffff842c6dd0>] ? wake_bit_function+0x40/0x40
[<ffffffff843bd031>] __filemap_fdatawait_range+0x111/0x190
[<ffffffff843cade1>] ? do_writepages+0x21/0x50
[<ffffffff843bd0c4>] filemap_fdatawait_range+0x14/0x30
[<ffffffff843bfab6>] filemap_write_and_wait_range+0x56/0x90
[<ffffffffc02669fa>] ext4_sync_file+0xba/0x320 [ext4]
[<ffffffff84483a07>] do_fsync+0x67/0xb0
[<ffffffff84483cf0>] SyS_fsync+0x10/0x20
[<ffffffff84993f92>] system_call_fastpath+0x25/0x2a
Code: 48 03 05 6c 40 01 01 48 d3 f8 48 8b 0d a2 40 01 01 44 3b 35 3b 40 01 01 75 95 48 01 f0 49 03 0c 24 48 3d ff c9 9a 3b 76 28 31 d2 <48> 2d 00 ca 9a 3b 83 c2 01 48 3d ff c9 9a 3b 77 ef 49 89 44 24 
Complete report #9302
Frame # Function Binary Source or offset Line
1
ktime_get_ts64
vmlinux 0xffffffff91306d3f
Build id: 3.10.0-1127.el7.x86_64
0xaf
2
__delayacct_blkio_start
vmlinux 0xffffffff9135e5d2
Build id: 3.10.0-1127.el7.x86_64
0x22
3
io_schedule_timeout
vmlinux 0xffffffff9198545b
Build id: 3.10.0-1127.el7.x86_64
0x8b
4
io_schedule
vmlinux 0xffffffff91985518
Build id: 3.10.0-1127.el7.x86_64
0x18
5
bit_wait_io
vmlinux 0xffffffff91983ee1
Build id: 3.10.0-1127.el7.x86_64
0x11
6
__wait_on_bit
vmlinux 0xffffffff91983a07
Build id: 3.10.0-1127.el7.x86_64
0x67
7
wait_on_page_bit
vmlinux 0xffffffff913bd691
Build id: 3.10.0-1127.el7.x86_64
0x81
8
? wake_bit_function
vmlinux 0xffffffff912c7840
Build id: 3.10.0-1127.el7.x86_64
0x40
9
__filemap_fdatawait_range
vmlinux 0xffffffff913bd7c1
Build id: 3.10.0-1127.el7.x86_64
0x111
10
? do_writepages
vmlinux 0xffffffff913cb1b1
Build id: 3.10.0-1127.el7.x86_64
0x21
11
filemap_fdatawait_range
vmlinux 0xffffffff913bd854
Build id: 3.10.0-1127.el7.x86_64
0x14
12
filemap_write_and_wait_range
vmlinux 0xffffffff913bfd36
Build id: 3.10.0-1127.el7.x86_64
0x56
13
xfs_file_fsync
xfs 0xffffffffc0465e96
Build id: 3.10.0-1127.el7.x86_64
0x66
14
do_fsync
vmlinux 0xffffffff91482f07
Build id: 3.10.0-1127.el7.x86_64
0x67
15
? system_call_after_swapgs
vmlinux 0xffffffff91992e09
Build id: 3.10.0-1127.el7.x86_64
0x96
16
SyS_fdatasync
vmlinux 0xffffffff91483213
Build id: 3.10.0-1127.el7.x86_64
0x13
17
system_call_fastpath
vmlinux 0xffffffff91992ed2
Build id: 3.10.0-1127.el7.x86_64
0x25
18
? system_call_after_swapgs
vmlinux 0xffffffff91992e15
Build id: 3.10.0-1127.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 21s! [postmaster:988]
Modules linked in: vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod ppdev vmw_balloon coretemp joydev sg pcspkr parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih e1000 libata mptbase crc32c_intel serio_raw drm_panel_orientation_quirks floppy
CPU: 5 PID: 988 Comm: postmaster Tainted: G             L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 06/22/2012
task: ffff9717e94b0000 ti: ffff9717e9708000 task.ti: ffff9717e9708000
RIP: 0010:[<ffffffff91306d3f>]  [<ffffffff91306d3f>] ktime_get_ts64+0xaf/0xf0
RSP: 0018:ffff9717e970bc58  EFLAGS: 00000216
RAX: 05dcaf15db3ed3c9 RBX: ffff9713f6afccc0 RCX: 000000000034d818
RDX: 000000003255953c RSI: 0000000021e9dcdf RDI: ffffffff91e2f200
RBP: ffff9717e970bc78 R08: ffff9717fffa2bf0 R09: d206030f43000000
R10: 2df9f1723dc3d0c0 R11: ffff9714533ce380 R12: ffff9713f6afccc0
R13: ffffffffc02312e0 R14: ffff9717e970bbd8 R15: ffff9714533ce380
FS:  00007ff03d28d8c0(0000) GS:ffff9717ffd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fefdcfd2642 CR3: 000000042970c000 CR4: 00000000000007e0
Call Trace:
[<ffffffff9135e5d2>] __delayacct_blkio_start+0x22/0x30
[<ffffffff9198545b>] io_schedule_timeout+0x8b/0x130
[<ffffffff91985518>] io_schedule+0x18/0x20
[<ffffffff91983ee1>] bit_wait_io+0x11/0x50
[<ffffffff91983a07>] __wait_on_bit+0x67/0x90
[<ffffffff913bd691>] wait_on_page_bit+0x81/0xa0
[<ffffffff912c7840>] ? wake_bit_function+0x40/0x40
[<ffffffff913bd7c1>] __filemap_fdatawait_range+0x111/0x190
[<ffffffff913cb1b1>] ? do_writepages+0x21/0x50
[<ffffffff913bd854>] filemap_fdatawait_range+0x14/0x30
[<ffffffff913bfd36>] filemap_write_and_wait_range+0x56/0x90
[<ffffffffc0465e96>] xfs_file_fsync+0x66/0x1c0 [xfs]
[<ffffffff91482f07>] do_fsync+0x67/0xb0
[<ffffffff91992e09>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffff91483213>] SyS_fdatasync+0x13/0x20
[<ffffffff91992ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff91992e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 01 01 44 3b 35 bb 26 01 01 75 95 48 01 f0 49 03 0c 24 48 3d ff c9 9a 3b 76 28 31 d2 48 2d 00 ca 9a 3b 83 c2 01 48 3d ff c9 9a 3b <77> ef 49 89 44 24 08 48 01 ca 5b 49 89 14 24 41 5c 41 5d 41 5e