Complete report #832388
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_user_generic_unrolled
|
vmlinux |
0xffffffff99fab901
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x41
|
|
2
|
?
iov_iter_copy_from_user_atomic
|
vmlinux |
0xffffffff99dc86a5
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x55
|
|
3
|
iomap_write_actor
|
vmlinux |
0xffffffff99ecb6d3
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x103
|
|
4
|
|
vmlinux |
0xffffffff99ecb5d0
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x1e0
|
|
5
|
iomap_apply
|
vmlinux |
0xffffffff99ecba8a
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0xba
|
|
6
|
iomap_file_buffered_write
|
vmlinux |
0xffffffff99ecbbd1
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0xa1
|
|
7
|
|
vmlinux |
0xffffffff99ecb5d0
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x1e0
|
|
8
|
xfs_file_buffered_aio_write
|
xfs |
0xffffffffc0497f3d
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x12d
|
|
9
|
|
vmlinux |
0xffffffff9a3b7920
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x320
|
|
10
|
xfs_file_aio_write
|
xfs |
0xffffffffc049826d
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x18d
|
|
11
|
?
xfs_file_buffered_aio_write
|
xfs |
0xffffffffc04980e0
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x2d0
|
|
12
|
do_io_submit
|
vmlinux |
0xffffffff99eb5ca3
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x3e3
|
|
13
|
SyS_io_submit
|
vmlinux |
0xffffffff99eb6170
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x10
|
|
14
|
system_call_fastpath
|
vmlinux |
0xffffffff9a3c539a
Build id: 3.10.0-1160.99.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 155s! [ib_pg_flush-1:2166]
Modules linked in: binfmt_misc xt_multiport xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon snd_ens1371 aesni_intel snd_rawmidi snd_ac97_codec lrw gf128mul ac97_bus glue_helper ablk_helper cryptd snd_seq snd_seq_device snd_pcm nfit libnvdimm pcspkr joydev snd_timer snd sg soundcore i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic
pata_acpi vmwgfx drm_kms_helper syscopyarea mptspi sysfillrect sysimgblt scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih fb_sys_fops ttm crc32c_intel mptbase ata_piix drm libata serio_raw e1000 drm_panel_orientation_quirks fuse
CPU: 0 PID: 2166 Comm: ib_pg_flush-1 Kdump: loaded Tainted: G L ------------ 3.10.0-1160.99.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
task: ffff8b1cc9c5e300 ti: ffff8b117b930000 task.ti: ffff8b117b930000
RIP: 0010:[<ffffffff99fab901>] [<ffffffff99fab901>] copy_user_generic_unrolled+0x41/0xc0
RSP: 0018:ffff8b117b933ba8 EFLAGS: 00050202
RAX: 00007f6914de2000 RBX: ffff8b131e57c618 RCX: 0000000000000040
RDX: 0000000000000000 RSI: 00007f6914de2000 RDI: ffff8b12f26be000
RBP: ffff8b117b933bc0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8b117b933b38
R13: ffff8b146b115c98 R14: ffff8b146b115b48 R15: 0000000000000000
FS: 00007f67a6aee700(0000) GS:ffff8b1ccd600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c04378ff0 CR3: 0000000c090fa000 CR4: 00000000003607f0
Call Trace:
[<ffffffff99dc86a5>] ? iov_iter_copy_from_user_atomic+0x55/0xb0
[<ffffffff99ecb6d3>] iomap_write_actor+0x103/0x1f0
[<ffffffff99ecb5d0>] ? iomap_dirty_actor+0x1e0/0x1e0
[<ffffffff99ecba8a>] iomap_apply+0xba/0x160
[<ffffffff99ecbbd1>] iomap_file_buffered_write+0xa1/0xe0
[<ffffffff99ecb5d0>] ? iomap_dirty_actor+0x1e0/0x1e0
[<ffffffffc0497f3d>] xfs_file_buffered_aio_write+0x12d/0x2d0 [xfs]
[<ffffffff9a3b7920>] ? __schedule+0x320/0x680
[<ffffffffc049826d>] xfs_file_aio_write+0x18d/0x1b0 [xfs]
[<ffffffffc04980e0>] ? xfs_file_buffered_aio_write+0x2d0/0x2d0 [xfs]
[<ffffffff99eb5ca3>] do_io_submit+0x3e3/0x8a0
[<ffffffff99eb6170>] SyS_io_submit+0x10/0x20
[<ffffffff9a3c539a>] system_call_fastpath+0x25/0x2a
Code: f7 d9 29 ca 8a 06 88 07 48 ff c6 48 ff c7 ff c9 75 f2 89 d1 83 e2 3f c1 e9 06 74 4a 4c 8b 06 4c 8b 4e 08 4c 8b 56 10 4c 8b 5e 18 <4c> 89 07 4c 89 4f 08 4c 89 57 10 4c 89 5f 18 4c 8b 46 20 4c 8b
Complete report #11273
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_user_generic_unrolled
|
vmlinux |
0xffffffffa6f93d01
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x41
|
|
2
|
?
iov_iter_copy_from_user_atomic
|
vmlinux |
0xffffffffa6dbdd95
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x55
|
|
3
|
iomap_write_actor
|
vmlinux |
0xffffffffa6eba4e3
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x103
|
|
4
|
|
vmlinux |
0xffffffffa6eba3e0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x1d0
|
|
5
|
iomap_apply
|
vmlinux |
0xffffffffa6eba89a
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xba
|
|
6
|
iomap_file_buffered_write
|
vmlinux |
0xffffffffa6eba9e1
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0xa1
|
|
7
|
|
vmlinux |
0xffffffffa6eba3e0
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x1d0
|
|
8
|
xfs_file_buffered_aio_write
|
xfs |
0xffffffffc04027ed
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x12d
|
|
9
|
xfs_file_aio_write
|
xfs |
0xffffffffc0402b0d
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x18d
|
|
10
|
?
xfs_file_buffered_aio_write
|
xfs |
0xffffffffc0402980
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x2c0
|
|
11
|
do_io_submit
|
vmlinux |
0xffffffffa6ea4e33
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x3e3
|
|
12
|
SyS_io_submit
|
vmlinux |
0xffffffffa6ea5300
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x10
|
|
13
|
system_call_fastpath
|
vmlinux |
0xffffffffa7392ed2
Build id: 3.10.0-1127.19.1.el7.x86_64
|
0x25
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 24s! [mysqld:1881]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac coretemp iosf_mbi ppdev crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul sg glue_helper pcspkr ablk_helper cryptd joydev vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod vmwgfx crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm nfit ahci libnvdimm drm crct10dif_pclmul crct10dif_common libahci crc32c_intel serio_raw
ata_piix libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 1881 Comm: mysqld Kdump: loaded Tainted: G L ------------ 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9006ece920e0 ti: ffff900374f34000 task.ti: ffff900374f34000
RIP: 0010:[<ffffffffa6f93d01>] [<ffffffffa6f93d01>] copy_user_generic_unrolled+0x41/0xc0
RSP: 0018:ffff900374f37ba8 EFLAGS: 00010202
RAX: 00007f213e89e000 RBX: ffff9007134288f0 RCX: 0000000000000040
RDX: 0000000000000000 RSI: 00007f213e89e000 RDI: ffff9003b8124000
RBP: ffff900374f37bc0 R08: 30300000eea599a3 R09: e4978ce537303130
R10: 8e8de5a9a4e5acba R11: bbe5b996e6978ce5 R12: ffff900374f37b38
R13: ffff900375afa450 R14: ffff900375afa300 R15: 0000000000000000
FS: 00007f210e418700(0000) GS:ffff90097fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffec4af5c08 CR3: 0000000623c28000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa6dbdd95>] ? iov_iter_copy_from_user_atomic+0x55/0xb0
[<ffffffffa6eba4e3>] iomap_write_actor+0x103/0x1f0
[<ffffffffa6eba3e0>] ? iomap_dirty_actor+0x1d0/0x1d0
[<ffffffffa6eba89a>] iomap_apply+0xba/0x160
[<ffffffffa6eba9e1>] iomap_file_buffered_write+0xa1/0xe0
[<ffffffffa6eba3e0>] ? iomap_dirty_actor+0x1d0/0x1d0
[<ffffffffc04027ed>] xfs_file_buffered_aio_write+0x12d/0x2c0 [xfs]
[<ffffffffc0402b0d>] xfs_file_aio_write+0x18d/0x1b0 [xfs]
[<ffffffffc0402980>] ? xfs_file_buffered_aio_write+0x2c0/0x2c0 [xfs]
[<ffffffffa6ea4e33>] do_io_submit+0x3e3/0x8a0
[<ffffffffa6ea5300>] SyS_io_submit+0x10/0x20
[<ffffffffa7392ed2>] system_call_fastpath+0x25/0x2a
Code: f7 d9 29 ca 8a 06 88 07 48 ff c6 48 ff c7 ff c9 75 f2 89 d1 83 e2 3f c1 e9 06 74 4a 4c 8b 06 4c 8b 4e 08 4c 8b 56 10 4c 8b 5e 18 <4c> 89 07 4c 89 4f 08 4c 89 57 10 4c 89 5f 18 4c 8b 46 20 4c 8b
Complete report #3940
| Frame # |
Function |
Binary |
Source or offset |
Line |
|
1
|
copy_user_generic_unrolled
|
vmlinux |
0xffffffffac584761
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x41
|
|
2
|
?
iov_iter_copy_from_user_atomic
|
vmlinux |
0xffffffffac3b597d
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x5d
|
|
3
|
iomap_write_actor
|
vmlinux |
0xffffffffac4ac996
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x106
|
|
4
|
|
vmlinux |
0xffffffffac4ac890
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x80
|
|
5
|
iomap_apply
|
vmlinux |
0xffffffffac4acf67
Build id: 3.10.0-957.12.2.el7.x86_64
|
0xb7
|
|
6
|
iomap_file_buffered_write
|
vmlinux |
0xffffffffac4ad0a1
Build id: 3.10.0-957.12.2.el7.x86_64
|
0xa1
|
|
7
|
|
vmlinux |
0xffffffffac4ac890
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x80
|
|
8
|
xfs_file_buffered_aio_write
|
xfs |
0xffffffffc049ea4d
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x12d
|
|
9
|
xfs_file_aio_write
|
xfs |
0xffffffffc049ed6d
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x18d
|
|
10
|
?
xfs_file_buffered_aio_write
|
xfs |
0xffffffffc049ebe0
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x2c0
|
|
11
|
do_io_submit
|
vmlinux |
0xffffffffac497c03
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x3e3
|
|
12
|
SyS_io_submit
|
vmlinux |
0xffffffffac4980d0
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x10
|
|
13
|
system_call_fastpath
|
vmlinux |
0xffffffffac975ddb
Build id: 3.10.0-957.12.2.el7.x86_64
|
0x22
|
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 44s! [mysqld:20948]
Modules linked in: tcp_diag udp_diag inet_diag ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack binfmt_misc ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev sb_edac coretemp vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev vmw_vmci parport_pc sg parport i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_pclmul
crct10dif_common crc32c_intel serio_raw ata_generic pata_acpi vmxnet3 vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 20948 Comm: mysqld Kdump: loaded Tainted: G L ------------ 3.10.0-957.12.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9a431be230c0 ti: ffff9a5683674000 task.ti: ffff9a5683674000
RIP: 0010:[<ffffffffac584761>] [<ffffffffac584761>] copy_user_generic_unrolled+0x41/0xc0
RSP: 0018:ffff9a5683677bc0 EFLAGS: 00050202
RAX: 00007fe8eb322000 RBX: 000000000102005a RCX: 0000000000000040
RDX: 0000000000000000 RSI: 00007fe8eb322000 RDI: ffff9a59c7ad9000
RBP: ffff9a5683677bd0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff9a58a48c1958
R13: 00000000ac3b6af0 R14: ffff9a5683677b48 R15: ffff9a5a0f7b7af0
FS: 00007fe8df7fe700(0000) GS:ffff9a5a8fd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f44867db000 CR3: 00000014a0bfc000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffac3b597d>] ? iov_iter_copy_from_user_atomic+0x5d/0x90
[<ffffffffac4ac996>] iomap_write_actor+0x106/0x1d0
[<ffffffffac4ac890>] ? iomap_write_end+0x80/0x80
[<ffffffffac4acf67>] iomap_apply+0xb7/0x150
[<ffffffffac4ad0a1>] iomap_file_buffered_write+0xa1/0xe0
[<ffffffffac4ac890>] ? iomap_write_end+0x80/0x80
[<ffffffffc049ea4d>] xfs_file_buffered_aio_write+0x12d/0x2c0 [xfs]
[<ffffffffc049ed6d>] xfs_file_aio_write+0x18d/0x1b0 [xfs]
[<ffffffffc049ebe0>] ? xfs_file_buffered_aio_write+0x2c0/0x2c0 [xfs]
[<ffffffffac497c03>] do_io_submit+0x3e3/0x8a0
[<ffffffffac4980d0>] SyS_io_submit+0x10/0x20
[<ffffffffac975ddb>] system_call_fastpath+0x22/0x27
Code: f7 d9 29 ca 8a 06 88 07 48 ff c6 48 ff c7 ff c9 75 f2 89 d1 83 e2 3f c1 e9 06 74 4a 4c 8b 06 4c 8b 4e 08 4c 8b 56 10 4c 8b 5e 18 <4c> 89 07 4c 89 4f 08 4c 89 57 10 4c 89 5f 18 4c 8b 46 20 4c 8b